Healthcheck tests
^^^^^^^^^^^^^^^^^
-In Danube, healthcheck tests have been refactored and rely on SNAPS, an
+Since Danube, healthcheck tests have been refactored and rely on SNAPS, an
OPNFV middleware project.
SNAPS stands for "SDN/NFV Application development Platform and Stack".
snaps.use_keystone is True, functest must have access to the cloud's private
network. This suite consists in 49 tests (test duration < 2 minutes).
-snaps_health_check creates instance, allocate floating IP, connect to the VM.
-This test replaced the previous Colorado healthcheck test.
+Snaps_health_check creates a VM with a single port with an IPv4 address that
+is assigned by DHCP and then validates the expected IP with the actual.
Self-obviously, successful completion of the 'healthcheck' testcase is a
necessary pre-requisite for the execution of all other test Tiers.
* neutron
* nova
* quotas
+ * ceilometer
A basic SLA (stop test on errors) has been implemented.
cases. Test case 'rally_full' executes the full defined set of Rally tests.
-Refstack-client to run Defcore testcases
------------------------------------------
+Refstack-client to run OpenStack interop testcases
+--------------------------------------------------
Refstack-client `[8]`_ is a command line utility that allows you to
execute Tempest test runs based on configurations you specify.
-It is the official tool to run Defcore `[9]`_ testcases,
-which focuses on testing interoperability between OpenStack clouds.
+It is the official tool to run Openstack Interop (previously known as Defcore)
+testcases `[9]`_, which focus on testing interoperability between OpenStack
+clouds.
Refstack-client is integrated in Functest, consumed by Dovetail, which
intends to define and provide a set of OPNFV related validation criteria
that will provide input for the evaluation of the use of OPNFV trademarks.
This progress is under the guideline of Compliance Verification Program(CVP).
-Defcore testcases
-^^^^^^^^^^^^^^^^^^
-
-*Danube Release*
-
-Set of DefCore tempest test cases not flagged and required.
-According to `[10]`_, some tests are still flagged due to outstanding bugs
-in the Tempest library, particularly tests that require SSH. Refstack developers
-are working on correcting these bugs upstream. Please note that although some tests
-are flagged because of bugs, there is still an expectation that the capabilities
-covered by the tests are available. It only contains Openstack core compute
-(no object storage). The approved guidelines (2016.08) are valid for Kilo,
-Liberty, Mitaka and Newton releases of OpenStack.
-The list can be generated using the Rest API from RefStack project:
-https://refstack.openstack.org/api/v1/guidelines/2016.08/tests?target=compute&type=required&alias=true&flag=false
-
Running methods
^^^^^^^^^^^^^^^
This test case contains tests that setup and destroy environments with VMs with
and without Floating IPs with a newly created user and project. Set the config
-value snaps.use_floating_ips (True|False) to toggle this functionality. When
-the config value of snaps.use_keystone is True, Functest must have access
+value snaps.use_floating_ips (True|False) to toggle this functionality.
+Please note that When the configuration value of snaps.use_keystone is True, Functest must have access
the cloud's private network.
This suite consists in 38 tests (test duration < 10 minutes)
SDN Controllers
---------------
-There are currently 3 available controllers:
-
- * OpenDaylight (ODL)
- * ONOS
- * OpenContrail (OCL)
-
OpenDaylight
^^^^^^^^^^^^
code returned by OpenDaylight.
-ONOS
-^^^^
-
-TestON Framework is used to test the ONOS SDN controller functions.
-The test cases deal with L2 and L3 functions.
-The ONOS test suite can be run on any ONOS compliant scenario.
-
-The test cases are described as follows:
-
- * onosfunctest: The main executable file contains the initialization of
- the docker environment and functions called by FUNCvirNetNB and
- FUNCvirNetNBL3
-
- * FUNCvirNetNB
-
- * Create Network: Post Network data and check it in ONOS
- * Update Network: Update the Network and compare it in ONOS
- * Delete Network: Delete the Network and check if it's NULL in ONOS or
- not
- * Create Subnet: Post Subnet data and check it in ONOS
- * Update Subnet: Update the Subnet and compare it in ONOS
- * Delete Subnet: Delete the Subnet and check if it's NULL in ONOS or not
- * Create Port: Post Port data and check it in ONOS
- * Update Port: Update the Port and compare it in ONOS
- * Delete Port: Delete the Port and check if it's NULL in ONOS or not
-
- * FUNCvirNetNBL3
-
- * Create Router: Post data for create Router and check it in ONOS
- * Update Router: Update the Router and compare it in ONOS
- * Delete Router: Delete the Router data and check it in ONOS
- * Create RouterInterface: Post Router Interface data to an existing Router
- and check it in ONOS
- * Delete RouterInterface: Delete the RouterInterface and check the Router
- * Create FloatingIp: Post data for create FloatingIp and check it in ONOS
- * Update FloatingIp: Update the FloatingIp and compare it in ONOS
- * Delete FloatingIp: Delete the FloatingIp and check that it is 'NULL' in
- ONOS
- * Create External Gateway: Post data to create an External Gateway for an
- existing Router and check it in ONOS
- * Update External Gateway: Update the External Gateway and compare the change
- * Delete External Gateway: Delete the External Gateway and check that it is
- 'NULL' in ONOS
-
-
Features
--------
+-----------------+---------+----------+--------+-----------+
| fds | | | X | X |
+-----------------+---------+----------+--------+-----------+
-| moon | | X | | X |
+| moon | | X | | |
+-----------------+---------+----------+--------+-----------+
| multisite | | X | X | |
+-----------------+---------+----------+--------+-----------+
+-----------------+---------+----------+--------+-----------+
| orchestra | | | X | X |
+-----------------+---------+----------+--------+-----------+
-| parser | | | X | |
+| parser | | | X | X |
+-----------------+---------+----------+--------+-----------+
| promise | X | X | X | X |
+-----------------+---------+----------+--------+-----------+
:align: center
:alt: vIMS architecture
+
+cloudify_ims_perf
+^^^^^^^^^^^^^^^^^
+This testcase extends the cloudify_ims test case.
+The first part is similar but the testing part is different.
+The testing part consists in automating a realistic signaling load on the vIMS
+using an Ixia loader (proprietary tools)
+ - You need to have access to an Ixia licence server defined in the configuration
+ file and have ixia image locally.
+
+This test case is available but not declared in testcases.yaml. The declaration
+of the testcase is simple, connect to your functest-vnf docker, add the following
+section in /usr/lib/python2.7/site-packacges/functest/ci/testcases.yaml::
+
+ -
+ case_name: cloudify_ims_perf
+ project_name: functest
+ criteria: 80
+ blocking: false
+ description: >-
+ Stress tests based on Cloudify. Ixia loader images and access to Ixia
+ server license.
+ dependencies:
+ installer: ''
+ scenario: 'os-nosdn-nofeature-ha'
+ run:
+ module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf'
+ class: 'CloudifyImsPerf'
+
orchestra_openims
^^^^^^^^^^^^^^^^^
Orchestra test case deals with the deployment of OpenIMS with OpenBaton
Orchestra test case deals with the deployment of Clearwater vIMS with OpenBaton
orchestrator.
-parser
-^^^^^^
-
-See parser user guide for details: `[12]`_
-
-
vyos-vrouter
^^^^^^^^^^^^
-
This test case deals with the deployment and the test of vyos vrouter with
Cloudify orchestrator. The test case can do testing for interchangeability of
BGP Protocol using vyos.
The vyos-vrouter architecture is described in `[14]`_
+
.. _`[2]`: http://docs.openstack.org/developer/tempest/overview.html
.. _`[3]`: https://rally.readthedocs.org/en/latest/index.html
.. _`[5]`: https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater/blob/master/openstack-blueprint.yaml
.. _`[8]`: https://github.com/openstack/refstack-client
.. _`[10]`: https://github.com/openstack/interop/blob/master/2016.08/procedure.rst
.. _`[11]`: http://robotframework.org/
-.. _`[12]`: http://artifacts.opnfv.org/parser/colorado/docs/userguide/index.html
+.. _`[12]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html
.. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO
.. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter
.. _`[15]`: https://aptira.com/testing-openstack-tempest-part-1/
Code Review / functest.git / blobdiff