X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=blobdiff_plain;f=docs%2Ftesting%2Fuser%2Fuserguide%2Ftest_details.rst;h=43a75b73d97d1e6b148bb81c8594e09b477dc63b;hb=8e1f8d7b761102c17d7b9233b24906d3ac33085c;hp=f23abc5090853f4cce78bf456c3db6ce2f74a3e8;hpb=d20297e509ad127b237a90ba5cedbc2500bae753;p=functest.git diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst index f23abc509..43a75b73d 100644 --- a/docs/testing/user/userguide/test_details.rst +++ b/docs/testing/user/userguide/test_details.rst @@ -9,7 +9,7 @@ VIM (Virtualized Infrastructure Manager) Healthcheck tests ^^^^^^^^^^^^^^^^^ -In Danube, healthcheck tests have been refactored and rely on SNAPS, an +Since Danube, healthcheck tests have been refactored and rely on SNAPS, an OPNFV middleware project. SNAPS stands for "SDN/NFV Application development Platform and Stack". @@ -36,8 +36,8 @@ Neutron and Nova and may perform some simple queries. When the config value of snaps.use_keystone is True, functest must have access to the cloud's private network. This suite consists in 49 tests (test duration < 2 minutes). -snaps_health_check creates instance, allocate floating IP, connect to the VM. -This test replaced the previous Colorado healthcheck test. +Snaps_health_check creates a VM with a single port with an IPv4 address that +is assigned by DHCP and then validates the expected IP with the actual. Self-obviously, successful completion of the 'healthcheck' testcase is a necessary pre-requisite for the execution of all other test Tiers. @@ -195,6 +195,7 @@ The OPNFV Rally scenarios are based on the collection of the actual Rally scenar * neutron * nova * quotas + * ceilometer A basic SLA (stop test on errors) has been implemented. @@ -207,35 +208,20 @@ NOTE: Test case 'rally_sanity' executes a limited number of Rally smoke test cases. Test case 'rally_full' executes the full defined set of Rally tests. -Refstack-client to run Defcore testcases ------------------------------------------ +Refstack-client to run OpenStack interop testcases +-------------------------------------------------- Refstack-client `[8]`_ is a command line utility that allows you to execute Tempest test runs based on configurations you specify. -It is the official tool to run Defcore `[9]`_ testcases, -which focuses on testing interoperability between OpenStack clouds. +It is the official tool to run Openstack Interop (previously known as Defcore) +testcases `[9]`_, which focus on testing interoperability between OpenStack +clouds. Refstack-client is integrated in Functest, consumed by Dovetail, which intends to define and provide a set of OPNFV related validation criteria that will provide input for the evaluation of the use of OPNFV trademarks. This progress is under the guideline of Compliance Verification Program(CVP). -Defcore testcases -^^^^^^^^^^^^^^^^^^ - -*Danube Release* - -Set of DefCore tempest test cases not flagged and required. -According to `[10]`_, some tests are still flagged due to outstanding bugs -in the Tempest library, particularly tests that require SSH. Refstack developers -are working on correcting these bugs upstream. Please note that although some tests -are flagged because of bugs, there is still an expectation that the capabilities -covered by the tests are available. It only contains Openstack core compute -(no object storage). The approved guidelines (2016.08) are valid for Kilo, -Liberty, Mitaka and Newton releases of OpenStack. -The list can be generated using the Rest API from RefStack project: -https://refstack.openstack.org/api/v1/guidelines/2016.08/tests?target=compute&type=required&alias=true&flag=false - Running methods ^^^^^^^^^^^^^^^ @@ -314,8 +300,8 @@ snaps_smoke This test case contains tests that setup and destroy environments with VMs with and without Floating IPs with a newly created user and project. Set the config -value snaps.use_floating_ips (True|False) to toggle this functionality. When -the config value of snaps.use_keystone is True, Functest must have access +value snaps.use_floating_ips (True|False) to toggle this functionality. +Please note that When the configuration value of snaps.use_keystone is True, Functest must have access the cloud's private network. This suite consists in 38 tests (test duration < 10 minutes) @@ -323,12 +309,6 @@ This suite consists in 38 tests (test duration < 10 minutes) SDN Controllers --------------- -There are currently 3 available controllers: - - * OpenDaylight (ODL) - * ONOS - * OpenContrail (OCL) - OpenDaylight ^^^^^^^^^^^^ @@ -397,7 +377,7 @@ Functest has been supporting several feature projects since Brahpamutra: +-----------------+---------+----------+--------+-----------+ | fds | | | X | X | +-----------------+---------+----------+--------+-----------+ -| moon | | X | | X | +| moon | | X | | | +-----------------+---------+----------+--------+-----------+ | multisite | | X | X | | +-----------------+---------+----------+--------+-----------+ @@ -409,7 +389,7 @@ Functest has been supporting several feature projects since Brahpamutra: +-----------------+---------+----------+--------+-----------+ | orchestra | | | X | X | +-----------------+---------+----------+--------+-----------+ -| parser | | | X | | +| parser | | | X | X | +-----------------+---------+----------+--------+-----------+ | promise | X | X | X | X | +-----------------+---------+----------+--------+-----------+ @@ -446,6 +426,35 @@ The Clearwater architecture is described as follows: :align: center :alt: vIMS architecture + +cloudify_ims_perf +^^^^^^^^^^^^^^^^^ +This testcase extends the cloudify_ims test case. +The first part is similar but the testing part is different. +The testing part consists in automating a realistic signaling load on the vIMS +using an Ixia loader (proprietary tools) + - You need to have access to an Ixia licence server defined in the configuration + file and have ixia image locally. + +This test case is available but not declared in testcases.yaml. The declaration +of the testcase is simple, connect to your functest-vnf docker, add the following +section in /usr/lib/python2.7/site-packacges/functest/ci/testcases.yaml:: + + - + case_name: cloudify_ims_perf + project_name: functest + criteria: 80 + blocking: false + description: >- + Stress tests based on Cloudify. Ixia loader images and access to Ixia + server license. + dependencies: + installer: '' + scenario: 'os-nosdn-nofeature-ha' + run: + module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf' + class: 'CloudifyImsPerf' + orchestra_openims ^^^^^^^^^^^^^^^^^ Orchestra test case deals with the deployment of OpenIMS with OpenBaton @@ -456,15 +465,8 @@ orchestra_clearwaterims Orchestra test case deals with the deployment of Clearwater vIMS with OpenBaton orchestrator. -parser -^^^^^^ - -See parser user guide for details: `[12]`_ - - vyos-vrouter ^^^^^^^^^^^^ - This test case deals with the deployment and the test of vyos vrouter with Cloudify orchestrator. The test case can do testing for interchangeability of BGP Protocol using vyos. @@ -482,13 +484,14 @@ The Workflow is as follows: The vyos-vrouter architecture is described in `[14]`_ + .. _`[2]`: http://docs.openstack.org/developer/tempest/overview.html .. _`[3]`: https://rally.readthedocs.org/en/latest/index.html .. _`[5]`: https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater/blob/master/openstack-blueprint.yaml .. _`[8]`: https://github.com/openstack/refstack-client .. _`[10]`: https://github.com/openstack/interop/blob/master/2016.08/procedure.rst .. _`[11]`: http://robotframework.org/ -.. _`[12]`: http://artifacts.opnfv.org/parser/colorado/docs/userguide/index.html +.. _`[12]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html .. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO .. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter .. _`[15]`: https://aptira.com/testing-openstack-tempest-part-1/