X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=blobdiff_plain;f=docs%2Ftesting%2Fuser%2Fuserguide%2Ftest_details.rst;h=992b546f535e788978ec43d09bec9908225b9f98;hb=refs%2Fchanges%2F91%2F59491%2F6;hp=935298fddcac47c877909d1409160271c12ac46b;hpb=7e208b21593d98b9b9600c7616b74f4cf4b3cfbf;p=functest.git diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst index 935298fdd..992b546f5 100644 --- a/docs/testing/user/userguide/test_details.rst +++ b/docs/testing/user/userguide/test_details.rst @@ -38,6 +38,11 @@ network. This suite consists in 49 tests (test duration < 2 minutes). Snaps_health_check creates a VM with a single port with an IPv4 address that is assigned by DHCP and then validates the expected IP with the actual. +The flavors for the SNAPS test cases are able to be configured giving new +metadata values as well as new values for the basic elements of flavor (i.e. +ram, vcpu, disk, ephemeral, swap etc). The snaps.flavor_extra_specs dict in the +config_functest.yaml file could be used for this purpose. + Self-obviously, successful completion of the 'healthcheck' testcase is a necessary pre-requisite for the execution of all other test Tiers. @@ -161,17 +166,14 @@ console output is stored to a *log* file for further analysis. The Tempest testcases are distributed across three Tiers: - * Smoke Tier - Test Case 'tempest_smoke_serial' - * Components Tier - Test case 'tempest_full_parallel' + * Smoke Tier - Test Case 'tempest_smoke' + * Components Tier - Test case 'tempest_full' * Neutron Trunk Port - Test case 'neutron_trunk' * OpenStack interop testcases - Test case 'refstack_defcore' * Testing and verifying RBAC policy enforcement - Test case 'patrole' -NOTE: Test case 'tempest_smoke_serial' executes a defined set of tempest smoke -tests with a single thread (i.e. serial mode). Test case -'tempest_full_parallel' executes all defined Tempest tests using several -concurrent threads (i.e. parallel mode). The number of threads activated -corresponds to the number of available logical CPUs. +NOTE: Test case 'tempest_smoke' executes a defined set of tempest smoke +tests. Test case 'tempest_full' executes all defined Tempest tests. NOTE: The 'neutron_trunk' test set allows to connect a VM to multiple VLAN separated networks using a single NIC. The feature neutron trunk ports have @@ -236,6 +238,10 @@ Please note that When the configuration value of snaps.use_keystone is True, Functest must have access the cloud's private network. This suite consists in 120 tests (test duration ~= 50 minutes) +The flavors for the SNAPS test cases are able to be configured giving new +metadata values as well as new values for the basic elements of flavor (i.e. +ram, vcpu, disk, ephemeral, swap etc). The snaps.flavor_extra_specs dict in +the config_functest.yaml file could be used for this purpose. SDN Controllers --------------- @@ -302,39 +308,43 @@ Features Functest has been supporting several feature projects since Brahmaputra: -+-----------------+---------+----------+--------+-----------+ -| Test | Brahma | Colorado | Danube | Euphrates | -+=================+=========+==========+========+===========+ -| barometer | | | X | X | -+-----------------+---------+----------+--------+-----------+ -| bgpvpn | | X | X | X | -+-----------------+---------+----------+--------+-----------+ -| copper | | X | | | -+-----------------+---------+----------+--------+-----------+ -| doctor | X | X | X | X | -+-----------------+---------+----------+--------+-----------+ -| domino | | X | X | X | -+-----------------+---------+----------+--------+-----------+ -| fds | | | X | X | -+-----------------+---------+----------+--------+-----------+ -| moon | | X | | | -+-----------------+---------+----------+--------+-----------+ -| multisite | | X | X | | -+-----------------+---------+----------+--------+-----------+ -| netready | | | X | | -+-----------------+---------+----------+--------+-----------+ -| odl_sfc | | X | X | X | -+-----------------+---------+----------+--------+-----------+ -| opera | | | X | | -+-----------------+---------+----------+--------+-----------+ -| orchestra | | | X | X | -+-----------------+---------+----------+--------+-----------+ -| parser | | | X | X | -+-----------------+---------+----------+--------+-----------+ -| promise | X | X | X | X | -+-----------------+---------+----------+--------+-----------+ -| security_scan | | X | X | | -+-----------------+---------+----------+--------+-----------+ ++-----------------+---------+----------+--------+-----------+-----------+ +| Test | Brahma | Colorado | Danube | Euphrates | Fraser | ++=================+=========+==========+========+===========+===========+ +| barometer | | | X | X | X | ++-----------------+---------+----------+--------+-----------+-----------+ +| bgpvpn | | X | X | X | X | ++-----------------+---------+----------+--------+-----------+-----------+ +| copper | | X | | | | ++-----------------+---------+----------+--------+-----------+-----------+ +| doctor | X | X | X | X | X | ++-----------------+---------+----------+--------+-----------+-----------+ +| domino | | X | X | X | | ++-----------------+---------+----------+--------+-----------+-----------+ +| fds | | | X | X | X | ++-----------------+---------+----------+--------+-----------+-----------+ +| moon | | X | | | | ++-----------------+---------+----------+--------+-----------+-----------+ +| multisite | | X | X | | | ++-----------------+---------+----------+--------+-----------+-----------+ +| netready | | | X | | | ++-----------------+---------+----------+--------+-----------+-----------+ +| odl_sfc | | X | X | X | X | ++-----------------+---------+----------+--------+-----------+-----------+ +| opera | | | X | | | ++-----------------+---------+----------+--------+-----------+-----------+ +| orchestra | | | X | X | X | ++-----------------+---------+----------+--------+-----------+-----------+ +| parser | | | X | X | X | ++-----------------+---------+----------+--------+-----------+-----------+ +| promise | X | X | X | X | X | ++-----------------+---------+----------+--------+-----------+-----------+ +| security_scan | | X | X | | | ++-----------------+---------+----------+--------+-----------+-----------+ +| clover | | | | | X | ++-----------------+---------+----------+--------+-----------+-----------+ +| stor4nfv | | | | | X | ++-----------------+---------+----------+--------+-----------+-----------+ Please refer to the dedicated feature user guides for details. @@ -362,41 +372,11 @@ The goal of this test suite consists of: The Clearwater architecture is described as follows: -.. figure:: ../../../images/clearwater-architecture.png +.. figure:: ../../../images/clearwater-architecture-v2.png :align: center :alt: vIMS architecture -cloudify_ims_perf -^^^^^^^^^^^^^^^^^ -This testcase extends the cloudify_ims test case. -The first part is similar but the testing part is different. -The testing part consists in automating a realistic signaling load on the vIMS -using an Ixia loader (proprietary tools) - - - You need to have access to an Ixia licence server defined in the - configuration file and have ixia image locally. - -This test case is available but not declared in testcases.yaml. The declaration -of the testcase is simple, connect to your functest-vnf docker, add the -following section in -/usr/lib/python2.7/site-packacges/functest/ci/testcases.yaml:: - - - - case_name: cloudify_ims_perf - project_name: functest - criteria: 80 - blocking: false - description: >- - Stress tests based on Cloudify. Ixia loader images and access to Ixia - server license. - dependencies: - installer: '' - scenario: 'os-nosdn-nofeature-ha' - run: - module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf' - class: 'CloudifyImsPerf' - vyos-vrouter ^^^^^^^^^^^^ This test case deals with the deployment and the test of vyos vrouter with @@ -418,13 +398,41 @@ The vyos-vrouter architecture is described in `[14]`_ juju_epc ^^^^^^^^ +The Evolved Packet Core (EPC) is the main component of the System Architecture +Evolution (SAE) which forms the core of the 3GPP LTE specification. + +vEPC has been integrated in Functest to demonstrate the capability to deploy a +complex mobility-specific NFV scenario on the OPNFV platform. The OAI EPC +supports most of the essential functions defined by the 3GPP Technical Specs; +hence the successful execution of functional tests on the OAI EPC provides a +good endorsement of the underlying NFV platform. + +This integration also includes ABot, a Test Orchestration system that enables +test scenarios to be defined in high-level DSL. ABot is also deployed as a +VM on the OPNFV platform; and this provides an example of the automation +driver and the Test VNF being both deployed as separate VNFs on the underlying +OPNFV platform. + +The Workflow is as follows: + * Deploy Orchestrator + Deploy Juju controller using Bootstrap command. + * Deploy VNF + Deploy ABot orchestrator and OAI EPC as Juju charms. + Configuration of ABot and OAI EPC components is handled through + built-in Juju relations. + * Test VNF + Execution of ABot feature files triggered by Juju actions. + This executes a suite of LTE signalling tests on the OAI EPC. + * Reporting + ABot test results are parsed accordingly and pushed to Functest Db. +Details of the ABot test orchestration tool may be found in `[15]`_ Kubernetes (K8s) ---------------- Kubernetes testing relies on sets of tests, which are part of the Kubernetes -source tree, such as the Kubernetes End-to-End (e2e) tests `[15]`_. +source tree, such as the Kubernetes End-to-End (e2e) tests `[16]`_. The kubernetes testcases are distributed across various Tiers: @@ -456,4 +464,5 @@ The kubernetes testcases are distributed across various Tiers: .. _`[12]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html .. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO .. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter -.. _`[15]`: https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-tests.md +.. _`[15]`: https://www.rebaca.com/abot-test-orchestration-tool/ +.. _`[16]`: https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-tests.md