installation to validate and verify the basic functions of the
infrastructure.
-The current list of test suites can be distributed over 4 main domains: VIM
+The current list of test suites can be distributed over 5 main domains: VIM
(Virtualised Infrastructure Manager), Controllers (i.e. SDN Controllers),
-Features and VNF (Virtual Network Functions).
+Features, VNF (Virtual Network Functions) and MANO stacks.
+
+Functest test suites are also distributed in the OPNFV testing categories:
+healthcheck, smoke, features, components, performance, VNF, Stress tests.
+
+All the Healthcheck and smoke tests of a given scenario must be succesful to
+validate the scenario for the release.
+-------------+---------------+----------------+----------------------------------+
| Domain | Tier | Test case | Comments |
+=============+===============+================+==================================+
| VIM | healthcheck | healthcheck | Verify basic operation in VIM |
+| | +----------------+----------------------------------+
+| | | connection | Check OpenStack connectivity |
+| | | _check | through SNAPS framework |
+| | +----------------+----------------------------------+
+| | | api_check | Check OpenStack API through |
+| | | | SNAPS framework |
| +---------------+----------------+----------------------------------+
| | smoke | vPing_SSH | NFV "Hello World" using an SSH |
| | | | connection to a destination VM |
| | +----------------+----------------------------------+
| | | rally_sanity | Run a subset of the OpenStack |
| | | | Rally Test Suite in smoke mode |
+| | +----------------+----------------------------------+
+| | | snaps_smoke | Run a subset of the OpenStack |
+| | | | Rally Test Suite in smoke mode |
| +---------------+----------------+----------------------------------+
-| | openstack | tempest_full | Generate and run a full set of |
+| | components | tempest_full | Generate and run a full set of |
| | | \_parallel | the OpenStack Tempest Test Suite.|
| | | | See the OpenStack reference test |
| | | | suite `[2]`_. The generated |
| | | | benchmarking OpenStack modules |
| | | | See the Rally documents `[3]`_. |
+-------------+---------------+----------------+----------------------------------+
-| Controllers | sdn_suites | odl | Opendaylight Test suite |
+| Controllers | smoke | odl | Opendaylight Test suite |
| | | | Limited test suite to check the |
| | | | basic neutron (Layer 2) |
| | | | operations mainly based on |
| | | | See `ONOSFW User Guide`_ for |
| | | | details. |
+-------------+---------------+----------------+----------------------------------+
-| Features | features | Promise | Resource reservation and |
+| Features | features | promise | Resource reservation and |
| | | | management project to identify |
| | | | NFV related requirements and |
| | | | realize resource reservation for |
| | | | See `Promise User Guide`_ for |
| | | | details. |
| | +----------------+----------------------------------+
-| | | Doctor | Doctor platform, as of Colorado |
+| | | doctor | Doctor platform, as of Colorado |
| | | | release, provides the three |
| | | | features: |
| | | | * Immediate Notification |
| | | | installer environment) |
| | +----------------+----------------------------------+
| | | onos-sfc | SFC testing for onos scenarios |
-| | | | TODO See for details |
+| | | | See `ONOSFW User Guide`_ for |
+| | | | details |
| | +----------------+----------------------------------+
| | | odl-sfc | SFC testing for odl scenarios |
-| | | | TODO See for details |
+| | | | See `SFC User Guide`_ for details|
| | +----------------+----------------------------------+
| | | domino | Domino provides TOSCA template |
| | | | distribution service for network |
| | | | See `Domino User Guide`_ for |
| | | | details |
| | +----------------+----------------------------------+
-| | | Copper | Copper develops OPNFV platform |
+| | | copper | Copper develops OPNFV platform |
| | | | support for policy management, |
| | | | using open source projects such |
| | | | as OpenStack Congress, focused |
| | | | in the `Copper User Guide`_. |
| | +----------------+----------------------------------+
| | | multisites | Multisites |
-| | | | TODO See for details |
+| | | | See `Multisite User Guide`_ for |
+| | | | details |
| | +----------------+----------------------------------+
| | | moon | Security management system |
-| | | | TODO See for details |
+| | | | See `Moon User Guide`_ for |
+| | | | details |
+-------------+---------------+----------------+----------------------------------+
-| VNF | vnf | vims | Example of a real VNF deployment |
+| VNF | vnf | cloudify_ims | Example of a real VNF deployment |
| | | | to show the NFV capabilities of |
| | | | the platform. The IP Multimedia |
| | | | Subsytem is a typical Telco test |
| | | | case, referenced by ETSI. |
| | | | It provides a fully functional |
| | | | VoIP System |
+| | +----------------+----------------------------------+
+| | | opera_ims | vIMS deployment using openBaton |
+| | +----------------+----------------------------------+
+| | | orchestra_ims | vIMS deployment using open-O |
+ +---------------+----------------+----------------------------------+
| | | parser | Parser is an integration project |
| | | | which aims to provide |
'healthcheck' Tier testcase fails, or if there are any failures in the 'smoke'
Tier testcases, there is little point to launch a full testcase execution round.
+In Danube, we merged smoke and sdn controller tiers in smoke tier.
+
An overview of the Functest Structural Concept is depicted graphically below:
.. figure:: ../images/concepts_mapping_final.png
sources and scenarios and to show them on a `Functest Dashboard`_. A screenshot
of a live Functest Dashboard is shown below:
-.. figure:: ../images/FunctestDashboard.png
+** TODO **
+.. figure:: ../images/FunctestDashboardDanube.png
:align: center
:alt: Functest Dashboard
-There is no real notion of Test domain or Test coverage. Basic components
-(VIM, SDN controllers) are tested through their own suites. Feature projects
-also provide their own test suites with different ways of running their tests.
+Basic components (VIM, SDN controllers) are tested through their own suites.
+Feature projects also provide their own test suites with different ways of
+running their tests.
+
+The notion of domain has been introduced in the description of the test cases
+stored in the Database.
+This parameters as well as possible tags can be used for the Test case catalog.
vIMS test case was integrated to demonstrate the capability to deploy a
relatively complex NFV scenario on top of the OPNFV infrastructure.
-Functest considers OPNFV as a black box. As of Colorado release the OPNFV
+Functest considers OPNFV as a black box. As of Danube release the OPNFV
offers a lot of potential combinations:
* 3 controllers (OpenDaylight, ONOS, OpenContrail)
DEPLOY_SCENARIO) to automatically determine the valid test cases, for each given
environment.
-In the Colorado OPNFV System release a convenience Functest CLI utility is also
-introduced to simplify setting up the Functest evironment, management of the
-OpenStack environment (e.g. resource clean-up) and for executing tests.
+A convenience Functest CLI utility is also available to simplify setting up the
+Functest evironment, management of the OpenStack environment (e.g. resource
+clean-up) and for executing tests.
The Functest CLI organised the testcase into logical Tiers, which contain in
turn one or more testcases. The CLI allows execution of a single specified
testcase, all test cases in a specified Tier, or the special case of execution
.. _`Copper User Guide`: http://artifacts.opnfv.org/copper/colorado/docs/userguide/index.html
.. _`Doctor User Guide`: http://artifacts.opnfv.org/doctor/colorado/userguide/index.html
.. _`Promise User Guide`: http://artifacts.opnfv.org/promise/colorado/docs/userguide/index.html
-.. _`ONOSFW User Guide`: http://artifacts.opnfv.org/onosfw/brahmaputra/docs/userguide/index.html
-.. _`SDNVPN User Guide`: http://artifacts.opnfv.org/sdnvpn/brahmaputra/docs/userguide/featureusage.html
-.. _`Domino User Guide`: http://artifacts.opnfv.org/domino/docs/userguide/index.html
-.. _`Parser User Guide`: http://artifacts.opnfv.org/parser/docs/userguide/index.html
+.. _`ONOSFW User Guide`: http://artifacts.opnfv.org/onosfw/colorado/userguide/index.html
+.. _`SDNVPN User Guide`: http://artifacts.opnfv.org/sdnvpn/colorado/docs/userguide/index.html
+.. _`Domino User Guide`: http://artifacts.opnfv.org/domino/docs/userguide-single/index.html
+.. _`Parser User Guide`: http://artifacts.opnfv.org/parser/colorado/docs/userguide/index.html
.. _`Functest Dashboard`: http://testresults.opnfv.org/kibana_dashboards/
+.. _`SFC User Guide`: http://artifacts.opnfv.org/sfc/colorado/userguide/index.html
+.. _`Multisite User Guide`: http://artifacts.opnfv.org/multisite/docs/userguide/index.html
+.. _`Moon User Guide`: http://artifacts.opnfv.org/moon/docs/userguide/index.html