+----------------+---------------------------------------------------------+
| Project | Description |
+================+=========================================================+
-| Bottlenecks | This project aims to find system bottlenecks by testing |
+| Bottlenecks | This project aims to find system bottlenecks by testing |
| | and verifying OPNFV infrastructure in a staging |
| | environment before committing it to a production |
| | environment. Instead of debugging a deployment in |
| | to run those tools on OPNFV's infrastructure. |
+----------------+---------------------------------------------------------+
| Dovetail | This project intends to define and provide a set of |
-| | OPNFV related validation criteria that will provide |
-| | input for the evaluation of the use of OPNFV trademarks.|
-| | The dovetail project is executed with the guidance and |
-| | oversight of the Compliance and Certification committee |
-| | and work to secure the goals of the C&C committee for |
-| | each release. The project intends to incrementally |
-| | define qualification criteria that establish the |
-| | foundations of how we are able to measure the ability to|
-| | utilize the OPNFV platform, how the platform itself |
-| | should behave, and how applications may be deployed on |
-| | the platform. |
+| | OPNFV related validation criteria/tests that will |
+| | provide input for the OPNFV Complaince Verification |
+| | Program. The Dovetail project is executed with the |
+| | guidance and oversight of the Complaince and |
+| | Certification (C&C) committee and work to secure the |
+| | goals of the C&C committee for each release. The |
+| | project intends to incrementally define qualification |
+| | criteria that establish the foundations of how one is |
+| | able to measure the ability to utilize the OPNFV |
+| | platform, how the platform itself should behave, and |
+| | how applications may be deployed on the platform. |
+----------------+---------------------------------------------------------+
| Functest | This project deals with the functional testing of the |
| | VIM and NFVI. It leverages several upstream test suites |
| | project to launch feature test suites in CI/CD. |
| | The project is used for scenario validation. |
+----------------+---------------------------------------------------------+
-| Qtip | QTIP as the project for "Platform Performance |
+| NFVbench | NFVbench is a compact and self contained data plane |
+| | performance measurement tool for OpensStack based NFVi |
+| | platforms. It is agnostic of the NFVi distribution, |
+| | Neutron networking implementation and hardware. |
+| | It runs on any Linux server with a DPDK compliant |
+| | NIC connected to the NFVi platform data plane and |
+| | bundles a highly efficient software traffic generator. |
+| | Provides a fully automated measurement of most common |
+| | packet paths at any level of scale and load using |
+| | RFC-2544. Available as a Docker container with simple |
+| | command line and REST interfaces. |
+| | Easy to use as it takes care of most of the guesswork |
+| | generally associated to data plane benchmarking. |
+| | Can run in any lab or in production environments. |
++----------------+---------------------------------------------------------+
+| QTIP | QTIP as the project for "Platform Performance |
| | Benchmarking" in OPNFV aims to provide user a simple |
| | indicator for performance, supported by comprehensive |
| | testing data and transparent calculation formula. |
| | performance benchmarking which helps users to build |
| | indicators by themselves with ease. |
+----------------+---------------------------------------------------------+
-| Storperf | The purpose of this project is to provide a tool to |
+| StorPerf | The purpose of this project is to provide a tool to |
| | measure block and object storage performance in an NFVI.|
| | When complemented with a characterization of typical VF |
| | storage performance requirements, it can provide |
The following collections are declared in this database:
* pods: the list of pods used for production CI
* projects: the list of projects providing test cases
- * testcases: the test cases related to a given project
+ * test cases: the test cases related to a given project
* results: the results of the test cases
* scenarios: the OPNFV scenarios tested in CI
This database can be used by any project through the Test API.
Please note that projects may also use additional databases. The Test
Database is mainly use to collect CI test results and generate scenario
-trust indicators. The Test Database is cloned for OPNFV Plugfests in
+trust indicators. The Test Database is also cloned for OPNFV Plugfests in
order to provide a private datastore only accessible to Plugfest participants.
Test API description
--------------------
The Test API is used to declare pods, projects, test cases and test results.
-Pods correspond to the cluster of machines (3 controller and 2 compute nodes in
-HA mode) used to run the tests and defined in Pharos project.
+Pods correspond to a cluster of machines (3 controller and 2 compute nodes in
+HA mode) used to run the tests and are defined in the Pharos project.
The results pushed in the database are related to pods, projects and test cases.
Trying to push results generated from a non-referenced pod will return an error
message by the Test API.
-An additional method dashboard has been added to post-process the raw results in
-the Brahmaputra release (deprecated in Colorado release).
-
The data model is very basic, 5 objects are available:
* Pods
* Projects
- * Testcases
+ * Test cases
* Results
* Scenarios
to be added in the request using a header 'X-Auth-Token' for access to the database.
e.g::
+
headers['X-Auth-Token']
The value of the header i.e the token can be accessed in the jenkins environment variable
headers['X-Auth-Token'] = os.environ.get('TestApiToken')
The above example is in Python. Token based authentication has been added so
-that only CI pods running Jenkins jobs can access to the database. Please note
+that only CI pods running Jenkins jobs can access the database. Please note
that currently token authorization is implemented but is not yet enabled.
:alt: Testing group Euphrates reporting page
An evolution of the reporting page is planned to unify test reporting by creating
-a landing page that shows the scenario status with one glance (this information was
+a landing page that shows the scenario status in one glance (this information was
previously consolidated manually on a wiki page). The landing page will be displayed
per scenario and show:
with project_name: bottlenecks, functest, qtip, storperf, vsperf, yardstick
A test case catalog has now been realized `[TST4]`_. Roll over the project then
-click to get the list of test cases, click on the case to get more details.
+click to get the list of test cases, and then click on the case to get more details.
.. figure:: ../../images/TestcaseCatalog.png
:align: center
The Test Dashboard is used to provide a consistent view of the results collected in CI.
The results shown on the dashboard are post processed from the Database, which only
contains raw results.
-The dashboard can be used in addition of the reporting page (high level view) to allow
+The dashboard can be used in addition to the reporting page (high level view) to allow
the creation of specific graphs according to what the test owner wants to show.
In Brahmaputra, a basic dashboard was created in Functest.
:alt: Testing group testcase catalog
+.. include:: ./energy-monitoring.rst
+
+
OPNFV Test Group Information
============================
mailing list: test-wg@lists.opnfv.org
-IRC chan: #opnfv-testperf
+IRC channel: #opnfv-testperf
weekly meeting (https://wiki.opnfv.org/display/meetings/TestPerf):
* Usual time: Every Thursday 15:00-16:00 UTC / 7:00-8:00 PST
+----------------+---------------------------------------------------------+
| Project | Documentation links |
+================+=========================================================+
-| Bottlenecks | https://wiki.opnfv.org/display/bottlenecks/Bottlenecks |
+| Bottlenecks | https://wiki.opnfv.org/display/bottlenecks/Bottlenecks |
+----------------+---------------------------------------------------------+
| CPerf | https://wiki.opnfv.org/display/cperf |
+----------------+---------------------------------------------------------+
+----------------+---------------------------------------------------------+
| Functest | https://wiki.opnfv.org/display/functest/ |
+----------------+---------------------------------------------------------+
-| Qtip | https://wiki.opnfv.org/display/qtip |
+| NFVbench | https://wiki.opnfv.org/display/nfvbench/ |
++----------------+---------------------------------------------------------+
+| QTIP | https://wiki.opnfv.org/display/qtip |
+----------------+---------------------------------------------------------+
-| Storperf | https://wiki.opnfv.org/display/storperf/Storperf |
+| StorPerf | https://wiki.opnfv.org/display/storperf/Storperf |
+----------------+---------------------------------------------------------+
-| VSperf | https://wiki.opnfv.org/display/vsperf |
+| VSPERF | https://wiki.opnfv.org/display/vsperf |
+----------------+---------------------------------------------------------+
| Yardstick | https://wiki.opnfv.org/display/yardstick/Yardstick |
+----------------+---------------------------------------------------------+
Code Review / opnfvdocs.git / blobdiff