X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=blobdiff_plain;f=docs%2Ftesting%2Fuser%2Fuserguide%2Foverview.rst;h=746f24c432ce6a63e842bfaba83d65f38098daff;hb=66fbd5fcfafd5dcfde58a61aab33c1812405e097;hp=a93022977ee833157a36799a7be88bd84b743a75;hpb=b39f17a60e91552ffe94c520a30e6b271077f294;p=bottlenecks.git diff --git a/docs/testing/user/userguide/overview.rst b/docs/testing/user/userguide/overview.rst index a9302297..746f24c4 100644 --- a/docs/testing/user/userguide/overview.rst +++ b/docs/testing/user/userguide/overview.rst @@ -2,30 +2,45 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Huawei Technologies Co.,Ltd and others. -********************* -Project Testing Guide -********************* +********** +User Guide +********** For each *testsuite*, you can either setup *teststory* or *testcase* to run certain test. *teststory* comprises several *testcases* as a set in one configuration file. You could call *teststory* or *testcase* by using -Bottlencks user interfaces. +Bottlenecks user interfaces. Details will be shown in the following section. Brief Introdcution of the Test suites in Project Releases ============================================================= -Brahmaputra: rubbos is introduced, which is an end2end NFVI perforamnce tool. -Virtual switch test framework(VSTF) is also introduced, -which is an test framework used for vswitch performance test. +Brahmaputra: -Colorado: rubbos is refactored by using puppet, which makes it quite flexible -to configure with different number of load generator (Client), worker (tomcat). -vstf is refactored by extracting the test case's configuration information. +* rubbos is introduced, which is an end2end NFVI perforamnce tool. +* Virtual switch test framework (VSTF) is also introduced, which is an test framework used for vswitch performance test. -Danube: posca testsuite is introduced to implementing stress (factor), scenario and -tuning test in parametric manner. Two testcases are developed and integrated into -community CI pipeline. Rubbos and VSTF are not supported any more. +Colorado: + +* rubbos is refactored by using puppet, and ease the integration with several load generators(Client) and worker(Tomcat). +* VSTF is refactored by extracting the test case's configuration information. + +Danube: + +* posca testsuite is introduced to implement stress (factor), feature and tuning test in parametric manner. +* Two testcases are developed and integrated into community CI pipeline. +* Rubbos and VSTF are not supported any more. + +Euphrates: + +* Introduction of a simple monitoring module, i.e., Prometheus+Collectd+Node+Grafana to monitor the system behavior when executing stress tests. +* Support VNF scale up/out tests to verify NFVI capability to adapt the resource consuming. +* Extend Life-cycle test to data-plane to validate the system capability to handle concurrent networks usage. +* Testing framework is revised to support installer-agnostic testing. + +These enhancements and test cases help the end users to gain more comprehensive understanding of the SUT. +Graphic reports of the system behavior additional to test cases are provided to indicate the confidence level of SUT. +Installer-agnostic testing framework allow end user to do stress testing adaptively over either Open Source or commercial deployments. Integration Description ======================= @@ -38,39 +53,22 @@ Integration Description +-------------+----------------------+----------------------+ | Danube | Compass | POSCA | +-------------+----------------------+----------------------+ +| Euphrates | Any | POSCA | ++-------------+----------------------+----------------------+ Test suite & Test case Description ================================== -+--------+-------------------------------+ -|POSCA | posca_factor_ping | -| +-------------------------------+ -| | posca_factor_system_bandwidth | -+--------+-------------------------------+ -|Rubbos | rubbos_basic | -| +-------------------------------+ -| | rubbos_TC1101 | -| +-------------------------------+ -| | rubbos_TC1201 | -| +-------------------------------+ -| | rubbos_TC1301 | -| +-------------------------------+ -| | rubbos_TC1401 | -| +-------------------------------+ -| | rubbos_heavy_TC1101 | -+--------+-------------------------------+ -|vstf | vstf_Ti1 | -| +-------------------------------+ -| | vstf_Ti2 | -| +-------------------------------+ -| | vstf_Ti3 | -| +-------------------------------+ -| | vstf_Tn1 | -| +-------------------------------+ -| | vstf_Tn2 | -| +-------------------------------+ -| | vstf_Tu1 | -| +-------------------------------+ -| | vstf_Tu2 | -| +-------------------------------+ -| | vstf_Tu3 | -+--------+-------------------------------+ ++--------+-------------------------------------+ +|POSCA | posca_factor_ping | +| +-------------------------------------+ +| | posca_factor_system_bandwidth | +| +-------------------------------------+ +| | posca_facotor_soak_througputs | +| +-------------------------------------+ +| | posca_feature_vnf_scale_up | +| +-------------------------------------+ +| | posca_feature_vnf_scale_out | ++--------+-------------------------------------+ + +As for the abandoned test suite in the previous Bottlenecks releases, please +refer to http://docs.opnfv.org/en/stable-danube/submodules/bottlenecks/docs/testing/user/userguide/deprecated.html.