1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
3 .. (c) Huawei Technologies Co.,Ltd and others.
12 The POSCA (Parametric Bottlenecks Testing Catalogue) testsuite
13 classifies the bottlenecks test cases and results into 5 categories.
14 Then the results will be analyzed and bottlenecks will be searched
15 among these categories.
17 The POSCA testsuite aims to locate the bottlenecks in parmetric
18 manner and to decouple the bottlenecks regarding the deployment
20 The POSCA testsuite provides an user friendly way to profile and
21 understand the E2E system behavior and deployment requirements.
23 Goals of the POSCA testsuite:
24 a) Automatically locate the bottlenecks in a iterative manner.
25 b) Automatically generate the testing report for bottlenecks in different categories.
26 c) Implementing Automated Staging.
28 Scopes of the POSCA testsuite:
29 a) Modeling, Testing and Test Result analysis.
30 b) Parameters choosing and Algorithms.
32 Test stories of POSCA testsuite:
33 a) Factor test (Stress test): base test cases that Feature test and Optimization will be dependant on.
34 b) Feature test: test cases for features/scenarios.
35 c) Optimization test: test to tune the system parameter.
37 Detailed workflow is illutrated below.
39 * https://wiki.opnfv.org/display/bottlenecks
44 * Please refer to release notes.
49 POSCA testsuite is hight automated regarding test environment preparation, installing testing tools, excuting tests and show the report/analysis. A few steps are needed to run it locally.
51 It is presumed that a user is using Compass4nfv to deploy OPNFV Danube and the user logins jumper server as root.
53 Downloading Bottlenecks Software
54 --------------------------------
60 git clone https://gerrit.opnfv.org/gerrit/bottlenecks
62 Preparing Python Virtual Evnironment
63 ------------------------------------
69 Excuting Specified Testcase
70 ---------------------------
72 Bottlencks provide a CLI interface to run the tests, which is one of the most convenient way since it is more close to our natural languge. An GUI interface with rest API will also be provided in later update.
76 bottlenecks [testcase run <testcase>] [teststory run <teststory>]
78 For the *testcase* command, testcase name should be the same as the name of the test case configuration file located in testsuites/posca/testcase_cfg.
79 For the *teststory* command, a user could specified the test cases to be excuted by defined it in a teststory configuration file located in testsuites/posca/testsuite_story. There is also an example there named *posca_factor_test*.
81 There are also other 2 ways to run test cases and test stories.
82 The first one is using shell script.
86 bash run_tests.sh [-h|--help] [-s <test suite>] [-c <test case>]
88 The second is using python interpreter.
92 python testsuites/posca/run_posca.py [testcase <testcase>] [teststory <teststory>]
95 Cleaning Up Environment
96 -----------------------
103 Run POSCA through Community CI
104 ==============================
105 * POSCA test cases are runned by OPNFV CI now. See https://build.opnfv.org for more information.
107 Test Result Description
108 =======================
109 * Please refer to release notes and also https://wiki.opnfv.org/display/testing/Result+alignment+for+ELK+post-processing