POSCA Introduction
====================
-The POSCA (Parametric Bottlenecks Testing Catalogue) testsuite
+The POSCA (Parametric Bottlenecks Testing Catalogue) test suite
classifies the bottlenecks test cases and results into 5 categories.
Then the results will be analyzed and bottlenecks will be searched
among these categories.
-The POSCA testsuite aims to locate the bottlenecks in parmetric
+The POSCA testsuite aims to locate the bottlenecks in parametric
manner and to decouple the bottlenecks regarding the deployment
requirements.
The POSCA testsuite provides an user friendly way to profile and
b) Parameters choosing and Algorithms.
Test stories of POSCA testsuite:
- a) Factor test (Stress test): base test cases that Feature test and Optimization will be dependant on.
+ a) Factor test (Stress test): base test cases that Feature test and Optimization will be dependant on or stress test to validate system.
b) Feature test: test cases for features/scenarios.
c) Optimization test: test to tune the system parameter.
* Docker: https://docs.docker.com/engine/installation/
* For Ubuntu, please refer to https://docs.docker.com/engine/installation/linux/ubuntu/
+[Since Euphrates release, the docker-compose package is not required.]
+
* Docker-Compose: https://docs.docker.com/compose/
.. code-block:: bash
Run POSCA Locally
=================
-POSCA testsuite is highly automated regarding test environment preparation, installing testing tools, excuting tests and showing the report/analysis.
+The test environment preparation, the installation of the testing tools,
+the execution of the tests and the reporting/analyisis of POSCA test suite
+are highly automated.
A few steps are needed to run it locally.
-It is presumed that a user is using Compass4nfv to deploy OPNFV Danube and the user logins jumper server as root.
+In Euphrates, Bottlenecks has modified its framework to support installer-agnostic
+testing which means that test cases could be executed over different deployments.
Downloading Bottlenecks Software
. pre_virt_env.sh
-Excuting Specified Testcase
+Preparing configuration/description files
+-----------------------------------------
+
+Put OpenStack RC file (admin_rc.sh), os_carcert and pod.yaml (pod descrition file) in /tmp directory.
+Edit admin_rc.sh and add the following line
+
+.. code-block:: bash
+
+ export OS_CACERT=/tmp/os_cacert
+
+If you have deployed your openstack environment by compass,
+you could use the following command to get the required files. For Fuel, Apex and JOID installer, we only provide limited support now
+for retrieving the configuration/description files. If you find that the following command can not do the magic, you should put the
+required files in /tmp manually.
+
+.. code-block:: bash
+
+ bash /utils/env_prepare/config_prepare.sh -i <installer> [--debug]
+
+Note that if we execute the command above, then admin_rc.sh and pod.yml gets created automatically in /tmp folder along with the line `export OS_CACERT=/tmp/os_cacert` added in admin_rc.sh file.
+
+
+Executing Specified Testcase
---------------------------
-Bottlencks provide a CLI interface to run the tests, which is one of the most convenient way since it is more close to our natural languge. An GUI interface with rest API will also be provided in later update.
+1. Bottlenecks provides a CLI interface to run the tests, which is one of the most convenient way since it is more close to our natural languge. An GUI interface with rest API will also be provided in later update.
.. code-block:: bash
- bottlenecks [testcase run <testcase>] [teststory run <teststory>]
+ bottlenecks testcase|teststory run <testname>
-For the *testcase* command, testcase name should be as the same name of the test case configuration file located in testsuites/posca/testcase_cfg.
-For stress tests in Danube, *testcase* should be replaced by either *posca_factor_ping* or *posca_factor_system_bandwidth*.
-For the *teststory* command, a user could specified the test cases to be excuted by defined it in a teststory configuration file located in testsuites/posca/testsuite_story. There is also an example there named *posca_factor_test*.
+For the *testcase* command, testname should be as the same name of the test case configuration file located in testsuites/posca/testcase_cfg.
+For stress tests in Danube/Euphrates, *testcase* should be replaced by either *posca_factor_ping* or *posca_factor_system_bandwidth*.
+For the *teststory* command, a user can specify the test cases to be executed by defining it in a teststory configuration file located in testsuites/posca/testsuite_story. There is also an example there named *posca_factor_test*.
-There are also other 2 ways to run test cases and test stories.
-The first one is using shell script.
+2. There are also other 2 ways to run test cases and test stories.
+
+ The first one is to use shell script.
.. code-block:: bash
- bash run_tests.sh [-h|--help] [-s <testsuite>] [-c <testcase>]
+ bash run_tests.sh [-h|--help] -s <testsuite>|-c <testcase>
+
-The second is using python interpreter.
+ The second is to use python interpreter.
.. code-block:: bash
- docker-compose -f docker/bottleneck-compose/docker-compose.yml up -d
- docker pull tutum/influxdb:0.13
+ $REPORT=False
+ opts="--privileged=true -id"
+ docker_volume="-v /var/run/docker.sock:/var/run/docker.sock -v /tmp:/tmp"
+ docker run $opts --name bottlenecks-load-master $docker_volume opnfv/bottlenecks:latest /bin/bash
sleep 5
POSCA_SCRIPT="/home/opnfv/bottlenecks/testsuites/posca"
- docker exec bottleneckcompose_bottlenecks_1 python ${POSCA_SCRIPT}/run_posca.py [testcase <testcase>] [teststory <teststory>]
+ docker exec bottlenecks-load-master python ${POSCA_SCRIPT}/../run_posca.py testcase|teststory <testname> ${REPORT}
Showing Report
If you want to clean the dockers that established during the test, you can excute the additional commands below.
-.. code-block:: bash
-
- docker-compose -f docker/bottleneck-compose/docker-compose.yml down -d
- docker ps -a | grep 'influxdb' | awk '{print $1}' | xargs docker rm -f >/dev/stdout
-
-Or you can just run the following command
-
.. code-block:: bash
bash run_tests.sh --cleanup
.. code-block:: bash
- bash run_tests.sh [-h|--help] [-s <testsuite>] [-c <testcase>] --report
+ bash run_tests.sh [-h|--help] -s <testsuite>|-c <testcase> --report
2. Report testing result by python interpreter
.. code-block:: bash
- docker-compose -f docker/bottleneck-compose/docker-compose.yml up -d
- docker pull tutum/influxdb:0.13
+ REPORT=True
+ opts="--privileged=true -id"
+ docker_volume="-v /var/run/docker.sock:/var/run/docker.sock -v /tmp:/tmp"
+ docker run $opts --name bottlenecks-load-master $docker_volume opnfv/bottlenecks:latest /bin/bash
sleep 5
REPORT="True"
POSCA_SCRIPT="/home/opnfv/bottlenecks/testsuites/posca"
- docker exec bottleneckcompose_bottlenecks_1 python ${POSCA_SCRIPT}/run_posca.py [testcase <testcase>] [teststory <teststory>] REPORT
+ docker exec bottlenecks_load-master python ${POSCA_SCRIPT}/../run_posca.py testcase|teststory <testcase> ${REPORT}
Test Result Description
=======================
Code Review / bottlenecks.git / blobdiff