Adding Posca Testsuite in Bottlenecks

JIRA: BOTTLENECK-92

The POSCA (Parametric Bottlenecks Testing Catalogue) testsuite
classifies the bottlenecks test cases and results into 5 categories.
Then the results will be analyzed and decoupled  bottlenecks will be
searched among these categories.

The POSCA testsuite aims to locate the bottlenecks in parmetric
manner and to decouple the bottlenecks regarding the deployment
requirements.

The POSCA testsuite provides an user friendly way to profile and
understand the E2E system behavior and deployment requirements.

Goals of the POSCA testsuite:
a) Automatically locate the bottlenecks in a iterative manner.
b) Automatically generate the testing report for bottlenecks in
   different categories.
c) Implementing Automated Staging.

Scopes of the POSCA testsuite:
a) Modeling, Testing and Test Result analysis.
b) Parameters choosing and Algorithms

Test stories of the POSCA testsuite:
a) Factor test: base test cases that Feature test and Optimization
   will be dependant on
b) Feature test: test cases for features/scenarios
c) Optimization test: test to tune the system parameter

For more details of the proposal, please refer to
https://jira.opnfv.org/secure/attachment/11479/
POSCA%20Testsuite%20Proposal%20in%20Bottlenecks.pdf

Changes:

1. Add Posca testsuite directories and files to Bottleneks,
leaving adding test cases for latter commits.

2. Fix file appd issue.

3. Delete the un-used packages in testsuites/posca/run_posca.py

4. Add testcase processing logic to run_posca.py

5. Add config dimensions

6. Add scripts directory

Change-Id: I54d9b51381a804b76a83680912227e058da3165e
Signed-off-by: yuyang <Gabriel.yuyang@huawei.com>

diff --git a/run_tests.sh b/run_tests.sh
index d5c3059..ad0b31c 100755 (executable)
--- a/run_tests.sh
+++ b/run_tests.sh
@@ -26,7 +26,7 @@ examples:
 
 report=true
 
-arr_test_suite=(rubbos vstf)
+arr_test_suite=(rubbos vstf posca)
 
 function check_testcase(){
 
@@ -38,6 +38,9 @@ function check_testcase(){
          "-vstf")
              SUITE_PREFIX=$SUITE_PREFIX_CONFIG/vstf/testcase_cfg
          ;;
+         "-posca")
+             SUITE_PREFIX=$SUITE_PREFIX_CONFIG/posca/testcase_cfg
+         ;;
     esac
 
     TEST_CASE=$2
@@ -99,6 +102,23 @@ function run_test(){
                 python /home/opnfv/bottlenecks/testsuites/vstf/run_vstf.py -c $file
             done
         ;;
+        "posca")
+            info "Running posca test suite"
+            test_file="/home/opnfv/bottlenecks/testsuites/posca/testsuite_story/posca_factor_test"
+            if [[ -f $test_file ]]; then
+                testcases=($(cat $test_file))
+            else
+                error "no posca test suite file "
+            fi
+            for i in "${testcases[@]}"; do
+                #check if the testcase is legal or not
+                check_testcase -posca $i
+                #adjust config parameters
+                #run test case
+                file=${BASEDIR}/testsuites/posca/testcase_cfg/${i}.yaml
+                python /home/opnfv/bottlenecks/testsuites/posca/run_posca.py -c $file
+            done
+        ;;
     esac
 }
 

diff --git a/testsuites/posca/run_posca.py b/testsuites/posca/run_posca.py
new file mode 100755 (executable)
index 0000000..9f90b3c
--- /dev/null
+++ b/testsuites/posca/run_posca.py
@@ -0,0 +1,39 @@
+#!/usr/bin/env python\r
+##############################################################################\r
+# Copyright (c) 2016 Huawei Technologies Co.,Ltd and others.\r
+#\r
+# All rights reserved. This program and the accompanying materials\r
+# are made available under the terms of the Apache License, Version 2.0\r
+# which accompanies this distribution, and is available at\r
+# http://www.apache.org/licenses/LICENSE-2.0\r
+##############################################################################\r
+\r
+import os\r
+import sys\r
+import subprocess\r
+\r
+INTERPRETER = "/usr/bin/python"\r
+#------------------------------------------------------\r
+# run posca testcase\r
+# ------------------------------------------------------\r
+def posca_run(arg):\r
+    print "========== run posca ==========="\r
+    if(arg == "factor_system_bandwidth"):\r
+        print "========== run posca_system_bandwidth ==========="\r
+        cmd = '/home/opnfv/bottlenecks/testsuites/posca/testcase_script/posca_factor_system_bandwidth.py'\r
+        pargs = [INTERPRETER,cmd]\r
+        sub_result = subprocess.Popen(pargs)\r
+        sub_result.wait()\r
+\r
+def posca_env_check():\r
+    print "========== posca env check ==========="\r
+\r
+def main():\r
+    para_testname = sys.argv[0]\r
+    para_test_arg = sys.argv[1]\r
+    posca_env_check()\r
+    posca_run(para_test_arg)\r
+    sys.exit(0)\r
+\r
+if __name__=='__main__':\r
+    main()\r

diff --git a/testsuites/posca/testcase_cfg/posca_factor_system_bandwidth.yaml b/testsuites/posca/testcase_cfg/posca_factor_system_bandwidth.yaml
new file mode 100755 (executable)
index 0000000..113997c
--- /dev/null
+++ b/testsuites/posca/testcase_cfg/posca_factor_system_bandwidth.yaml
@@ -0,0 +1,11 @@
+[config]\r
+test_ip: 192.168.23.2:8888\r
+tool: netperf\r
+protocol: tcp\r
+test_time: 30\r
+tx pkt sizes: 8, 16, 32, 64, 128, 256, 512, 1024\r
+rx pkt sizes: 8, 16, 32, 64, 128, 256, 512, 1024\r
+tx cache sizes: 2304, 4608, 9216, 18432, 32768, 65536\r
+rx cache sizes: 1152, 2304, 4608, 9216, 18432, 32768, 65536, 131072\r
+cpu load: 0.9\r
+latency: 100000\r

diff --git a/testsuites/posca/testcase_script/__init__.py b/testsuites/posca/testcase_script/__init__.py
new file mode 100644 (file)
index 0000000..83b8d15
--- /dev/null
+++ b/testsuites/posca/testcase_script/__init__.py
@@ -0,0 +1,8 @@
+##############################################################################
+# Copyright (c) 2015 Huawei Technologies Co.,Ltd and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################

diff --git a/testsuites/posca/testsuite_story/posca_factor_test b/testsuites/posca/testsuite_story/posca_factor_test
new file mode 100644 (file)
index 0000000..a19fce5
--- /dev/null
+++ b/testsuites/posca/testsuite_story/posca_factor_test
@@ -0,0 +1 @@
+posca_factor_system_bandwidth
\ No newline at end of file