CI: terminate hanging vsperf before its execution

In case that vsperf terminates unexpectedly, then vsperf itself
or any of executed applications might remain active. It is essential
to terminate any hanging instance of vsperf, ovs, qemu, etc.
before re-execution of vsperf itself. Otherwise vsperf will fail
again.

Change-Id: Idd65288a5171d19fcbfb907a156347eb4b645a88
JIRA: VSPERF-155
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
Reviewed-by: Gurpreet Singh <gurpreet.singh@spirent.com>
Reviewed-by: Tv Rao <tv.rao@freescale.com>
(cherry picked from commit b2b11bf301bbe37befdda23e2a0f1501c079405f)

diff --git a/ci/build-vsperf.sh b/ci/build-vsperf.sh
index 13c17a7..7c84767 100755 (executable)
--- a/ci/build-vsperf.sh
+++ b/ci/build-vsperf.sh
@@ -48,6 +48,30 @@ TESTPARAM_DAILY='--test-params pkt_sizes=64,128,512,1024,1518'
 # functions
 #
 
+# terminate vsperf and all its utilities
+# it is expected that vsperf is the only python3 app
+# and no other ovs or qemu instances are running
+# at CI machine
+# parameters:
+#   none
+function terminate_vsperf() {
+    sudo pkill stress &> /dev/null
+    sudo pkill python3 &> /dev/null
+    sudo killall -9 qemu-system-x86_64 &> /dev/null
+
+    # sometimes qemu resists to terminate, so wait a bit and kill it again
+    if pgrep qemu-system-x86_64 &> /dev/null ; then
+        sleep 5
+        sudo killall -9 qemu-system-x86_64 &> /dev/null
+        sleep 5
+    fi
+
+    sudo pkill ovs-vswitchd &> /dev/null
+    sleep 1
+    sudo pkill ovsdb-server &> /dev/null
+    sleep 1
+}
+
 # check and print testcase execution status
 # parameters:
 #   $1 - directory with results
@@ -153,8 +177,11 @@ case $1 in
         echo "VSPERF daily job"
         echo "================"
 
+        terminate_vsperf
         execute_vsperf OVS_with_DPDK_and_vHost_User $1
+        terminate_vsperf
         execute_vsperf OVS_vanilla $1
+        terminate_vsperf
 
         exit $EXIT
         ;;