vsperf: Performance Matrix functionality support 57/53857/3
authorRichard Elias <richardx.elias@intel.com>
Wed, 28 Feb 2018 15:48:00 +0000 (15:48 +0000)
committerRichard Elias <richardx.elias@intel.com>
Mon, 19 Mar 2018 11:15:54 +0000 (11:15 +0000)
commit82a17a3d7c59a49105d6f3b2e4044ab19f77b0a0
treea581b058b4fe287b317360f5505b3baf96d7e40a
parent50658b85db1d848ab1ebfa7daaac8ce479c793b4
vsperf: Performance Matrix functionality support

The patch expands the vsperf --test-params argument with list
functionality, which enables running multiple tests with different
parameters. If more tests are run then parameters provided, the
last parameters will be reused. Example:
./vsperf --test-params "['','TRAFFICGEN_PKTSIZE = (64,)']"
phy2phy_cont phy2phy_cont
CUMULATIVE_PARAMS if true, instead of using the default settings,
each test will take the parameters of the previous test before
applying it's own.
The patch also adds the vsperf --matrix argument which analyzes and
compares the results of all the tests run, printing it as a table,
as well as saving it into a file in the results directory.
MATRIX_METRIC metric used by Performance Matrix to compare tests.

JIRA: VSPERF-554

Change-Id: I71530ddf110890612236a7e57039f249609c835b
Signed-off-by: Richard Elias <richardx.elias@intel.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
ci/build-vsperf.sh
conf/00_common.conf
docs/testing/user/userguide/testusage.rst
requirements.txt
testcases/testcase.py
tools/report/report_foot.rst
vsperf