Xena_Doc_mono_install_update: Update mono install pointer

Mono Version 5.10 breaks mono execution of Xena2544.exe.

Fix Xena install notes to point to a working version of Mono(5.8)
when setting up to use Xena2544.exe.

JIRA:VSPERF-561

Change-Id: I3bbfaaeef55fc465cba943eb8b1a31abe2faba1f
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

Merge "Trex_speed_improvement: Add logic for dealing with high speed cards"

Trex_speed_improvement: Add logic for dealing with high speed cards

Adds configuration options and logic to detect maximum supported
speed of T-Rex server side cards.

1. Adds logic to pull maximum supported speed from port info
2. Adds forcable option to have user specify maximum speed
3. If logic cannot detect speed from port_info because it is
   not available or the forcable option is not set it will
   assume 10G speeds.
4. Tested on Intel XXV25G and Mellanox ConnectX-5 cards
5. Added packet structure logging to show packet info for
   better debugging capabilities
6. Adds core mask to take advantage of multiple cores if
   server is started with more than default number of cores
7. Adds packets lost logging to RFC2544 Throughput testing

JIRA: VSPERF-559

Change-Id: I7fcfda7ccc408c30830950ee3668e01b8624c20a
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

Merge "LTD: Expand requirements of Soak/Long Duration Tests"

Merge "loadgen: Support for Stressor-VMs as a Loadgen"

python34_update: Updated build scripts for Centos and RHEL to python34

Recent patch update to collectd requires new packages that must
have python34 as a minimum version available. This updates the
build scripts for Centos and RHEL to use scl enabled python34.

1. Adds needed packages to build correctly for Centos and RHEL
2. Changes repo location to pull scl enabled python34
3. Modifies enablement of scl to new package string
4. Updates documentation to reflect change

JIRA: VSPERF-558

Change-Id: I52257ecfc6abc2112c30137c1de3ced272f21066
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

collector: Support for collectd as one of the collectors.

The changes include:
1. Configuration in 05_collector.conf. User can specify the
metrics that are of their interest and also fine tune the same.
2. collectd_bucky.py: This is derived from python_bucky library.
Formal approval for reuse is obtained.
3. collectd.py: This is the main file that receives the samples,
stores the required ones, plots the values and saves the values in
a data-file in results folder.
4. Fixed some pylint errors
5. Added required libraries to requirements.txt. hashlib is supported by
default. However, matplotlib and numpy required
6. added __init__.py to overcome import errors.
7. Crypto needs pycrypto module
8. Builtin filter function usage return bad-builtin pylint warning.
Rewrite to remove filter function.
9. Added support for reporting the ovs processes statistics.
10. Import order corrected according to pylint
11. Fixed copyright issues, removed python-3 checking, enhance to
include VPP and qemu processes.
12. Removed pylint disables.
13. Added documentaion to describe sample information
14. Added @staticmethod to remove disable-no-self-use.
15. Fixed year in Copyright.
16. Updated print_results function to print the process values.
17. Removed example values form INTELRDT_XKEYS in conf file
18. Removed use of len(SEQUENCE) in loop and renamed data to pdata to
avoid name clashes
19. Fixed issue with requirements.txt
Note: A separate documentation covering all external tools will be
provided.

JIRA: VSPERF-505

Change-Id: Ieedcbe7b92e8933263d34df12b1e24be8447b524
Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>

LTD: Expand requirements of Soak/Long Duration Tests

In Project discussion on Jan 17, 2018 including review of RFC 8204,
we identified the following five areas to improve the Soak tests.
See the JIRA ticket for details.

JIRA: VSPERF-557

Change-Id: I5314c283a383c57d99863fcf7262c4f28e1c12a8
Signed-off-by: Al Morton <acmorton@att.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
Reviewed-by: Richard Elias <richardx.elias@intel.com>

Merge "llc-management: Support for LLC management with RMD"

bugfix: Define _EXECUTED_PIDS in config file

Add definition of internal _EXECUTED_PIDS list into generic
configuration file.

JIRA: VSPERF-556

Change-Id: I6c237cb563c0a37bf806561d1f841587796399b8
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>

llc-management: Support for LLC management with RMD

This patch adds support for LLC-Last level cache
management using RMD. The changes include:
1. 08_llcmanagement.conf: The configuration file to define cache
allocation policy.
2. testcase.py: To trigger llc-allocation and cleanup before and
after the test, respectively.
3. llc_management/rmd.py: The main file the performs the llc
allocation and cleanup.
4. llc_management/resthttp.py: Generic utility to call rest APIs,
with http.
5. Fixed a build error due to change in name mismatch.
6. Fixed pylint errors. Override built-in function error.
7. Added empty __init__.py to avoid import errors
8. Fixed deprecated functions errors
9. Fixed copyright issues, removed python-3 checking.
10. Removed pylint disables.
11. resthttp.py is already part of stcrestclient used in
tools/pkt_gen/testcenter/ scripts. Hence, deleted from the source.
12. Year update from the license
13. Fixed some testcases.py collision issue

JIRA: VSPERF-544

Change-Id: I7cbd155dd66f5a0cef544751841e71b95c9b6821
Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>

loadgen: Support for Stressor-VMs as a Loadgen

This patch adds support for stressor-VMs as a loadgen in VSPERF.
The changes include:
1. 07_loadgen.conf: User can specify the stressor-VM specific
configuration. It includes, image-name, path, nics, memory, etc.
2. loadgen/stressorvm/stressor_vm.py: This file implement ILoadGenerator
interface. It implements all the necessary APIs for starting and
stopping the loads.
3. Fixed Pylint Error
4. Removed the network device configuration from qemu-system-x86_64
command as stessor-vm mostly focus on CPU and memory stressing.
5. Moved the creation of loadgen after the VNFs are setup in
testcase.py.
6. Fixed copyright issues.
7. Removed python-3 checking. Improved exception handling
8. Set Default loadgen as DummyLoadGen
9. Improved OSError Printing with directory name and error
10. Update the year in license. 2017-2018.

JIRA: VSPERF-504

Change-Id: Iad6c0780c184f8e36eddcbcae2a580f41118e8dc
Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>

Merge "documentation: Traffic Capture methods"

Merge "trex: Improve error handling"

trex: Improve error handling

Print an error message in case, that vsperf is not able
to connect to the T-Rex server. User is instructed
to check if T-Rex server is up and running.

Change-Id: I70c6d27fa8331921d43e25c5eaa3638ded6c015b
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Jose Lausuch <jalausuch@suse.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

ixia: Add support of LISTs in TRAFFIC

The control script for Ixia (i.e. Ixia and IxNet classes)
is written in TCL and thus VSPERF must translate TRAFFIC
dictionary into TCL notation. The method for data type
conversion was updated to correctly process new 'capture'
settings, which uses python lists for tx and rx capture
ports definition.

JIRA: VSPERF-556

Change-Id: I639942b11ea11ce1b443a2a2e99c3da6619ba569
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
Reviewed-by: Richard Elias <richardx.elias@intel.com>

documentation: Traffic Capture methods

Created new chapter on Traffic Capture and added short descriptions and explanation for each of the three methods currently supported by VSPERF. Subchapters include Traffic Capture in the VM, Traffic for testing NICs with Hardware acceleration and Traffic Capture on the Trraffic Generator.

JIRA: VSPERF-556

Change-Id: I280c54aa73961e0bbb61b40e8fe476bd3e92ff7d
Signed-off-by: Richard Elias <richardx.elias@intel.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

trex: Add support for traffic capture

A support of traffic capture was added into T-Rex.
It allows to write a functional tests, which will
verify proper vSwitch functionality by inspection
of packets received by T-Rex.
A testcase example was added into integration
testcases.

JIRA: VSPERF-556

Change-Id: I5ad28479ca2ec29760b68f24510af1a6d74866ae
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
Reviewed-by: Richard Elias <richardx.elias@intel.com>

capture: Traffic capture examples

Traffic can be captured also at DUT side. Two options are supported by
vsperf:
1) Traffic is captured inside VM after it has been processed
   by vSwitch. This can be used for verification of vSwitch frame
   modification functionality, including HW offloading at ingress side.
2) Another NIC under the test (NIC2) is added into standard VSPERF DUT
   setup. Traffic is then forwarded from TGen through NIC and vSwitch
   to NIC2 and then over patch cable back to NIC2, the vSwitch, NIC and
   to the traffic generator. This setup supports also verification of HW
   offloading at egress side of NIC2 and thus it can be used for validation
   of smart NICS.
Both options above are traffic generator agnostic if compared to direct
support of traffic capture by traffic generator. This patch introduces
example testcases for both options.
Detailed documentation will be pushed as a standalone patch.

JIRA: VSPERF-556

Change-Id: I23e12e45768ae4dbe9442f74d8391c3d5b2c7895
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Richard Elias <richardx.elias@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

vswitch: Improve add_phy_port error messages

In case that vSwitch can't add required number of physical
ports, then error message should be printed. Previous implementation
was not consistent. Thus error handling has been united across
all vswitches and message explicitly states the cause of the failure.

NOTE: A better solution would be a refactoring of all vSwitch
classes to follow VPP port handling and to move port related
structures and physical port check into parent vswitch class.

JIRA: VSPERF-555

Change-Id: Id09a61432ea93e261f563254829348ac61f5dc8a
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Jose Lausuch <jalausuch@suse.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "trafficgen: Postponed call of connect()"

Merge "documentation: Installation prerequisites"

documentation: Installation prerequisites

Added a mention of the default Python3 location used by virtualenv
creation script after Python3 enforcement patch(/usr/bin/python3).
Also added a paragraph warning of held-back kernel package version and
DNF/APT/YUM configuration preventing modification of kernel packages.

JIRA: MSP-10754

Change-Id: I1ba630cf85af4d99cc2cbed40163a9137b95c390
Signed-off-by: Richard Elias <richardx.elias@intel.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
Reviewed-by: Ian Stokes <ian.stokes@intel.com>

Merge "Trex_2544_verification: Add verification functionality to Trex code"

Trex_2544_verification: Add verification functionality to Trex code

- Adds T-Rex verification step as implemented by other trafficgens
- Adds check in rfc2544 loop for if no packets are received to fail test
  immediately
- Refactors the trial code to support the verification option
- Adds trial_run function
- Removed unused line speed configuration setting

JIRA: VSPERF-553

Change-Id: Ie324fe8fb6bf79fe0dc337b91af2bf83e901a8ab
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

trafficgen: Postponed call of connect()

In order to support execution of SW traffic generator inside VM,
we have to postpone the call of connect() method from traffic controller.
Currently it is called during initialization phase, when VMs are not
active and thus connect() would always fail. This patch moves the call
of connect() inside configure() method, which is called later when VMs
are up and running.
This modification allows to configure VM to VM scenario, where 1st VM is
running a loopback application and 2nd one executes one of supported SW
traffic generators, i.e. T-Rex or MoonGen.

Change-Id: Ifeb659aadaa2e1cd5975564965e1719c1e72a12e
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Add support for SLES 15

Change-Id: I555fea7514475153ae74b6f3d2ab2df1c30bb1aa
Signed-off-by: Jose Lausuch <jalausuch@suse.com>

rhel_path_fix: Fix pathing issue introduce by other commit

Fixes pathing issue with virtual environment creation caused by
commit 21b23522861e364daf150cd997c5d60851847ef2

Change-Id: Ia2c0c44da898e40e122571e477301343d047f974
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

Merge "teststeps: Improvements of step driven TC"

Add support for openSuse Tumbleweed

This patch aims at adding support for Tumbleweed, which is
a rolling release. That means that the VERSION_ID is not fixed
and can't be mapped to a directory e.g
- VERSION_ID="20171120"
- VERSION_ID="20171121"

Change-Id: I99c9a4f1042c22b73879e954147c54c87b6a8b21
Signed-off-by: Jose Lausuch <jalausuch@suse.com>

Merge "trex_learning: Add learning packet option to T-Rex testing"

trex_learning: Add learning packet option to T-Rex testing

Adds packet learning option which is available on other
Traffic generators as a feature. This adds this feature to the
Trex implementation inside of VSPerf.

JIRA: VSPERF-547

Change-Id: Iaf4d0721b22eb780c25e29295c112d4fcb47b22c
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

teststeps: Improvements of step driven TC

This patch adds support of access to VMs deployed automatically
by deployment scenario (e.g. pvp, pvvp4, etc.). It also modifies
filter for checking output of executed commands to correctly
handle different end of line encodings (i.e. \n vs. \n\r).

JIRA: VSPERF-539

Change-Id: Ifc796d992b90e316bad5853aaefce79e439e294d
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

install: Python virtualenv python3 enforcement

It was reported that virtualenv-3.x sometimes uses python2.7 instead of
python3. This update of the python virtual environment creation scripts,
enforces the use of python3 from /usr/bin/python3.
The scripts were tested on fresh installations of each currently
supported Linux OS.

JIRA:VSPERF-551

Change-Id: If909dd9578a3c42b14c1a3e4f9ed7b07a675fb58
Signed-off-by: Richard Elias <richardx.elias@intel.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

tools: Update versions of tools

Default versions of DPDK, OVS, VPP and QEMU have been
updated.
VPP socket file configuration is mandatory for VPP 17.10.

Change-Id: I3465161ae9ba9b50dfd90bb19d84f128c72d44ea
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>

ovs_vanilla: Fix vswitch shutdown process

Datapath must be removed before vswitch termination.

JIRA: VSPERF-548

Change-Id: I7408ddc1b21e2bc02a44785dca07d9c026bc28c1
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "trex_stat_packet_counts: Add packet counts to trex reporting"

Merge "sriov: Determine path to the bind-tool"

Merge "regression: OVS/DPDK regression tests"

sriov: Determine path to the bind-tool

In case of SRIOV path to the bind-tool must be known before
NIC driver is re-initialized. It means before TestCase.__init__()
is called.

Change-Id: Ib6fdba233e665144510dd492462c0a21aa19ed71
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

trex_stat_packet_counts: Add packet counts to trex reporting

Adds packet count totals for transmit and receive to reporting
at the end of test execution.

JIRA: VSPERF-546

Change-Id: I51027fae109d0ea794c0db03f5781ddb2fcbb941
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

install: Fedora installation script update

An update of the Fedora installation scripts for versions 24,25 and 26.

Change-Id: Ib510d78f4040e1029bc68d2fa303a4c0a6357de3
Signed-off-by: Richard Elias <richardx.elias@intel.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

regression: OVS/DPDK regression tests

A set of regression tests for verification of OVS/DPDK
features was introduced. Following features are tested:
* DPDK NIC Support
* DPDK Hotplug Support
* RX Checksum Support
* Flow Control Support
* Multiqueue Support
* Vhost User
* Virtual Devices Support
* NUMA Support
* Jumbo Frame Support
* Rate Limiting
* Quality of Service

JIRA: VSPERF-539

Change-Id: I6c83e9c38e29f197a5236ac68c88d7045fef775b
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Signed-off-by: Martin Goldammer <martinx.goldammer@intel.com>
Signed-off-by: Richard Elias <richardx.elias@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

ci: Skip VPP tests if needed

If VPP can't be executed properly, then VPP tests will be skipped
for VERIFY and MERGE jobs. It sometimes happens, that VPP can't
be executed at builders used for VERIFY & MERGE jobs execution.
These failures are not caused by patches themselves, but they
are blocking standard review & merge process. Thus VPP tests
will be suppressed in case that VPP can't be started at all.
This change is not related to DAILY job, which is executed
at POD dedicated to VSPERF project. It means that VPP tests
are always executed by DAILY job.

Change-Id: I4752601561b94a82482c403305bc3f24779f538b
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "paths: Modify algorithm for PATHS verification"

Merge "teststeps: Improvements and bugfixing of teststeps"

vpp: Define vppctl socket name

Newer versions of VPP require an explicit definition of CLI socket
file for proper functionality. Otherwise vppctl is not able to connect
to running vpp process and thus vsperf is not able to configure
and control vpp.

Change-Id: Id0df75fd0894e22b1b4d7fa05e16f49efe20d3ab
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

paths: Modify algorithm for PATHS verification

VSPERF verifies if PATHS to the tools (vSwitch, VNF, etc.)
are valid. This check was modified to take into the account
tools required by selected testcase. It will help to avoid
runtime errors triggered by missing tools, which
are not essential for testcase execution.

Change-Id: Ifcfea983cad1bdbc8c1f08c7a0281001005410df
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "load_gen: Supporting loading of load_gen via loader."

vpp: Multiqueue support for VPP

VPP class was updated to configure RX queues of VPP interfaces
based on VSWITCH_DPDK_MULTI_QUEUES value.

Change-Id: I855dbaef86cf935b2639976e8b1757eb1af4e9ac
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

load_gen: Supporting loading of load_gen via loader.

Currently all tools are loaded via loader utility, except load_gen.
Load_gens were loaded directly through component_factory.
This patch adds support to load load_gens through loader utility.

The changes are as follows:
1. Configuration changes:The common.conf include configuration of
   directory. testcases.conf includes changes to load configuration,
   where tool is no more part of the load-configuration. The
   custom.conf has configuration of LOADGEN to be used - this configuration
   replaces the earlier 'tool' configuration parameter.
2. loader_utility_changes:  In loader.py, loadgen_loader is
   defined, which is used in new get_loadgen function.
3. component_factory changes: in create_loadgen, similar
   to other tools, the function just retuns the object of loadgen_class.
4. Renaming of Dummy load_gen: Loader fails to load properly a dummy
   loadgen due to name-clash with dummy in pkt_gen. To avoid this name clash
   dummy is renamed to dummyloadgen.
5. testcase changes: create_loadgen is now called with output of
   loader's get_loadgen_class.
6. Fixed Pylint Errors and extra-space at the end.
7. Included CLI options support for --loadgen and --list-loadgens.
   Thanks to Martin K.
8. Added the missing loadgen parameter in testcases.conf.
9. Fixed the missing comma error.
10. Added CI change in build-vsperf.sh
11. Fixed configuration reading in stress/stress.py

JIRA: VSPERF-533

Change-Id: I3fbb259618825a12fef55320a748a4f02509190b
Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>

teststeps: Improvements and bugfixing of teststeps

This patch introduces several improvements and small bugfixes
of teststeps. These changes were identified during implementation
of OVS/DPDK regression tests.

Patch content:
* teststeps: step aliases were implemented
* teststeps: improved filtering by regex for any step, which returns string
  or list of stings; filter will process all lines
* teststeps: support for log object
* teststeps: support for trafficgen get_results call
* teststeps: configurable suppression of step validation
* trafficgen: remove old results before traffic is executed
* trafficgen: support for flow control on/off (IxNet)
* trafficgen: support for configurable learning frames (IxNet)
* trafficgen: support for runtime changes of TRAFFICGEN_PKT_SIZES, _DURATION
  and _LOSSRATE
* vnf: flush pexpect output of previous commands
* vnf: use execute_and_wait() to ensure correct cmds order
* vnf: dpdk vHost User interface name set according to its type,
  e.g. dpdkvhostuserclient
* vswitch: support for OVS restart
* decap: simplify configuration of tunneling decapsulation tests
* settings: values of all configuration options are restored after TC execution
* modified formatting of test description used by --list
* testcase name and description is logged before its execution
* small bugfixes

JIRA: VSPERF-539

Change-Id: I550ba0d897ece89abd3f33d6d66f545c4d863e7b
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

INFO: Update PTL

https://lists.opnfv.org/pipermail/opnfv-tech-discuss/2017-October/018501.html

Change-Id: I9305ec2e13e19b32e142640dab63487bab9a2b6a
Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>

Merge "trex_multistream: Add multistream support to T-Rex"

trex_multistream: Add multistream support to T-Rex

Adds incremental source multi-stream functionality to
T-Rex traffic generator code in VSPerf.

JIRA: VSPERF-532

Change-Id: Ib5ba326699d89350ac1715c9a4276e5fa46a133e
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

conf: Fix default VPP configuration

Number of VPP workers (aka PMD threads) was not specified
explicitly in default configuration file. Configuration
has been fixed to use the same cores as OVS with DPDK.

Change-Id: Icd3e5d6f429435fbc1ff580891f4b182d1c48729
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "doc_review: Review docs for accuracy and grammar"

doc_review: Review docs for accuracy and grammar

Fix issues with grammar and accuracy.

JIRA: VSPERF-536

Change-Id: Ia703d306bb7a14b11e09ab4077b8adfa5fb5e6af
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

Merge "Removed drafts of test specificaiton now published as RFC8204. Also removed IETF license notice and added a file with link to RFC8204"

Removed drafts of test specificaiton now published as RFC8204. Also removed IETF license notice and added
a file with link to RFC8204

Change-Id: I44b4cb75afeff74a58f402a7cb8816ddfbe52b87
Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>

Updated project description to clarify VSPERF independence from Open Stack

Change-Id: Iab653c7ad1cf6cbdb9c15056337a2768a489a87a
Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>

trex_prom: Add promiscuous option for trex for SRIOV tests

Adds option to enable promiscuous mode for Trex traffic
generator to allow for SR-IOV tests to work correctly.
Trex by default only accepts packets based on destination
MAC address matching what is configured on server in
/etc/trex-cfg.yaml. This removes that limitation since
SR-IOV tests require mac addresses to be modified in the
generated packets to correctly route packets to the relevant
VFs.

JIRA: VSPERF-531

Change-Id: I19b64f364c8a1b656994c82c96f47a68b92486d8
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

Merge "trex: RFC2544 Throughput bugfixing"

trex: RFC2544 Throughput bugfixing

Changes introduced by patch:
* Ensure that correct results with given lossreate are returned
* Add threshold value to specify the accurancy of RFC2544 Throughput
  measurement. Default value is 0.05.

JIRA: VSPERF-530

Change-Id: I8148deba9039c3a0feb6394d6671aa10c5afaf0a
Signed-off-by: Martin Goldammer <martin.goldammer6@gmail.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

CI: Update hugepages configuration

Changes introduced by this patch:
* Configure more hugepages to be used during CI run.
* Fix detection of correct ovs-vswitchd execution.
* Skip VPP shutdown overhead in case that VPP execution
  has failed.

Change-Id: Id1badd61334141dcc331691af4a1401a09d06807
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>

Merge "trex: Enforce 10 iterations for RFC2544 tput"

Tuning_doc: Add tuning doc for installation of cpu-partitioning profile

Add documentation for installing and configuring tuned-adm using
the cpu-partitioning profile. Tested and verified by Martin and
Christian.

JIRA: VSPERF-514

Change-Id: I7ae25898abe0854c22ae16ee8d3d0a6001cda2f6
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

trex: Enforce 10 iterations for RFC2544 tput

This patch enforces 10 iterations during search
for besti RFC2544 Throughput with 0% packet loss. Unless
0% packet loss is detected during first iteration
with initial frame rate.

JIRA: VSPERF-530

Change-Id: I99e49e9dbf4e1424253d0cfec804819516c00df8
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
Reviewed-by: Martin Goldammer <martin.goldammer6@gmail.com>

Merge "CI: Update hugepages configuration"

CI: Update hugepages configuration

Use the number of free hugepages to correctly set hugepages
configuration. Limit the number of allocated hugepages to
avoid memory exhaustion.

Change-Id: Id5239200579f8c43706ffcc769e4963280b9eb2b
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>

Updated project description in user and developer guides.
Updated info on RFC8204 in developer guide.
Removed whitespaces causing rst validation failure.

Change-Id: Ibdefee2d0fea03873bac8aba514b68de1f1e0dbe
Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "Added two extra packages"

Added two extra packages

Change-Id: Ibd7f5506d5600d8921dec42f10960df06f0f5250
Signed-off-by: Marco Varlese <marco.varlese@suse.com>

Merge "trex: fix Trex makefile"

Merge "doc: Include list of TCs in doc"

Merge "openSUSE support"

openSUSE support

Modified support for Leap 42.2 so not to be prompted for y/n on update
and installation of required packages.
Added Leap 42.3 as supported OS.
Upgraded documentation accordingly.

Change-Id: I33f3046a5b33bc842534a16911423a2febaa9417
Signed-off-by: Marco Varlese <marco.varlese@suse.com>

trex: fix Trex makefile

Trex makefile was fix to checkout correct commit ID.
Add trex repo folder to .gitignore file.

JIRA: VSPERF-528

Change-Id: Icd0406eaf088b88c28d90cf8a7dd9e1fa53a6f3b
Signed-off-by: Martin Goldammer <martin.goldammer6@gmail.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

opnfvresultdb: Report failed TCs into result DB

In following cases, TC will be reported as FAIL in result DB:

* CSV file with VSPERF results is missing (e.g. execution
  of vSwitch, VNF or trafficgen has failed)
* 0 RX FPS is detected for at least one packet size during
  RFC2544 Throughput test (e.g. vSwitch is not properly
  configured due insufficient hugepages size)

JIRA: VSPERF-488

Change-Id: I453bbe32db87497644e1464e74ef597f0f0f323c
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Mars Toktonaliev <mars.toktonaliev@nokia.com>
Reviewed-by: Morgan Richomme <morgan.richomme@orange.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "trex: Add support Trex traffic generator"

trex: Add support Trex traffic generator

Topology are two physical servers, on first is trex and second is VSPERF.
Trex is running in stateless mode this means that on server where is located
trex repo is running trex binary file and VSPERF working with server via
python API.

JIRA: VSPERF-528

Change-Id: Id8819495325ebc13fdce365f4af0e040ce68cd0e
Signed-off-by: Martin Goldammer <martin.goldammer6@gmail.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "docs: Update E release notes"

docs: Update E release notes

Release notes were updated with VSPERF additions during Euphrates
release timeframe.

Change-Id: I87154036676450065906c6fd8bc6db4604c84260
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

doc: Include list of TCs in doc

List of vsperf "inbox" testcases was added into documentation.
This patch also fixes:
* a few SPHINX warnings and errors
* empty userguide index

JIRA: VSPERF-519

Change-Id: I417ed40130f0c7de4f76551f3f1cac0cb1e6da89
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "opnfvresultdb: Add mapping for VPP TCs"

report: Update paths after doc reorganization

Directory with report files was moved to different
location. Thus CI build script must be updated too.

Change-Id: Id5b4c8b2740c01c821c25f8bfdaeb25352410e76
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "build: Rename cloned repositories and remove cuse"

opnfvresultdb: Add mapping for VPP TCs

Patch adds testcase mapping among native VSPERF VPP TCs and appropriate
TC names defined in results DB.

JIRA: VSPERF-488

Change-Id: I5c4a50587a354a741c738d255105726943a701f5
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Mars Toktonaliev <mars.toktonaliev@nokia.com>
Reviewed-by: Morgan Richomme <morgan.richomme@orange.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

build: Rename cloned repositories and remove cuse

VSPERF expects folders to be named a certain way, but this may not be
the case if another repository is specified via the package-list.mk
file. One example of this is the DPDK stable repo which gets cloned to
the "dpdk-stable" directory unless otherwise specified. Thus Makefiles
were updated to enforce expected folder names.

Unused configuration related to vHostCuse support was removed.

Change-Id: Ifa48898d51e96c72e7d740f43cb635d729b2a796
Signed-off-by: Cian Ferriter <cian.ferriter@intel.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

gitignore: Ignore non vsperf files

.gitignore was updated to ignore following files:
* VPP project directory
* l2fwd binary files
* tags

JIRA: VSPERF-529

Change-Id: Idc10eae7b14c72a863f3473e02cbdc7c47d31b41
Signed-off-by: Cian Ferriter <cian.ferriter@intel.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "opnfvresultdb: Update data reported to result DB"

opnfvresultdb: Update data reported to result DB

Data reported to result DB were updated to comply
with recent API version and requirements.

JIRA: VSPERF-488

Change-Id: I6ab02ea54d99099e88b0d5ab6e9745ee8297f544
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Signed-off-by: Mars Toktonaliev <mars.toktonaliev@nokia.com>
Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

Moved devguide for consitency with docs dir structure for all testing projects
Updated RFC description based on IETF approval of Internet Draft

Change-Id: Ifd37da946866f350b8968bbbe8c5a3f5ce762cfa
Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>

Merge "driverctl: Add driverctl binding tool"

driverctl: Add driverctl binding tool

Adds driverctl as a binding tool option for dpdk devices. JIRA
ticket contains test results to verify it works when using
dpdk-devbind or driverctl.

JIRA: VSPERF-525

Change-Id: Ia7a6e43a4b6fb3ba21f5d87d601f4ea4b95b7dab
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

Merge "xena2544_verify_fix: Fix Xena verify options"

Merge "tests: L3, L4 and VxLAN tests for OVS & VPP"

xena2544_verify_fix: Fix Xena verify options

Fixes issue when running multiple packets sizes and using the Xena
verify option. Original values are restored to settings for each
new packet size or test case.

Also fixes a minor issue where if the test failed, the verify would
run regardless.

JIRA: VSPERF-515

Change-Id: I6b6ce5160b66e7ed4a3847711c4f05529ae8ca54
Signed-off-by: Christian Trautman <ctrautma@redhat.com>

tests: L3, L4 and VxLAN tests for OVS & VPP

A set of tests was introduced with focus on L3, L4 and VxLAN performance
of OVS and VPP. New testcases were created for phy2phy network scenario.
In case of PVP and PVVP, only OVS testcases are available.

Notes:
* two sets of OVS P2P testcases were created, one creates unique flow
  for each IP address involved in the test (performance sensitive);
  Second set inserts just one flow with large network mask (tests with
  _mask suffix).
* three different types of VPP P2P L3 testcases were created to
  demonstrate performance impact of multi ARP entries or IP routes.
* VPP multi ARP record based testcases use a set of "workarounds"
  to load a large number of ARP entries. It is not possible to use
  "set ip arp count" syntax, as it doesn't work well for large count
  values (e.g. 60K)
* OVS VxLAN testcases utilize existing OP2P deployment scenario and thus
  it can be used also with GRE and GENEVE tunnel types. Tunnel type
  to be used is defined by test configuration option "Tunnel Type".

JIRA: VSPERF-518

Change-Id: I65adad976f12d8625d918a1996eb42693c511ee1
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
Reviewed-by: Cian Ferriter <cian.ferriter@intel.com>

Merge "ci: Fix path to CI report"