yardstick.git
6 years agoCleanup GetDataDbClientTestCase unit tests 21/59321/2
Rodolfo Alonso Hernandez [Fri, 29 Jun 2018 11:01:31 +0000 (12:01 +0100)]
Cleanup GetDataDbClientTestCase unit tests

Removed unneeded output.

JIRA: YARDSTICK-1268

Change-Id: I2e9130e863ba915feeb52f54fc901898b1c8f792
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoCleanup VsperfDPDKTestCase unit tests 19/59319/2
Rodolfo Alonso Hernandez [Fri, 29 Jun 2018 10:37:42 +0000 (11:37 +0100)]
Cleanup VsperfDPDKTestCase unit tests

Removed unneeded output.

JIRA: YARDSTICK-1267

Change-Id: I0a3f25db2c5ef8ed71fd087b6b35641075c121e7
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoCleanup TestProxBinSearchProfile unit tests 15/59315/2
Rodolfo Alonso Hernandez [Fri, 29 Jun 2018 10:29:34 +0000 (11:29 +0100)]
Cleanup TestProxBinSearchProfile unit tests

Removed unneeded output and mocks.

JIRA: YARDSTICK-1266

Change-Id: I04fd1ba32ea46c7896548faebc5af1a43b0f750e
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoNFVI support for standalone, baremetal and heat contexts 35/58035/18
Chornyi, TarasX [Tue, 29 May 2018 14:59:49 +0000 (17:59 +0300)]
NFVI support for standalone, baremetal and heat contexts

JIRA: YARDSTICK-1257

Change-Id: I6733bd49ac91985e8f3d7722e6990e8733bb430e
Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
6 years agoMerge "Removed unused ACL config file."
Rodolfo Alonso Hernandez [Fri, 29 Jun 2018 08:21:57 +0000 (08:21 +0000)]
Merge "Removed unused ACL config file."

6 years agoMerge "Add testsuite "os-odl-ovs-noha""
Rex Lee [Fri, 29 Jun 2018 01:47:01 +0000 (01:47 +0000)]
Merge "Add testsuite "os-odl-ovs-noha""

6 years agoMerge "Cleanup process test cases"
Rodolfo Alonso Hernandez [Thu, 28 Jun 2018 15:56:08 +0000 (15:56 +0000)]
Merge "Cleanup process test cases"

6 years agoMerge "Cleanup OpenStack utils test cases"
Rodolfo Alonso Hernandez [Thu, 28 Jun 2018 15:55:59 +0000 (15:55 +0000)]
Merge "Cleanup OpenStack utils test cases"

6 years agoDecrease Sampling interval - Invalid Port results 01/59201/1
DanielMartinBuckley [Thu, 28 Jun 2018 13:51:09 +0000 (14:51 +0100)]
Decrease Sampling interval - Invalid Port results

JIRA: YARDSTICK-1219

The computed in/fwd throughputs displayed on the L2FWD,
L3FWD and VPE dashboards. The computed value is likely
4x the expected value.

It is required to return sampling information form Generator AND VNF at
least every  1 second.

Change-Id: I4435fd05ba3116ead836843a4c2fce133b767a28
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoRemoved unused ACL config file. 59/59159/1
Mytnyk, Volodymyr [Wed, 27 Jun 2018 19:13:19 +0000 (20:13 +0100)]
Removed unused ACL config file.

Change-Id: Ibdb1dd12587df0182b5e8d3b7bc6901be0aac3ec
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoMerge "Update to Grafana Dashboard for NSB PROX NFVI"
Abhijit Sinha [Wed, 27 Jun 2018 18:47:40 +0000 (18:47 +0000)]
Merge "Update to Grafana Dashboard for NSB PROX NFVI"

6 years agoMerge "NSB NFVi PROX BNG losing many packets"
Abhijit Sinha [Wed, 27 Jun 2018 18:47:35 +0000 (18:47 +0000)]
Merge "NSB NFVi PROX BNG losing many packets"

6 years agoNSB NFVi PROX BNG losing many packets 31/58731/9
DanielMartinBuckley [Tue, 19 Jun 2018 15:43:41 +0000 (16:43 +0100)]
NSB NFVi PROX BNG losing many packets

JIRA: YARDSTICK-1101

Problem was incorrect generator and changed the traffic profile to allow
a tolerated loss of 5%.

ARP packets are not managed (just passed thru the system) therefor the
tolerated loss must be higehr than normal.

Change-Id: I03bd62b397a66b50961e810d033c6894d0b3e9a6
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoUpdate to Grafana Dashboard for NSB PROX NFVI 47/59147/1
DanielMartinBuckley [Wed, 27 Jun 2018 16:47:33 +0000 (17:47 +0100)]
Update to Grafana Dashboard for NSB PROX NFVI

JIRA: YARDSTICK-1252

CPU grouping in grafana for better representation
Throughput table is difficult to read as the tests
  run longer the values can overlap
Default time in dashboard should be always last 1hour in
  dashboard (auto refresh can be also checked)
Add one column in table with Gbps throughput (Rx
  Throughput) plus code change in yardstick
Boxes are not aligned and big and small, need to be fixed
  boxes are now aligned,
Added Seperate graphe for LINE Rate

Change-Id: Ief31777c2e7a18aab4bf21fb6ccc0e93a3bb9178
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoMerge "Fix NSB NfVi support for 25 and 40Gbps"
Abhijit Sinha [Wed, 27 Jun 2018 16:06:17 +0000 (16:06 +0000)]
Merge "Fix NSB NfVi support for 25 and 40Gbps"

6 years agoCorrected scale-up command line arguments 35/59035/4
Chornyi, TarasX [Tue, 26 Jun 2018 11:39:56 +0000 (14:39 +0300)]
Corrected scale-up command line arguments

Change-Id: If308fde797dcb01e58f37ad81bb116fde542366a
Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
6 years agoFix NSB NfVi support for 25 and 40Gbps 39/58239/8
Xavier Simonart [Tue, 5 Jun 2018 12:06:29 +0000 (14:06 +0200)]
Fix NSB NfVi support for 25 and 40Gbps

When setting interface_speed option in the test case, NSB/PROX
was still considering 100% as 10 Gbps.
With this fix, when interface_speed is set to for instance 25 Gbps
then NSB/PROX will start generate at 25 Gbps.
Yardstick logging info printed on the screen will show performance result
as percentage of interface_speed (e.g. 100% will be 25 Gbps).
This is done by sending the command "speed 250" to PROX (PROX cannot
know the underlying interface speed in all cases e.g. in virtio, and
hence considers 100% as 10Gbps).

JIRA: YARDSTICK-1217

Change-Id: I4989161b04a7afa5e347935ee7ccef7dfe5a504d
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoMerge "Resolve NameError in test_utils.py"
Rodolfo Alonso Hernandez [Wed, 27 Jun 2018 12:29:24 +0000 (12:29 +0000)]
Merge "Resolve NameError in test_utils.py"

6 years agoNSB NFVi PROX VPE losing many packets 85/58685/8
DanielMartinBuckley [Mon, 18 Jun 2018 18:27:41 +0000 (19:27 +0100)]
NSB NFVi PROX VPE losing many packets

JIRA: YARDSTICK-1105

Running a vPE test ends up with up to 25% drooped packets.
This is due to tolerated loss being incorrectly configured,
test duration was too short and test interval was too
short.

Note This is due to the fact that vPE use case have by
default a tolerated loss of 100%, Now changed to 0.001,
same as L2FWD

Also: Grafan dashboard has been updated to show all SUT
CPUS Utilization in 1 panel. And LINE rate MIN, MAX & TEST
Rate have been added

Change-Id: I7adae2199b3f656fe460705b6aeb3aa69c767d50
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoAdd testsuite "os-odl-ovs-noha" 87/58187/2
ting wu [Mon, 4 Jun 2018 12:18:15 +0000 (14:18 +0200)]
Add testsuite "os-odl-ovs-noha"

Add testsuite yaml file for scenario
"os-odl-ovs-noha"

JIRA: YARDSTICK-1208

Change-Id: I2671df847332379afed8c6481e54eb25dcba5e48
Signed-off-by: ting wu <ting.wu@enea.com>
6 years agoMerge "Define several containers per pod in a replication controller"
Rodolfo Alonso Hernandez [Wed, 27 Jun 2018 10:32:45 +0000 (10:32 +0000)]
Merge "Define several containers per pod in a replication controller"

6 years agoMerge "Decrease Sampling interval"
Abhijit Sinha [Wed, 27 Jun 2018 01:53:19 +0000 (01:53 +0000)]
Merge "Decrease Sampling interval"

6 years agoNSB NFVi BNG test fails to run - stops after one step 43/58543/7
DanielMartinBuckley [Thu, 14 Jun 2018 15:15:26 +0000 (16:15 +0100)]
NSB NFVi BNG test fails to run - stops after one step

JIRA: YARDSTICK-1244

When sending the start or stop command to PROX. We sould not send the same core.
We should not send "start 1,1" instead we should send "start 1"

Change-Id: Ie600dc3ed808fd00341e92f13bd52199e68dc51f
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoDecrease Sampling interval 61/58261/21
DanielMartinBuckley [Tue, 5 Jun 2018 18:06:38 +0000 (19:06 +0100)]
Decrease Sampling interval

JIRA: YARDSTICK-1219

Currently NSB for NFVI using PROX returns sampling information every 13
seconds.

This is too slow.

It is required to return sampling information form Generator AND VNF at
least every  1 second.

This change is depandant on JIRA: YARDSTICK-1212 and YARDSTICK-1220

Change-Id: Ica7ab795a2919d191d8cd846d028e15739e33fb7
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoMerge "Addition of Configurable Sampling and Configurable Confirmation Retry"
Abhijit Sinha [Tue, 26 Jun 2018 17:39:32 +0000 (17:39 +0000)]
Merge "Addition of Configurable Sampling and Configurable Confirmation Retry"

6 years agoMerge "Bugfix: heat conext of test case yamls jinja2 bypass sriov type"
Abhijit Sinha [Tue, 26 Jun 2018 17:39:18 +0000 (17:39 +0000)]
Merge "Bugfix: heat conext of test case yamls jinja2 bypass sriov type"

6 years agoResolve NameError in test_utils.py 57/59057/1
Emma Foley [Tue, 26 Jun 2018 17:13:35 +0000 (18:13 +0100)]
Resolve NameError in test_utils.py

The import of the yardstick module was removed from test_util.py, which relied
on this module location for a file name in
test_import_modules_from_package_no_mod and test_import_modules_from_package.

JIRA: YARDSTICK-1258
Change-Id: I754dbadfb8d788236f354e9a0c1a61ad6c4abf70
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoMerge "Extended Context class with get_physical_nodes functionality"
Rodolfo Alonso Hernandez [Tue, 26 Jun 2018 16:11:38 +0000 (16:11 +0000)]
Merge "Extended Context class with get_physical_nodes functionality"

6 years agoAddition of Configurable Sampling and Configurable Confirmation Retry 97/58297/31
DanielMartinBuckley [Wed, 6 Jun 2018 14:06:01 +0000 (15:06 +0100)]
Addition of Configurable Sampling and Configurable Confirmation Retry

JIRA: YARDSTICK-1221

In order to increase accuracy of result the following are done :-
  - Improve Measurement Accuracy (YARDSTICK-1212)
  - Improve Sampling Interval (YARDSTICK-1219)
  - Allow 4 PROX ports to be read simultaneously (YARDSTICK-1220)

This change does the following :-
  - Stores LINE Rate statistics of Sample
  - Requires a confirmation retry before deciding to increase or
    decrease this is configurable.
  - Allows the user to disable Sampling or specify a sample interval
  - Added Code Coverage of ProxDurationRunner based on YARDSTICK-1199

Change-Id: I27242ac1849c9a2712866385b5fbc05977c71516
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoMerge "Bugfix: HA kill process recovery has a conflict"
Abhijit Sinha [Tue, 26 Jun 2018 15:15:02 +0000 (15:15 +0000)]
Merge "Bugfix: HA kill process recovery has a conflict"

6 years agoMerge "NSB Topology fix for Prox 4 port test case"
Emma Foley [Tue, 26 Jun 2018 14:30:20 +0000 (14:30 +0000)]
Merge "NSB Topology fix for Prox 4 port test case"

6 years agoMerge "Refactor "utils.parse_ini_file" testing"
Rodolfo Alonso Hernandez [Tue, 26 Jun 2018 09:53:14 +0000 (09:53 +0000)]
Merge "Refactor "utils.parse_ini_file" testing"

6 years agoMerge "Refactor remote command execution in pktgen_dpdk"
Emma Foley [Tue, 26 Jun 2018 09:38:34 +0000 (09:38 +0000)]
Merge "Refactor remote command execution in pktgen_dpdk"

6 years agoExtended Context class with get_physical_nodes functionality 17/58417/15
Chornyi, TarasX [Mon, 11 Jun 2018 12:18:54 +0000 (15:18 +0300)]
Extended Context class with get_physical_nodes functionality

JIRA: YARDSTICK-1255

Change-Id: I446d715dc6cf716a4fcbc1b809c1b1d6303071e0
Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
6 years agoMerge "Add unit test file for DurationRunner"
Rodolfo Alonso Hernandez [Tue, 26 Jun 2018 07:43:20 +0000 (07:43 +0000)]
Merge "Add unit test file for DurationRunner"

6 years agoMerge "Refactor remote command execution in pktgen"
Rex Lee [Tue, 26 Jun 2018 06:35:55 +0000 (06:35 +0000)]
Merge "Refactor remote command execution in pktgen"

6 years agoNSB Topology fix for Prox 4 port test case 49/58349/8
Abhijit Sinha [Thu, 7 Jun 2018 18:59:08 +0000 (19:59 +0100)]
NSB Topology fix for Prox 4 port test case

The uplink and downlink interfaces for the 4 port
test cases were inconsistent and fixed in this commit.

This affects all the 4 port PROX test cases.

Tested on: Heat L2fwd-4 and L3fwd-4 test case.

JIRA: YARDSTICK-1185

Change-Id: Ia2ce7ed0dc395b812f84ede94e259b42dc833579
Signed-off-by:Abhijit Sinha <abhijit.sinha@intel.com>

6 years agoBugfix: HA kill process recovery has a conflict 13/58713/4
rexlee8776 [Tue, 19 Jun 2018 10:40:20 +0000 (10:40 +0000)]
Bugfix: HA kill process recovery has a conflict

It happens in Nokia SUT when running in the Plugfest. The problem happens when
the start_process start to recover the killed process (like nova-api), but the
self-cured mechanism already recovered. And somehow it lead to a conflict and
has problems.
So the recover of the HA attack-recover should be improved to only recover it
when it needs to.

JIRA: YARDSTICK-1222

Change-Id: I1acb5a7d59d6fe4e0de0b0c5942fa89e051dd1ff
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoAdd unit test file for DurationRunner 13/57813/7
Miikka Koistinen [Thu, 24 May 2018 12:30:12 +0000 (15:30 +0300)]
Add unit test file for DurationRunner

JIRA: YARDSTICK-1199

Change-Id: I5526935207ae3661a4cb8b8d25e93bc26adc3ef2
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoMerge "Cleanup CGNAPT unit tests"
Rodolfo Alonso Hernandez [Mon, 25 Jun 2018 11:27:25 +0000 (11:27 +0000)]
Merge "Cleanup CGNAPT unit tests"

6 years agoMerge "Add "volumeMounts" parameter in Kubernetes context"
Rodolfo Alonso Hernandez [Mon, 25 Jun 2018 11:19:12 +0000 (11:19 +0000)]
Merge "Add "volumeMounts" parameter in Kubernetes context"

6 years agoMerge "Add "volumes" parameter in Kubernetes context"
Rodolfo Alonso Hernandez [Mon, 25 Jun 2018 11:19:04 +0000 (11:19 +0000)]
Merge "Add "volumes" parameter in Kubernetes context"

6 years agoChange IxNextgen library load 71/58971/2
Rodolfo Alonso Hernandez [Mon, 25 Jun 2018 07:47:00 +0000 (08:47 +0100)]
Change IxNextgen library load

Changed IxNextgen library load in IXIA RFC2544 traffic generator call.

JIRA: YARDSTICK-1116

Change-Id: Icbdc5f1255dd2676584f3a1be767287e55e71f4a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Configure ACL via static file"
Abhijit Sinha [Fri, 22 Jun 2018 13:24:20 +0000 (13:24 +0000)]
Merge "Configure ACL via static file"

6 years agoConfigure ACL via static file 09/58109/9
Mytnyk, Volodymyr [Thu, 31 May 2018 09:03:10 +0000 (10:03 +0100)]
Configure ACL via static file

This patch allows user to configure ACL/vFW SampleVNF ACL
via configuration file provided in TC definition. The
Yardstick applies the rules to SampleVNF from specified config
file + rules generated by Yardstick (default rules).

The example of SampleVNF ACL CLI commands generated/applied
by Yardstick can be found at (using default ACL config file):

ACL VNF ACL CLI commands:
  acl/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
  http://paste.openstack.org/show/723303/

vFW VNF ACL CLI commands:
  vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
  http://paste.openstack.org/show/723304/

Change-Id: I76a630261a982083b628e3985fc3bec14ca495db
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoMerge "Cleanup EnvCommand test cases"
Rodolfo Alonso Hernandez [Thu, 21 Jun 2018 13:28:35 +0000 (13:28 +0000)]
Merge "Cleanup EnvCommand test cases"

6 years agoMerge "Add scale out TCs with availability zone support"
Rodolfo Alonso Hernandez [Thu, 21 Jun 2018 13:06:26 +0000 (13:06 +0000)]
Merge "Add scale out TCs with availability zone support"

6 years agoMerge "Document for Fraser test case results"
Jing Lu [Thu, 21 Jun 2018 02:35:29 +0000 (02:35 +0000)]
Merge "Document for Fraser test case results"

6 years agoNSB setup inventory name changed 53/58753/1
Rodolfo Alonso Hernandez [Wed, 20 Jun 2018 10:26:06 +0000 (11:26 +0100)]
NSB setup inventory name changed

NSB setup script [1] fails when retrieving the inventory file. This script
is going to be deprecated but we still support it.

[1] https://github.com/opnfv/yardstick/blob/master/nsb_setup.sh

JIRA: YARDSTICK-1247

Change-Id: I3798531652b0e3a1a007d1d9434998fdd54645d7
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Refactor RFC2455 TRex traffic profile injection"
Rodolfo Alonso Hernandez [Wed, 20 Jun 2018 07:37:34 +0000 (07:37 +0000)]
Merge "Refactor RFC2455 TRex traffic profile injection"

6 years agoMerge "Add "duration" parameter to test case definition"
Rodolfo Alonso Hernandez [Wed, 20 Jun 2018 07:37:29 +0000 (07:37 +0000)]
Merge "Add "duration" parameter to test case definition"

6 years agoMerge "Update pmd/lcore mask for OVS-DPDK context"
Emma Foley [Tue, 19 Jun 2018 08:58:27 +0000 (08:58 +0000)]
Merge "Update pmd/lcore mask for OVS-DPDK context"

6 years agoRefactor remote command execution in pktgen_dpdk 93/58493/4
Miikka Koistinen [Tue, 12 Jun 2018 13:19:16 +0000 (16:19 +0300)]
Refactor remote command execution in pktgen_dpdk

PktgenDPDKLatency.run will raise a RuntimeError if a remote command
fails. This commit makes it use the already existing exception raising
mechanism in yardstick's ssh client.

JIRA: YARDSTICK-1166

Change-Id: I3a3c7691399044b174f5d040c015c9b907b2fe5d
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoRefactor remote command execution in pktgen 11/57311/5
Miikka Koistinen [Wed, 9 May 2018 14:25:51 +0000 (17:25 +0300)]
Refactor remote command execution in pktgen

Some remote commands in Pktgen are executed without checking the exit
status. This patch makes all remote commands check exit value, and
removes unused variables that are captured from remote command
executions.

JIRA: YARDSTICK-1166

Change-Id: I42a667ebd22d086887d61e1671bc569b03c59d33
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoUpdate pmd/lcore mask for OVS-DPDK context 53/58653/2
Stepan Andrushko [Mon, 18 Jun 2018 11:23:16 +0000 (14:23 +0300)]
Update pmd/lcore mask for OVS-DPDK context

Added default values for pmd/lcore mask to the SampleVNF test case for
Trex TG.
lcore mask - specifies the CPU cores used for non-datapath OVS-DPDK
threads.
pmd mask - specifies the CPU cores used for data path packet processing.
Updated documentation for Standalone OVS-DPDK context.

JIRA: YARDSTICK-1246

Change-Id: I8d008a185c94d2047ffa4769d7974bc4d283fd7f
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
6 years agoCleanup EnvCommand test cases 21/58421/2
Rodolfo Alonso Hernandez [Mon, 11 Jun 2018 13:58:49 +0000 (14:58 +0100)]
Cleanup EnvCommand test cases

Refactor exception testing and remove console output.

JIRA: YARDSTICK-1235

Change-Id: I6376c3637ffb6476a2578ef963cd2420b63bab52
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoRefactor RFC2455 TRex traffic profile injection 29/57929/26
Rodolfo Alonso Hernandez [Thu, 24 May 2018 13:02:03 +0000 (14:02 +0100)]
Refactor RFC2455 TRex traffic profile injection

The TRex RFC2455 traffic profile configures the duration of the traffic
injection. Once the traffic is started, the traffic profile should poll
the traffic injector client for the injection status. When the injection
is finished, the traffic execution should end.

Instead of this, the traffic profile waits a fixed time (using time.sleep
method) and then stops the traffic. This approach is not accurate and may
cause shorter injection periods.

JIRA: YARDSTICK-1197

Change-Id: I5090df81ef4ec7945ff6c1aff070656b48e0fe77
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoAdd "duration" parameter to test case definition 59/57759/11
Rodolfo Alonso Hernandez [Wed, 23 May 2018 08:16:39 +0000 (09:16 +0100)]
Add "duration" parameter to test case definition

Add "duration" parameter to test case definition, in scenario:options
section. This parameter will be rendered in the traffic profile.

If the parameter is not present in the test case scenario options, the
default time written in the traffic profile options will be 30 seconds
(TrafficProfile.DEFAULT_DURATION = 30). If the traffic profile
injection time is not defined, the default injection time will be 30
seconds.

  testcase:scenario:options:duration (default = 30)
      render --> traffic_profile:duration
          parse --> TrafficProfile.duration (default = 30)

Target traffic profiles (RFC2544):
  - RFC2544Profile
  - IXIARFC2544Profile

JIRA: YARDSTICK-1194

Change-Id: I968922e6bb882d7ee15aa1c4db4037face7a3492
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge changes from topic 'convert-sla-asserts-to-raises'
Abhijit Sinha [Mon, 18 Jun 2018 10:54:49 +0000 (10:54 +0000)]
Merge changes from topic 'convert-sla-asserts-to-raises'

* changes:
  Replace MagicMocks with Mocks from SearchRunner UTs
  Add vsperf_dpdk SLA validation unit test cases
  Cleanup vsperf unit tests

6 years agoMerge "Convert SLA asserts to raises"
Abhijit Sinha [Mon, 18 Jun 2018 10:53:37 +0000 (10:53 +0000)]
Merge "Convert SLA asserts to raises"

6 years agoStop running tests from tests/unit 67/52367/14
Emma Foley [Tue, 20 Feb 2018 16:19:50 +0000 (16:19 +0000)]
Stop running tests from tests/unit

Since all tests have been moved into yardstick/tests,
temperary code to run tests in both tests and yardstick/tests
should be removed.

JIRA: YARDSTICK-837
Change-Id: I3bf3c9ece9bc8457d992e53ee8cc5cadb1dec8f0
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoMerge "Cleanup BaseMonitor unit tests"
Rodolfo Alonso Hernandez [Mon, 18 Jun 2018 08:42:12 +0000 (08:42 +0000)]
Merge "Cleanup BaseMonitor unit tests"

6 years agoCleanup BaseMonitor unit tests 75/58375/3
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 15:43:40 +0000 (16:43 +0100)]
Cleanup BaseMonitor unit tests

Stop multiprocessing queues to avoid BrokenPipe exception.
Assert correctly the raise of exceptions.

JIRA: YARDSTICK-1229

Change-Id: I1a1c92c49a439283a35c9a69b59d6866f35ec09a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoReplace MagicMocks with Mocks from SearchRunner UTs 23/57323/8
Miikka Koistinen [Fri, 11 May 2018 12:20:24 +0000 (15:20 +0300)]
Replace MagicMocks with Mocks from SearchRunner UTs

This patch replaces unneeded MagicMocks with Mocks from TestSearchRunner
and TestSearchRunnerHelper.

JIRA: YARDSTICK-1169

Change-Id: I30a7bbc71c0b1062e6c0dfe634a5a8f1ecb7c684
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoConvert SLA asserts to raises 61/57061/9
Miikka Koistinen [Fri, 8 Jun 2018 10:54:48 +0000 (13:54 +0300)]
Convert SLA asserts to raises

This commit converts Python assertions to a custom exception in all
places where SLA validation is checked with an assertion.

This commit also fixes all emerged pylint errors.

JIRA: YARDSTICK-966

Change-Id: If771ed03b2cbc0a43a57fcfb9293f18740b3ff80
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoAdd vsperf_dpdk SLA validation unit test cases 73/58373/4
Miikka Koistinen [Fri, 8 Jun 2018 14:53:29 +0000 (17:53 +0300)]
Add vsperf_dpdk SLA validation unit test cases

JIRA: YARDSTICK-1228

Change-Id: Ic32f2bcd7f8bdf718c9c266666409d32ecab4924
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoCleanup vsperf unit tests 71/58371/4
Miikka Koistinen [Fri, 8 Jun 2018 14:42:41 +0000 (17:42 +0300)]
Cleanup vsperf unit tests

Remove unneeded mock return values and improve the test execution.

Add unit test cases about SLA validation.

JIRA: YARDSTICK-1227

Change-Id: I90b8b9c4cb6a57ae4b0869e84af0da43ee22a670
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoMerge "assert[Greater,Equal] -> assert_{,not_}called"
Rodolfo Alonso Hernandez [Fri, 15 Jun 2018 08:41:37 +0000 (08:41 +0000)]
Merge "assert[Greater,Equal] -> assert_{,not_}called"

6 years agoBugfix: heat conext of test case yamls jinja2 bypass sriov type 21/58521/4
rexlee8776 [Thu, 14 Jun 2018 03:35:46 +0000 (03:35 +0000)]
Bugfix: heat conext of test case yamls jinja2 bypass sriov type

the "if provider" condition bypass when provider = "sriov",
related to https://gerrit.opnfv.org/gerrit/#/c/32789/

  networks:
    test:
      cidr: '10.0.1.0/24'
      {% if provider == "vlan" %}
      provider: {{provider}}
      physical_network: {{physical_network}}
        {% if segmentation_id %}
      segmentation_id: {{segmentation_id}}
        {% endif %}
      {% endif %}

JIRA: YARDSTICK-1242

Change-Id: I6a611e839654f401648d9bd6188589816f7b4946
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoMerge "Add "host_name_separator" variable to Context class"
Rodolfo Alonso Hernandez [Thu, 14 Jun 2018 17:07:41 +0000 (17:07 +0000)]
Merge "Add "host_name_separator" variable to Context class"

6 years agoMerge "Install dependencies: bare-metal, standalone"
Emma Foley [Thu, 14 Jun 2018 14:50:01 +0000 (14:50 +0000)]
Merge "Install dependencies: bare-metal, standalone"

6 years agoInstall dependencies: bare-metal, standalone 89/57389/5
Stepan Andrushko [Tue, 15 May 2018 09:46:42 +0000 (12:46 +0300)]
Install dependencies: bare-metal, standalone

This will install all dependencies needed on the bare-metal and standalone
physical nodes.

JIRA: YARDSTICK-1177

Change-Id: I719ca64d4da098d6dd60597e5fc4ad6f4715f79e
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoStop time.sleep mocks in unit tests 33/58533/1
Rodolfo Alonso Hernandez [Thu, 14 Jun 2018 13:14:27 +0000 (14:14 +0100)]
Stop time.sleep mocks in unit tests

tVpeApproxVnf and TestProxSocketHelper classes mock "time.sleep" in
the setUp method without deleting it at the end of the test execution.

JIRA: YARDSTICK-1243

Change-Id: Iff31d9c7b400ad8a47f37792aeb0d20328b9d9e1
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoDefine several containers per pod in a replication controller 89/57089/8
Rodolfo Alonso Hernandez [Sat, 5 May 2018 10:13:28 +0000 (11:13 +0100)]
Define several containers per pod in a replication controller

Add the ability to define not only one but many containers per pod in a
replication controller descriptor. This feature must be backwards
compatible; all current test cases using the "single container" server
definition must be accepted.

Example of single container pod definition:

  context:
    type: Kubernetes
    servers:
      host:
        image: ...
        commands: ...
        volumes:
          - name: volume1              # mandatory
            <volume type definition>   # mandatory

Example of several container pod definition:

  context:
    type: Kubernetes
    servers:
      host:
        containers:  # if this key is present, all container specific
                       parameters (image, commands, args, volumeMounts, etc.)
                       must be defined per container
          - image: ...
            commands: ...
          - image: ...
            commands: ...
        volumes:
          - name: volume1              # mandatory
            <volume type definition>   # mandatory

NOTE: other parameters, like "volumes" or "nodeSelector", are common to all
containers in the pod.

JIRA: YARDSTICK-1155

Change-Id: Ib95668c68e9c09e6de3f1aa41c903cc52e6809ad
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoAdd "volumeMounts" parameter in Kubernetes context 71/57071/12
Rodolfo Alonso Hernandez [Fri, 4 May 2018 15:38:52 +0000 (16:38 +0100)]
Add "volumeMounts" parameter in Kubernetes context

This new parameter, "volumeMounts", will allow the user to automatically
create new volumes mounted inside the container. Example of Kubernetes
context definition:

  context:
    type: Kubernetes
    servers:
      host:
        volumeMounts:
          - name: volume1              # mandatory
            mountPath: /dev/hugepages  # mandatory
            readOnly: True             # optional, default=False

JIRA: YARDSTICK-1151

Change-Id: Ic00b45e6d603a9f85e3f0e25becdf0bce14e637c
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoAdd "volumes" parameter in Kubernetes context 57/57057/14
Rodolfo Alonso Hernandez [Fri, 4 May 2018 11:55:52 +0000 (12:55 +0100)]
Add "volumes" parameter in Kubernetes context

This new parameter, "volumes", will allow the user to automatically create
new volumes. Example of Kubernetes context definition:

  context:
    type: Kubernetes
    servers:
      host:
        image: ...
        commands: ...
        volumes:
          - name: volume1              # mandatory
            <volume type definition>   # mandatory

The volume type and the definition must be one of the supported ones in
Kubernetes [1].

[1] https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes

JIRA: YARDSTICK-1152

Change-Id: I44a91c605f047de4f286407e28fb5aa2e921b00a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoassert[Greater,Equal] -> assert_{,not_}called 93/53593/8
Emma Foley [Mon, 12 Mar 2018 15:07:59 +0000 (15:07 +0000)]
assert[Greater,Equal] -> assert_{,not_}called

assertEqual(mock_xxx.call_count, 1) -> mock_xxx.assert_called_once
assertEqual(mock_xxx.call_count, 0) -> mock_xxx.assert_not_called
assertGreater(mock.call_count, 0) -> mock.assert_called()
assertGreaterEqual(mock.call_count, 1) -> mock.assert_called()

JIRA: YARDSTICK-1069
Change-Id: I890084d120c8e78304e169e2a0e5d30011a41525
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoImprove IXIA IxNetwork library and traffic profile (4) 93/55593/10
Rodolfo Alonso Hernandez [Thu, 12 Apr 2018 14:08:58 +0000 (15:08 +0100)]
Improve IXIA IxNetwork library and traffic profile (4)

This patch implements an active wait for the traffic injection. Once the
traffic is started, the traffic generator class will poll periodically the
IXIA traffic generator chassis to retrieve the status of the traffic
("started", "stopped").

Now the latency statistics are retrieved and reported for each injection
period.

JIRA: YARDSTICK-1116

Change-Id: I4422e2c88b4fc97b7cac3de8a82b2d75467c4117
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoImprove IXIA IxNetwork library and traffic profile (3) 35/55435/10
Rodolfo Alonso Hernandez [Wed, 11 Apr 2018 17:32:52 +0000 (18:32 +0100)]
Improve IXIA IxNetwork library and traffic profile (3)

This patch modifies IP packet parameters.

"IxNextgen.update_ip_packet" modifies the L3 packet according to the
test case and setup the IP addresses.

JIRA: YARDSTICK-1116

Change-Id: I46ff75ab1989d0e6f5cc876418a015386717e06f
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoImprove IXIA IxNetwork library and traffic profile (2) 21/55321/14
Rodolfo Alonso Hernandez [Tue, 10 Apr 2018 08:15:55 +0000 (09:15 +0100)]
Improve IXIA IxNetwork library and traffic profile (2)

This patch modifies the way the packet frame and the flow is configured
before the injection.

"IxNextgen.update_frame" modifies the L2 frame according to the
test case and setup the frame rate, frame size, traffic injection
duration and MAC addresses.

JIRA: YARDSTICK-1116

Change-Id: Ife08f15a4eda24d7835c92c4172b450854d112ee
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoImprove IXIA IxNetwork library and traffic profile (1) 11/55311/15
Rodolfo Alonso Hernandez [Tue, 10 Apr 2018 07:39:16 +0000 (08:39 +0100)]
Improve IXIA IxNetwork library and traffic profile (1)

This patch introduces a new way to configure the TCL IxNetwork server.
All the configuration is done using the TCL API, removing the need of
using the pre-saved configuration file.

"IxNextgen.assign_ports" creates and assigns the virtual ports for each
physical port defined in the test case.

"IxNextgen.create_traffic_item" creates one traffic item and two flow
groups per pair of ports, in both directions.

"IxNextgen.gt_statistics" retrieves only the required statistics to
generate the samples blob in the traffic generator.

JIRA: YARDSTICK-1116

Change-Id: I8f1c0c55e99c274b2ed8276ed9a385c502e16d93
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoMerge changes from topics 'YARDSTICK-1218', 'YARDSTICK-1216', 'YARDSTICK-1215', ...
Rodolfo Alonso Hernandez [Tue, 12 Jun 2018 07:27:51 +0000 (07:27 +0000)]
Merge changes from topics 'YARDSTICK-1218', 'YARDSTICK-1216', 'YARDSTICK-1215', 'YARDSTICK-1214'

* changes:
  Move IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptions
  Move ErrorClass definition to exceptions module
  Convert SSH custom exceptions to Yardstick exceptions
  Remove AnsibleCommon class method mock

6 years agoCleanup CGNAPT unit tests 69/58369/4
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 13:11:05 +0000 (14:11 +0100)]
Cleanup CGNAPT unit tests

Remove unneeded mocks, clean console output and improve the
test execution.

JIRA: YARDSTICK-1226

Change-Id: I83e91028c17208a39d3813e85b44c4df1961e01e
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoCleanup process test cases 09/58409/2
Rodolfo Alonso Hernandez [Mon, 11 Jun 2018 09:58:37 +0000 (10:58 +0100)]
Cleanup process test cases

Remove logging console output during test execution.

JIRA: YARDSTICK-1234

Change-Id: I8ecab0219ffd1919f2fa97a90d5be11018b11322
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoCleanup OpenStack utils test cases 07/58407/1
Rodolfo Alonso Hernandez [Mon, 11 Jun 2018 07:57:32 +0000 (08:57 +0100)]
Cleanup OpenStack utils test cases

Remove logging console output during test execution.

JIRA: YARDSTICK-1230

Change-Id: I1b61f0f18bfb6e5a63c9027537cfb736261c4926
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMove tests: unit/network_services/vnf_generic 71/56771/8
Emma Foley [Fri, 27 Apr 2018 10:30:20 +0000 (10:30 +0000)]
Move tests: unit/network_services/vnf_generic

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

JIRA: YARDSTICK-837

Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Change-Id: I3d95537a64bde923703421510d27bac0d6dd92bd

6 years agoRefactor "utils.parse_ini_file" testing 47/58247/2
Rodolfo Alonso Hernandez [Tue, 5 Jun 2018 14:44:30 +0000 (15:44 +0100)]
Refactor "utils.parse_ini_file" testing

JIRA: YARDSTICK-1218

Change-Id: I5136b22110d215beb7af72998ad47a8dba27f63b
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Remove print out of logger exception in TestUtils"
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 13:11:44 +0000 (13:11 +0000)]
Merge "Remove print out of logger exception in TestUtils"

6 years agoMerge "Cleanup TestYardstickNSCli class"
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 13:11:38 +0000 (13:11 +0000)]
Merge "Cleanup TestYardstickNSCli class"

6 years agoMerge "Remove __init__ method overriding in HeatContextTestCase"
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 10:38:59 +0000 (10:38 +0000)]
Merge "Remove __init__ method overriding in HeatContextTestCase"

6 years agoMerge "Standalone XML machine type is not longer valid"
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 10:38:42 +0000 (10:38 +0000)]
Merge "Standalone XML machine type is not longer valid"

6 years agoMerge "Error when adding SR-IOV interfaces in SR-IOV context"
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 10:38:35 +0000 (10:38 +0000)]
Merge "Error when adding SR-IOV interfaces in SR-IOV context"

6 years agoRemove print out of logger exception in TestUtils 43/58343/3
Rodolfo Alonso Hernandez [Thu, 7 Jun 2018 16:50:30 +0000 (17:50 +0100)]
Remove print out of logger exception in TestUtils

Remove print out of logger exception while executing
"TestUtils.test_parse_ini_file_missing_section_header" test case.

JIRA: YARDSTICK-1225

Change-Id: I2a4e447c5d119fb9585b477d74f3d0a002c8278e
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoCleanup TestYardstickNSCli class 41/58341/3
Rodolfo Alonso Hernandez [Thu, 7 Jun 2018 16:13:09 +0000 (17:13 +0100)]
Cleanup TestYardstickNSCli class

Remove the console output during the test execution. Refactor the
"parse_arguments" test case.

JIRA: YARDSTICK-1224

Change-Id: I68954b986dbea70752adb121e0445e162bdfddcb
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Fix latency in l2 and l3 PROX generator config files"
Abhijit Sinha [Thu, 7 Jun 2018 12:06:13 +0000 (12:06 +0000)]
Merge "Fix latency in l2 and l3 PROX generator config files"

6 years agoMerge "Use TRex release v2.41 to support both x86 and aarch64"
Rodolfo Alonso Hernandez [Thu, 7 Jun 2018 11:58:07 +0000 (11:58 +0000)]
Merge "Use TRex release v2.41 to support both x86 and aarch64"

6 years agoMerge "Remove instantiated contexts in "test_task""
Rodolfo Alonso Hernandez [Thu, 7 Jun 2018 11:57:31 +0000 (11:57 +0000)]
Merge "Remove instantiated contexts in "test_task""

6 years agoFix latency in l2 and l3 PROX generator config files 35/58135/3
Xavier Simonart [Fri, 1 Jun 2018 08:55:33 +0000 (10:55 +0200)]
Fix latency in l2 and l3 PROX generator config files

JIRA: YARDSTICK-1211

Change-Id: I0899d470ca5a0ec7d42d36a9ff9c39cc7369310c
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
6 years agoMerge "Add NSB PROX Baremetal dashboard for L3"
Abhijit Sinha [Wed, 6 Jun 2018 12:44:03 +0000 (12:44 +0000)]
Merge "Add NSB PROX Baremetal dashboard for L3"

6 years agoMerge "Fix NSB NFVi metrics accuracy"
Abhijit Sinha [Wed, 6 Jun 2018 12:43:51 +0000 (12:43 +0000)]
Merge "Fix NSB NFVi metrics accuracy"