yardstick.git
6 years agoNSB NFVi PROX Missing VNF Statistics for CPU Utilization 97/61597/2
DanielMartinBuckley [Thu, 30 Aug 2018 13:07:31 +0000 (14:07 +0100)]
NSB NFVi PROX Missing VNF Statistics for CPU Utilization

JIRA: YARDSTICK-1404

The test scripts require an extra argument to support collectd.

Also the CPU utilization is a different variable name structure
in the JSON file.

Change-Id: I127b7e86c0f93f7743331a2d71eb81d690e42471
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoBugfix: ansible version should be the same both in Dockerfile and requirements.txt 99/61499/1
rexlee8776 [Wed, 29 Aug 2018 06:13:58 +0000 (06:13 +0000)]
Bugfix: ansible version should be the same both in Dockerfile and requirements.txt

Because Dockerfile is using ansible script to install yardstick requirements.
Or it may lead to exception when installing Yardstick.

Change-Id: Ia83cf76675a7f3f1d0aa5c311e7b72be5c7ca5c3
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoMerge "Bugfix: openstack router list broken in Yardstick container - dovetail"
Rex Lee [Tue, 28 Aug 2018 11:15:50 +0000 (11:15 +0000)]
Merge "Bugfix: openstack router list broken in Yardstick container - dovetail"

6 years agoMerge "Change tc_trex files to execute standalone tests"
Rodolfo Alonso Hernandez [Mon, 27 Aug 2018 16:00:31 +0000 (16:00 +0000)]
Merge "Change tc_trex files to execute standalone tests"

6 years agoMerge "Make TRex IMIX traffic mode configurable"
Rodolfo Alonso Hernandez [Mon, 27 Aug 2018 08:01:32 +0000 (08:01 +0000)]
Merge "Make TRex IMIX traffic mode configurable"

6 years agoping bottlenecks failed when security group rule do not support ipv6 - dovetail 19/60919/3
rexlee8776 [Tue, 14 Aug 2018 04:15:49 +0000 (04:15 +0000)]
ping bottlenecks failed when security group rule do not support ipv6 - dovetail

JIRA: YARDSTICK-1380

Change-Id: I2163eeb9ab17e4f0216ddaed688e78acdf51a1ff
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoMerge "Make security group configurable - dovetail"
Jing Lu [Mon, 27 Aug 2018 01:49:38 +0000 (01:49 +0000)]
Merge "Make security group configurable - dovetail"

6 years agoBugfix: openstack router list broken in Yardstick container - dovetail 15/61315/1
rexlee8776 [Fri, 24 Aug 2018 07:04:59 +0000 (07:04 +0000)]
Bugfix: openstack router list broken in Yardstick container - dovetail

Dependancy of python-openstackclient is not constrained.
This patch specify openstacksdk and openstackclient's version according to [1].

This patch apply for bugfix and need to backported to fraser stable branch.
Continual work need to be done in Gambia according to YARDSTICK-1398.

[1]
https://github.com/openstack/requirements/blob/stable/pike/upper-constraints.txt

JIRA: DOVETAIL-728

Change-Id: I57c718e192699ac322150458d18508b06c93d22b
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoMake TRex IMIX traffic mode configurable 97/61297/2
Rodolfo Alonso Hernandez [Thu, 23 Aug 2018 14:09:08 +0000 (15:09 +0100)]
Make TRex IMIX traffic mode configurable

Enable TRex IMIX generation using two modes:
  - Distribution in packages.
  - Distribution in byes.

The default one will be distribution in packages, like the default
one in IxNetwork.

JIRA: YARDSTICK-1397

Change-Id: I86b2527968496218b0515349fc0a2680a84261e5
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Fix setting `flow` configuration in TC yaml"
Rodolfo Alonso Hernandez [Thu, 23 Aug 2018 16:13:48 +0000 (16:13 +0000)]
Merge "Fix setting `flow` configuration in TC yaml"

6 years agoMerge "Fix issues with 'Success_' stats reported by ProxBinSearchProfile"
Abhijit Sinha [Thu, 23 Aug 2018 13:39:34 +0000 (13:39 +0000)]
Merge "Fix issues with 'Success_' stats reported by ProxBinSearchProfile"

6 years agoMerge "Runners crash on test startup."
Abhijit Sinha [Thu, 23 Aug 2018 13:38:54 +0000 (13:38 +0000)]
Merge "Runners crash on test startup."

6 years agoMerge "Provide access to VM in Standalone context"
Rodolfo Alonso Hernandez [Thu, 23 Aug 2018 13:26:22 +0000 (13:26 +0000)]
Merge "Provide access to VM in Standalone context"

6 years agoMerge "Provide average latency results in IXIA KPIs"
Rodolfo Alonso Hernandez [Thu, 23 Aug 2018 09:46:02 +0000 (09:46 +0000)]
Merge "Provide average latency results in IXIA KPIs"

6 years agoRunners crash on test startup. 77/61177/5
DanielMartinBuckley [Tue, 21 Aug 2018 13:17:25 +0000 (14:17 +0100)]
Runners crash on test startup.

JIRA: YARDSTICK-1393

The problem resides in this traffic generator class: this class
is inheriting from SampleVNFTrafficGen and overriding the
__init__ method but is NOT calling the parent class __init__
one.

Change-Id: I206a66e361a3e2eb50bb5fa01ddadae25b4a9f54
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoProvide access to VM in Standalone context 51/59151/14
Stepan Andrushko [Wed, 27 Jun 2018 18:43:18 +0000 (21:43 +0300)]
Provide access to VM in Standalone context

Add possibility to access VM with SampleVNF image, which is spawned on
baremetal server using password or key pairs (provided or generated).
Changes provided for OVS-DPDK and SRIOV cases.

JIRA: YARDSTICK-1260

Change-Id: I89cf17dc83f02abe23d206b24b9d0ec1298d3231
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
6 years agoFix issues with 'Success_' stats reported by ProxBinSearchProfile 51/61051/7
Serhiy Pshyk [Fri, 17 Aug 2018 15:35:41 +0000 (16:35 +0100)]
Fix issues with 'Success_' stats reported by ProxBinSearchProfile

List of changes:
  - Added 'Status' field to indicate status of each search iteration.
    Possible values are 'Success', 'Failure', 'Result'.
  - Removed reporting of duplicated stats with 'Success_' prefix.
    Instead 'Status' field is set to 'Success'
  - Removed reporting of duplicated stats with 'Result_' prefix.
    Instead 'Status' field is set to 'Result'
  - Updated Prox_BM_L2FWD-4Port_MultiSize-1518452496550.json dashboard
    to query success and result status by 'Status' field value.

JIRA: YARDSTICK-1386

Change-Id: I9a21a102af6cb4db137e805011b5d81c4943d77b
Signed-off-by: Serhiy Pshyk <serhiyx.pshyk@intel.com>
6 years agoProvide average latency results in IXIA KPIs 55/61255/1
Rodolfo Alonso Hernandez [Wed, 22 Aug 2018 14:43:55 +0000 (15:43 +0100)]
Provide average latency results in IXIA KPIs

IXIA KPIs will provide an average value of all port latency values:
max, min and avg.

JIRA: YARDSTICK-1394

Change-Id: Ib7c7b5300e8343a1af0a20cb4e49c0bd669c1143
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Add virtual image generation to install script"
Rodolfo Alonso Hernandez [Tue, 21 Aug 2018 17:08:30 +0000 (17:08 +0000)]
Merge "Add virtual image generation to install script"

6 years agoMerge "Improvie TRex RFC2544 throughput calculation"
Rodolfo Alonso Hernandez [Tue, 21 Aug 2018 14:45:20 +0000 (14:45 +0000)]
Merge "Improvie TRex RFC2544 throughput calculation"

6 years agoMerge "Convert IXIA latency statistics to integer"
Rodolfo Alonso Hernandez [Tue, 21 Aug 2018 12:54:39 +0000 (12:54 +0000)]
Merge "Convert IXIA latency statistics to integer"

6 years agoConvert IXIA latency statistics to integer 17/61117/2
Rodolfo Alonso Hernandez [Mon, 20 Aug 2018 14:27:50 +0000 (15:27 +0100)]
Convert IXIA latency statistics to integer

JIRA: YARDSTICK-1385

Change-Id: Id50c393da7ded4b3c8e127f7d7a6501832a68446
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Make TRex latency statistics optional"
Rodolfo Alonso Hernandez [Tue, 21 Aug 2018 09:33:21 +0000 (09:33 +0000)]
Merge "Make TRex latency statistics optional"

6 years agoMerge "NSB NFVi PROX BNG and vPE losing many packets"
Abhijit Sinha [Tue, 21 Aug 2018 08:55:56 +0000 (08:55 +0000)]
Merge "NSB NFVi PROX BNG and vPE losing many packets"

6 years agoMerge "the recovery action of "baremetal down" should be triggered mandatory"
Rex Lee [Tue, 21 Aug 2018 01:29:57 +0000 (01:29 +0000)]
Merge "the recovery action of "baremetal down" should be triggered mandatory"

6 years agoMake TRex latency statistics optional 15/61115/1
Rodolfo Alonso Hernandez [Mon, 20 Aug 2018 11:39:27 +0000 (12:39 +0100)]
Make TRex latency statistics optional

Because of performance issues in TRex due to the extra power needed to
retrieve the latency statistics, this parameter will be optional and disabled
by default.

JIRA: YARDSTICK-1388

Change-Id: I07c0ec7bef86fd438d078cbd6748b98bc8cc0b37
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoImprovie TRex RFC2544 throughput calculation 13/61013/2
Rodolfo Alonso Hernandez [Thu, 16 Aug 2018 11:20:10 +0000 (12:20 +0100)]
Improvie TRex RFC2544 throughput calculation

Improve TRex RFC2544 throughput calculation by using the in/out packets
instead of using the throughput samples.

JIRA: YARDSTICK-1387

Change-Id: I877e901c3226d31c6435a877a2924bb95deb1cf0
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoFix RFC2544 IXIA latency information 11/61111/1
Rodolfo Alonso Hernandez [Mon, 20 Aug 2018 09:33:30 +0000 (10:33 +0100)]
Fix RFC2544 IXIA latency information

Do not overwrite the samples information with the latency parameters.

JIRA: YARDSTICK-1385

Change-Id: I2f144800ad19d7736eedf5c7b6783731031c4bf4
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMake security group configurable - dovetail 13/60813/5
rexlee8776 [Wed, 8 Aug 2018 08:54:59 +0000 (08:54 +0000)]
Make security group configurable - dovetail

Make sercurity group can be configured via the context.
The format would be:
context:
  name: demo
  image: yardstick-image
  flavor: yardstick-flavor
  user: ubuntu
  security_group:
    rules:
        - remote_ip_prefix: "0.0.0.0/0"
          protocol: "tcp"
          port_range_min: 1
          port_range_max: 65535
        - remote_ip_prefix: "0.0.0.0/0"
          protocol: "udp"
          port_range_min: 1
          port_range_max: 65535
        - remote_ip_prefix: "0.0.0.0/0"
          protocol: "icmp"

JIRA: YARDSTICK-1360

Change-Id: I00c45767ee2d70b790590e824599d5a4c274bced
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoMerge "Improve IXIA TG Rx/TX throughput calculation"
Rodolfo Alonso Hernandez [Fri, 17 Aug 2018 08:48:22 +0000 (08:48 +0000)]
Merge "Improve IXIA TG Rx/TX throughput calculation"

6 years agoAdd RFC2544 IXIA latency information 29/61029/1
Rodolfo Alonso Hernandez [Fri, 17 Aug 2018 07:36:53 +0000 (08:36 +0100)]
Add RFC2544 IXIA latency information

JIRA: YARDSTICK-1385

Change-Id: Icecceaabc0389279a3a7154884c50259f794eb76
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "IXIA traffic stops running after first iteration"
Emma Foley [Thu, 16 Aug 2018 15:27:23 +0000 (15:27 +0000)]
Merge "IXIA traffic stops running after first iteration"

6 years agoMerge "Disk Initialization and Custom Workload"
Rex Lee [Thu, 16 Aug 2018 13:32:43 +0000 (13:32 +0000)]
Merge "Disk Initialization and Custom Workload"

6 years agoImprove IXIA TG Rx/TX throughput calculation 89/60889/3
Mytnyk, Volodymyr [Mon, 13 Aug 2018 10:00:00 +0000 (11:00 +0100)]
Improve IXIA TG Rx/TX throughput calculation

Use float instead of integer to convert TG throughtput
IXIA statistic.

JIRA: YARDSTICK-1378

Change-Id: I39bd143adc4b10035c6f0986a06d407a18aa0887
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoFix setting `flow` configuration in TC yaml 93/60993/4
Mytnyk, Volodymyr [Thu, 16 Aug 2018 07:40:04 +0000 (08:40 +0100)]
Fix setting `flow` configuration in TC yaml

The `flow` configuration is not applied to the traffic
profile if `seed` or `count` option is missing in the
TC definition.

- Fix `seed` backward compatibility.

JIRA: YARDSTICK-1383

Change-Id: Ic874fb2be1c13009f94eb3a80194bfe81d7a923b
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoAdd virtual image generation to install script 09/57809/8
Stepan Andrushko [Thu, 24 May 2018 10:13:40 +0000 (13:13 +0300)]
Add virtual image generation to install script

Update install.yaml with image generation: nsb and normal.

JIRA: YARDSTICK-1198

Change-Id: I3b8773e9b3b9890ae8623bb6536d05f1151d84a8
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
6 years agoMerge "[docs] Remove the VTC chapter in the userguide"
Abhijit Sinha [Wed, 15 Aug 2018 13:07:51 +0000 (13:07 +0000)]
Merge "[docs] Remove the VTC chapter in the userguide"

6 years agoMerge "Fix KPI data reported by ProxBinSearchProfile"
Abhijit Sinha [Wed, 15 Aug 2018 12:48:39 +0000 (12:48 +0000)]
Merge "Fix KPI data reported by ProxBinSearchProfile"

6 years agoFix TRex RFC2544 traffic profile tolerance definition 41/60941/2
Rodolfo Alonso Hernandez [Tue, 14 Aug 2018 13:50:27 +0000 (14:50 +0100)]
Fix TRex RFC2544 traffic profile tolerance definition

In TRex RFC2544 traffic profile, the tolerance limit is not set correctly.
The parameters "tol_high" and "tol_low" in "get_drop_percentage" are
incorrect.

The tolerance limit should be reduced to 0.01.

JIRA: YARDSTICK-1382

Change-Id: If5fc1f3aec86effabd7903e0924724fe7eeb42ab
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoChange tc_trex files to execute standalone tests 47/60647/3
Stepan Andrushko [Mon, 6 Aug 2018 17:41:04 +0000 (20:41 +0300)]
Change tc_trex files to execute standalone tests

Changes made in test cases for trex TG files to be able to run tests
for standalone setup (OVS-DPDK/SRIOV).
- tg__0: trafficgen_1

Changes to improve templates for tc_trex:
- flavor: images: changed to default name: yardstick-nsb-image.img
- default path for files under contexts section
- RAM size

JIRA: YARDSTICK-1368

Change-Id: Ida85de219acd566840de1a94975b8050c7f59b9b
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
6 years agoMerge "Bugfix: pylint fix for yardstick/orchestrator/heat.py (no-value-for-parameter)"
Rex Lee [Tue, 14 Aug 2018 01:44:00 +0000 (01:44 +0000)]
Merge "Bugfix: pylint fix for yardstick/orchestrator/heat.py (no-value-for-parameter)"

6 years agoFix KPI data reported by ProxBinSearchProfile 99/60899/2
Serhiy Pshyk [Mon, 13 Aug 2018 14:16:44 +0000 (15:16 +0100)]
Fix KPI data reported by ProxBinSearchProfile

List of changes made in ProxBinSearchProfile:
  - Removed reporting of redundant records with zero values
  - Fixed issue with results data from one step reported in the next step
  - Added 'Step_Id' field to reflect to which search step current record belongs
  - Added 'Confirmation_Retry' field to reflect retry number of current test.
    This value depends on 'confirmation' configuration option of a runner.

JIRA: YARDSTICK-1379

Change-Id: Idc3f7e26ece41ac0dfbfee659daf310a2879e024
Signed-off-by: Serhiy Pshyk <serhiyx.pshyk@intel.com>
6 years agoMerge "Add QinQ support for IXIA traffic profile"
Abhijit Sinha [Fri, 10 Aug 2018 12:59:20 +0000 (12:59 +0000)]
Merge "Add QinQ support for IXIA traffic profile"

6 years agoIXIA traffic stops running after first iteration 05/60805/3
Mytnyk, Volodymyr [Thu, 9 Aug 2018 16:14:39 +0000 (17:14 +0100)]
IXIA traffic stops running after first iteration

IXIA traffic stops running after first iteration when
traffic profile duration is set to 60 sec.

JIRA: YARDSTICK-1374

Change-Id: I216b98d43b66aabc72ed88e895b89e28572b264f
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
Signed-off-by: Myron Sosyak <myronx.sosyak@intel.com>
6 years agoDisk Initialization and Custom Workload 13/59813/13
mbeierl [Fri, 13 Jul 2018 20:25:39 +0000 (16:25 -0400)]
Disk Initialization and Custom Workload

Removes the deprecated nowarm and nosdd parameters from the
call to StorPerf.

Adds a call to initialize the disk right after the stack
creation is complete so that the results from the
performance run are valid and not skewed by a disk that
is not operating at a consistent level of performance.

Adds the jobs API v2.0 when the custom parameter 'workloads'
is used instead of workload.

Improves documentation and adds more info about the
workload and workloads parameters.

Co-Authored-By: Ameed Ashour <Ameed.Ashour.Ext@nokia.com>
Change-Id: I17e96028f6a82e2c800c8f73c7be866e1cab6ad8
JIRA: YARDSTICK-1322
Signed-off-by: mbeierl <mark.beierl@dell.com>
6 years agoMerge "Add UT: DurationRunner worker exception handling"
Abhijit Sinha [Thu, 9 Aug 2018 16:28:54 +0000 (16:28 +0000)]
Merge "Add UT: DurationRunner worker exception handling"

6 years agoMerge "Add UT: DurationRunner worker normal operation"
Abhijit Sinha [Thu, 9 Aug 2018 16:28:50 +0000 (16:28 +0000)]
Merge "Add UT: DurationRunner worker normal operation"

6 years agoAdd QinQ support for IXIA traffic profile 39/60739/2
Myron Sosyak [Wed, 8 Aug 2018 10:46:46 +0000 (11:46 +0100)]
Add QinQ support for IXIA traffic profile

Implemented logic for parsing and applying QinQ in IXIA traffic profile.
For QinQ configuration add QinQ section to outerl2 section.
For example:
...
  outerl2:
    QinQ:
      S-VLAN:
        id: 128
        priority: 0
        cfi: 0
      C-VLAN:
        id: 512
        priority: 0
        cfi: 0
...

JIRA: YARDSTICK-1370

Change-Id: I29caa9eb7dd038e7de66faab316f03c7c29a742f
Signed-off-by: Myron Sosyak <myronx.sosyak@intel.com>
6 years agoBugfix: pylint fix for yardstick/orchestrator/heat.py (no-value-for-parameter) 57/60757/1
rexlee8776 [Thu, 9 Aug 2018 02:59:02 +0000 (02:59 +0000)]
Bugfix: pylint fix for yardstick/orchestrator/heat.py (no-value-for-parameter)

Problem
op_utils.get_volume_id takes 2 arguments and only pass 1. This would
cause tc006 fail!
volume_id = op_utils.get_volume_id(volume_name)
it will cause:
Running pylint...
You can speed this up by running it on 'HEAD~[0-9]' (e.g. HEAD~0, this
change only)...
Using config file /home/opnfv/repos/yardstick/.pylintrc
************* Module yardstick.orchestrator.heat
E:235,20: No value for argument 'volume_name' in function call
(no-value-for-parameter)

To Do
It doesn't need to call get_volume_id here, just use heat get_resource
would be enough. This fix should work with tc006 test case.

JIRA: YARDSTICK-1371

Change-Id: I913ffd085f742ae4464dbcc4269719e2c50e37b5
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoFix duration value for IXIA traffic item 17/60717/2
Myron Sosyak [Tue, 7 Aug 2018 16:31:03 +0000 (17:31 +0100)]
Fix duration value for IXIA traffic item

Removed code that overrides duration value in the update_frame
function

JIRA: YARDSTICK-1369

Change-Id: Ice06e4fbba4ec93ae086671a2530d521510f89da
Signed-off-by: Myron Sosyak <myronx.sosyak@intel.com>
6 years agoMerge "Change assert statements to raises in CheckValue"
Rex Lee [Tue, 7 Aug 2018 08:01:55 +0000 (08:01 +0000)]
Merge "Change assert statements to raises in CheckValue"

6 years agoMerge "Separate out test_parse_to_value_exception()"
Rex Lee [Tue, 7 Aug 2018 07:57:21 +0000 (07:57 +0000)]
Merge "Separate out test_parse_to_value_exception()"

6 years agoMerge "[workaround] Disable gui for aarch64"
Rex Lee [Tue, 7 Aug 2018 07:54:17 +0000 (07:54 +0000)]
Merge "[workaround] Disable gui for aarch64"

6 years agoMerge "Bugfix: HA test case baremetal down ipmi power off failed - dovetail"
Rex Lee [Tue, 7 Aug 2018 03:21:42 +0000 (03:21 +0000)]
Merge "Bugfix: HA test case baremetal down ipmi power off failed - dovetail"

6 years agoMerge "Add UT: ArithmeticRunner worker normal operation"
Abhijit Sinha [Fri, 3 Aug 2018 16:00:21 +0000 (16:00 +0000)]
Merge "Add UT: ArithmeticRunner worker normal operation"

6 years agoMerge "Add unit test file for ArithmeticRunner"
Abhijit Sinha [Fri, 3 Aug 2018 13:27:39 +0000 (13:27 +0000)]
Merge "Add unit test file for ArithmeticRunner"

6 years agoMerge "IXIA: Add IP random range `seed` configuration option"
Abhijit Sinha [Fri, 3 Aug 2018 13:25:37 +0000 (13:25 +0000)]
Merge "IXIA: Add IP random range `seed` configuration option"

6 years agoMerge "Add UDP ports configuration to IXIA traffic profile"
Abhijit Sinha [Fri, 3 Aug 2018 13:25:34 +0000 (13:25 +0000)]
Merge "Add UDP ports configuration to IXIA traffic profile"

6 years agoMerge "Change AutoConnectSSH to return error code by default"
Abhijit Sinha [Fri, 3 Aug 2018 13:23:40 +0000 (13:23 +0000)]
Merge "Change AutoConnectSSH to return error code by default"

6 years agoIXIA: Add IP random range `seed` configuration option 85/60485/4
Myron Sosyak [Thu, 2 Aug 2018 13:47:45 +0000 (14:47 +0100)]
IXIA: Add IP random range `seed` configuration option

Added IP random range `seed` configuration option into IXIA
traffic profile.

JIRA: YARDSTICK-1364

Change-Id: Ic050b6cc48ea75ac6d0470bf7ec1b188649811e1
Signed-off-by: Sosyak, Myronx <myronx.sosyak@intel.com>
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoNSB NFVi PROX BNG and vPE losing many packets 21/60521/1
DanielMartinBuckley [Thu, 2 Aug 2018 13:37:29 +0000 (14:37 +0100)]
NSB NFVi PROX BNG and vPE losing many packets

JIRA: YARDSTICK-1101

The traffic generated by the generator in bng-4 result in
non-symmetrical use of the cores on the SUT, as a whole
range of IP addresses is skipped.

Update cpe0 and cpe1 in gen files.

In addition, for the routing table (ipv4.lua) for the BNG
should be slighly different: while the BNG is a 4 ports
test, only 2 of the ports are used as outgoing routing
ports (the inet ports). Hence, the routing table should
only use the mac of those two ports. The bad configuration
(using 4 ports) result in an issue when BNG SUT and GEN
are connected through a switch, packets are sent by the
BNG SUT through the INET interfaces, but sometimes with
the MAC of the CPE interface...

Hence, packets are received on the wrong interface on the
Gen site (resulting in some interfaces, the CPE ones,
being overloaded)

Created ipv4_bng.lua and remove mac 0 and 2

Change-Id: I97019f0415d2f58d50ec855d04800eb705f41541
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoAdd UDP ports configuration to IXIA traffic profile 45/60445/6
Myron Sosyak [Thu, 2 Aug 2018 13:34:03 +0000 (14:34 +0100)]
Add UDP ports configuration to IXIA traffic profile

- Implemented handling of UDP source and destination ports from
  IXIA traffic profile.
- UDP ports can be defined as a single value or as a random range.
  Ports range is configured with two parameters 'fixed_bits' and
  'mask_bits'.
- For example '8-48' range definition will create a repeatable
  pattern of four values that fall within the range of 8 and 56.

JIRA: YARDSTICK-1363

Change-Id: I0ace722f6be843ea79c3d3f4de22cb8fa5669d4f
Signed-off-by: Myron Sosyak <myronx.sosyak@intel.com>
Signed-off-by: Pshyk Serhiy <serhiyx.pshyk@intel.com>
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoMerge "Remove hw_video_model and hw_firmware_type from metadata"
Abhijit Sinha [Thu, 2 Aug 2018 12:39:25 +0000 (12:39 +0000)]
Merge "Remove hw_video_model and hw_firmware_type from metadata"

6 years agoMerge "Replace assert statements in PipelineRules"
Abhijit Sinha [Thu, 2 Aug 2018 12:35:02 +0000 (12:35 +0000)]
Merge "Replace assert statements in PipelineRules"

6 years agoChange AutoConnectSSH to return error code by default 07/60507/2
Mytnyk, Volodymyr [Thu, 2 Aug 2018 08:17:48 +0000 (09:17 +0100)]
Change AutoConnectSSH to return error code by default

AutoConnectSSH execute method always returns exception when remote
command returns non 0 ret code. This behavior has been introduced
by https://gerrit.opnfv.org/gerrit/#/c/58579/ changes which break
the NSB functionality. There are 200+ places where return code
(event non zoro) is expected to be returned by
AutoConnectSSH.execute() method. Right now the method returns
always exception in case of remote command return not zero which
causes the problem.

Changed execute() method to have previous behavior (raise_on_error
is always False by default) and the exception is raised only in
case if raise_on_error=True is set explicitly.

Added UT.

JIRA: YARDSTICK-1365

Change-Id: Ib067583ea5eb704b9174084b45b920c24eb307ac
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoSeparate out test_parse_to_value_exception() 55/57455/2
John O'Loughlin [Wed, 16 May 2018 10:00:21 +0000 (10:00 +0000)]
Separate out test_parse_to_value_exception()

Separate out test_parse_to_value_exception(self)
into 3 different methods:
    test_parse_scanner(self)
    test_parse_parser(self)
    test_parse_reader(self)

This ensures that each side effect is tested, even if one fails.

JIRA: YARDSTICK-940

Change-Id: Ifdc84284c633fe1ddc1acf9cd985c5144030b430
Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
6 years agoRemove hw_video_model and hw_firmware_type from metadata 83/60483/1
Delia Popescu [Wed, 1 Aug 2018 14:02:35 +0000 (17:02 +0300)]
Remove hw_video_model and hw_firmware_type from metadata

Starting with the Openstack Queens release,
hw_firmware_type=uefi is implied on AArch64 by default
Using virtio-video, now default in nova, should be used
instead of VGA

JIRA: ARMBAND-388

Change-Id: If52206e2a2f0a05bde5632c91de9367fffdc2eb3
Signed-off-by: Delia Popescu <delia.popescu@enea.com>
6 years agoReplace assert statements in PipelineRules 43/60443/1
Miikka Koistinen [Tue, 31 Jul 2018 14:03:10 +0000 (17:03 +0300)]
Replace assert statements in PipelineRules

The class PipelineRules uses assert statements to check the correctness
of an input parameter IP address. This commit will modify these lines
to use a dedicated function "common.utils.make_ipv4_address".

JIRA: YARDSTICK-966

Change-Id: I1c4cc439a7716d36548125c87dfb0bc200714a56
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years ago[workaround] Disable gui for aarch64 37/60437/1
Cristina Pauna [Mon, 23 Jul 2018 12:21:45 +0000 (15:21 +0300)]
[workaround] Disable gui for aarch64

Currently the aarch64 yardstick image doesn't build because phantomsj and bower
don't install properly. Temporarily disable gui for aarch64 only so we can
still get the other updates.

JIRA: ARMBAND-391

Change-Id: I64f892641c7bdb1cb3a65b210f71b093878b1688
Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
6 years agoFix remote command execution in common.utils 79/58579/4
Miikka Koistinen [Fri, 15 Jun 2018 08:54:14 +0000 (11:54 +0300)]
Fix remote command execution in common.utils

yardstick.common.utils get_port_mac and get_port_ip both raise a
RuntimeError on positive remote command exit status. This commit fixes
them to use the error raising mechanism in yardstick.ssh.SSH.

Additionally, the class AutoConnectSSH class needed an additional
argument to allow the raising mechanism to work correctly.

JIRA: YARDSTICK-1240

Change-Id: Idad125ebbd668cef10a6149eb3e601a437a8d40d
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoCleanup pktgen_dpdk unit tests 97/58497/11
Miikka Koistinen [Wed, 13 Jun 2018 10:27:47 +0000 (13:27 +0300)]
Cleanup pktgen_dpdk unit tests

* Improve test execution
* Use common functionality in setUp()
* Disable logging
* Remove tests that test common/utils code. These tests are moved to
  common/utils's unit tests in a follow-up patch.

JIRA: YARDSTICK-1239

Change-Id: I88d047fe25c4c13bea446c9fc9bda92306420296
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoChange assert statements to raises in CheckValue 85/60285/2
Miikka Koistinen [Fri, 27 Jul 2018 13:53:51 +0000 (16:53 +0300)]
Change assert statements to raises in CheckValue

* Instead of using assert statements, raise a custom exception.
* Modify the unit tests so that they do not test unused parameters for
  equality.
* Fix pylint and pep8 errors.

JIRA: YARDSTICK-966

Change-Id: I41e13a5a22086827792eaf9de8da2e4ed6dd42de
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoMerge "Check for network already created k8"
Emma Foley [Tue, 31 Jul 2018 08:39:47 +0000 (08:39 +0000)]
Merge "Check for network already created k8"

6 years agoMerge "Cleanup pktgen unit tests"
Emma Foley [Tue, 31 Jul 2018 08:14:00 +0000 (08:14 +0000)]
Merge "Cleanup pktgen unit tests"

6 years agoMerge "Change assert statement to raise in common/utils"
Rex Lee [Tue, 31 Jul 2018 07:38:02 +0000 (07:38 +0000)]
Merge "Change assert statement to raise in common/utils"

6 years agoMerge "Refactor remote command execution in vsperf_dpdk"
Rex Lee [Tue, 31 Jul 2018 07:33:29 +0000 (07:33 +0000)]
Merge "Refactor remote command execution in vsperf_dpdk"

6 years agoMerge "Refactor remote command execution in vsperf"
Rex Lee [Tue, 31 Jul 2018 07:33:23 +0000 (07:33 +0000)]
Merge "Refactor remote command execution in vsperf"

6 years agothe recovery action of "baremetal down" should be triggered mandatory 43/60143/3
rexlee8776 [Tue, 24 Jul 2018 04:17:45 +0000 (04:17 +0000)]
the recovery action of "baremetal down" should be triggered mandatory

YARDSTICK-1222 has made attacker recover only when the sla not pass.
But for baremetal down test case, the node need to power on even the sla pass.

TODO:
Make attacker can support mandatory recover in some situation.

JIRA: YARDSTICK-1337

Change-Id: Ib567b9b9025e5ee421ea47140c468537ad16f090
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoCleanup pktgen unit tests 95/58495/12
Miikka Koistinen [Wed, 13 Jun 2018 13:25:21 +0000 (16:25 +0300)]
Cleanup pktgen unit tests

* Improve test execution
* Use common functionality in setUp()
* Disable logging

JIRA: YARDSTICK-1238

Change-Id: I0904bf99f0e151b437d506bf322da83f706dcf8f
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoChange assert statement to raise in common/utils 53/60353/1
Miikka Koistinen [Mon, 30 Jul 2018 13:01:14 +0000 (16:01 +0300)]
Change assert statement to raise in common/utils

This modification utilises a custom exception to be raised if an invalid
mac address is passed to the function "mac_address_to_hex_list".

JIRA: YARDSTICK-966

Change-Id: Ic2940276f500e9710c54963a445a137e73bd6b25
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoMake ixnet a class attribute in TestIxNextgen 59/58159/5
Emma Foley [Thu, 14 Jun 2018 08:39:23 +0000 (08:39 +0000)]
Make ixnet a class attribute in TestIxNextgen

Change-Id: Iad8e8397f4ef339f70c6eef80ef58b1b045ef3fa
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoRefactor remote command execution in vsperf_dpdk 29/59429/5
Miikka Koistinen [Mon, 2 Jul 2018 10:50:59 +0000 (13:50 +0300)]
Refactor remote command execution in vsperf_dpdk

* Remove unneeded variables
* Do not raise RuntimeError when a remote command fails, instead
  leave it for the yardstick ssh module to handle.
* Prevent CsvReader from raising StopIteration if csv output cannot be
  parsed. The SLA validation will take care of errors in that case.

JIRA: YARDSTICK-1166

Change-Id: I9e353d39dfd3bd27407dde32744ce71b5762e84e
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoRefactor remote command execution in vsperf 35/59335/7
Miikka Koistinen [Fri, 29 Jun 2018 13:31:31 +0000 (16:31 +0300)]
Refactor remote command execution in vsperf

* Remove unneeded variables
* Do not raise RuntimeError when a remote command fails, instead leave
  it for the yardstick ssh module to handle.
* Prevent CsvReader from raising StopIteration if csv output cannot be
  parsed. The SLA validation will take care of errors in that case.

JIRA: YARDSTICK-1166

Change-Id: I6cf550bd7bc8f511b3c0f25c67f8caab18bccd28
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoAdd UT: ArithmeticRunner worker normal operation 17/58717/6
Miikka Koistinen [Tue, 19 Jun 2018 07:36:08 +0000 (10:36 +0300)]
Add UT: ArithmeticRunner worker normal operation

* Use time.sleep to avoid multiprocessing.Queue broken pipe error.

JIRA: YARDSTICK-1245

Change-Id: I0176c062f44830804ddbeebeb012afc1f9e20d41
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoAdd unit test file for ArithmeticRunner 15/58715/5
Miikka Koistinen [Fri, 15 Jun 2018 12:29:21 +0000 (15:29 +0300)]
Add unit test file for ArithmeticRunner

* Initial test: test ArithmeticRunner._run_benchmark called arguments

JIRA: YARDSTICK-1245

Change-Id: Id18c36bd1345052863ab777bf899c00fe66fd713
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoAdd UT: DurationRunner worker exception handling 35/58235/10
Miikka Koistinen [Tue, 5 Jun 2018 11:29:16 +0000 (11:29 +0000)]
Add UT: DurationRunner worker exception handling

* Use time.sleep to avoid multiprocessing.Queue broken pipe error.

JIRA: YARDSTICK-1199

Change-Id: Icd35dad547bfa1ac4fab564c00628ef0d30ac338
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoAdd UT: DurationRunner worker normal operation 33/58233/10
Miikka Koistinen [Tue, 5 Jun 2018 10:33:17 +0000 (10:33 +0000)]
Add UT: DurationRunner worker normal operation

* Use time.sleep to avoid multiprocessing.Queue broken pipe error.
* Use a real duration in one of the test cases to test that the worker
  has successfully completed multiple iterations.

JIRA: YARDSTICK-1199

Change-Id: I0e14682d9993b69e2aa9fd9340a5a7620e9c0d8e
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoMerge "Enable "wait_until_true" when used ouf the main thread"
Rodolfo Alonso Hernandez [Thu, 26 Jul 2018 14:56:14 +0000 (14:56 +0000)]
Merge "Enable "wait_until_true" when used ouf the main thread"

6 years agoMerge "NSB TREX RFC2544 scenario fails with correlated traffic"
Rodolfo Alonso Hernandez [Thu, 26 Jul 2018 14:45:38 +0000 (14:45 +0000)]
Merge "NSB TREX RFC2544 scenario fails with correlated traffic"

6 years agoMerge "Make injection time configurable for IXIA RFC2544 test cases"
Rodolfo Alonso Hernandez [Thu, 26 Jul 2018 14:33:17 +0000 (14:33 +0000)]
Merge "Make injection time configurable for IXIA RFC2544 test cases"

6 years agoEnable "wait_until_true" when used ouf the main thread 15/60215/2
Rodolfo Alonso Hernandez [Thu, 26 Jul 2018 14:21:45 +0000 (15:21 +0100)]
Enable "wait_until_true" when used ouf the main thread

"util.wait_until_true" uses "util.Timer" to create an active wait for a
condition. "Timer" class uses "signal" to create a watchdog to track the
time lapsed.

When used out of the main thread, "Timer" raises the following error:
  ValueError: signal only works in main thread

To make "util.wait_until_true" usable always, a new waitting method is
implemented.

JIRA: YARDSTICK-1358

Change-Id: Ifb5ba0b17b5beca0af5ceab4f6431d58b7928762
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoNSB TREX RFC2544 scenario fails with correlated traffic 05/60205/4
Mytnyk, Volodymyr [Thu, 26 Jul 2018 08:47:19 +0000 (09:47 +0100)]
NSB TREX RFC2544 scenario fails with correlated traffic

When correlated traffic TC is running, the traffic is
running only on uplink ports instead of both type of ports
(yardstick/network_services/traffic_profile/rfc2544.py +113).

Trex TG fails to get samples on downlink ports.

JIRA: YARDSTICK-1357

Change-Id: I66d78b73f2dc5bb531fe8607d0e46f60b828953e
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoMake injection time configurable for IXIA RFC2544 test cases 65/57765/13
Rodolfo Alonso Hernandez [Wed, 23 May 2018 17:03:19 +0000 (18:03 +0100)]
Make injection time configurable for IXIA RFC2544 test cases

JIRA: YARDSTICK-1196

Change-Id: I77b4772b9224ff5cd21fd9fc67eb7e4b821fcf77
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agokubernetes_utils: avoid 404 error code in delete_config_map() 73/59973/5
Mytnyk, Volodymyr [Thu, 19 Jul 2018 07:47:02 +0000 (08:47 +0100)]
kubernetes_utils: avoid 404 error code in delete_config_map()

JIRA: YARDSTICK-1318

Change-Id: I0c04bd2928d5a4166b483f0c4f9f75fa9e5e00d2
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agokubernetes_utils: avoid 404 error code in delete_replication_controller() 77/59977/5
Mytnyk, Volodymyr [Thu, 19 Jul 2018 08:04:06 +0000 (09:04 +0100)]
kubernetes_utils: avoid 404 error code in delete_replication_controller()

JIRA: YARDSTICK-1319

Change-Id: I8370c5ef4e43e58a8bb9c3aac60a089b0981dda0
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoMerge "Fix typo in classname AclUknownActionTemplate"
Rodolfo Alonso Hernandez [Wed, 25 Jul 2018 06:58:17 +0000 (06:58 +0000)]
Merge "Fix typo in classname AclUknownActionTemplate"

6 years agoIxNetwork IPv4 mask format is x.x.x.x 55/60155/3
Rodolfo Alonso Hernandez [Tue, 24 Jul 2018 14:11:37 +0000 (15:11 +0100)]
IxNetwork IPv4 mask format is x.x.x.x

When updating the IP packet in IxNextgen library, the IPv4 mask should
be defined in the following format: x.x.x.x

This mask defines the number of lower bits used to generate a random IP.
For example:
  - mask length: 16
  - mask string: 0.0.255.255

JIRA: YARDSTICK-1351

Change-Id: Ie3cbf0205aac83aea8d7f665bc068e8566238055
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoFix typo in classname AclUknownActionTemplate 81/59981/3
Emma Foley [Thu, 19 Jul 2018 09:33:04 +0000 (10:33 +0100)]
Fix typo in classname AclUknownActionTemplate

AclUknownActionTemplate -> AclUnknownActionTemplate

JIRA: YARDSTICK-1334
Change-Id: I796e080078c657c5e0e59144d24ec797e244a8f7
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoMerge changes from topics 'YARDSTICK-1351', 'YARDSTICK-1339'
Rodolfo Alonso Hernandez [Tue, 24 Jul 2018 11:00:12 +0000 (11:00 +0000)]
Merge changes from topics 'YARDSTICK-1351', 'YARDSTICK-1339'

* changes:
  Add IP mask to IxNetwork configuration
  Improve NetworkServiceTestCase._get_ip_flow_range() function

6 years agoMerge changes from topics 'YARDSTICK-1346', 'YARDSTICK-1345', 'YARDSTICK-1344'
Rodolfo Alonso Hernandez [Tue, 24 Jul 2018 11:00:01 +0000 (11:00 +0000)]
Merge changes from topics 'YARDSTICK-1346', 'YARDSTICK-1345', 'YARDSTICK-1344'

* changes:
  Add pktgen test case base files
  Add pktgen traffic profile
  Add DPDK pktgen traffic generator