yardstick.git
6 years agoNSB: move interface probe to VNF, and attempt driver-only probe first 81/44281/46
Ross Brattain [Wed, 4 Oct 2017 20:38:43 +0000 (13:38 -0700)]
NSB: move interface probe to VNF, and attempt driver-only probe first

If no devices are present we can't detect MAC address so
we can't match Heat ports to interfaces.

If only the driver is missing we can try to probe the driver using
lspci.  We can use lspci to ask the kernel what driver it should use
for the PCI device.

If we can't probe at all because the device is already bound, we can
use dpkd-devind to find all the PCI address we care about and create a
map with PCI device and real kernel driver.

Then we can dpdk force rebind to the kernel driver.
Once we have rebound to the kernel driver we can detect
MAC address and all the other attributes that are required.

Fix VnfSshHelper to allow override of wait timeout

And a bunch of other refactors that got swept up in this

JIRA: YARDSTICK-835

Change-Id: I14cb657ed289a77941d048345d06ced5b5d5da52
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "Add methods to get an existing stack"
Ross Brattain [Thu, 1 Mar 2018 16:17:51 +0000 (16:17 +0000)]
Merge "Add methods to get an existing stack"

6 years agoMerge "Use context name instead of key_uuid"
Ross Brattain [Thu, 1 Mar 2018 16:16:12 +0000 (16:16 +0000)]
Merge "Use context name instead of key_uuid"

6 years agoMerge changes from topics 'feat/keep_vnf', 'YARDSTICK-886'
Ross Brattain [Thu, 1 Mar 2018 16:15:11 +0000 (16:15 +0000)]
Merge changes from topics 'feat/keep_vnf', 'YARDSTICK-886'

* changes:
  Add _create_new_stack method
  Update TaskParser to deal with qualified name in Context
  Add qualified name to Context

6 years agoMerge "vFW scale-up template."
Ross Brattain [Thu, 1 Mar 2018 15:11:31 +0000 (15:11 +0000)]
Merge "vFW scale-up template."

6 years agoMerge "Add arguments to the traffic profile render"
Ross Brattain [Thu, 1 Mar 2018 15:11:20 +0000 (15:11 +0000)]
Merge "Add arguments to the traffic profile render"

6 years agoMerge "Import "traffic_profile" modules only once"
Ross Brattain [Thu, 1 Mar 2018 15:11:03 +0000 (15:11 +0000)]
Merge "Import "traffic_profile" modules only once"

6 years agoAdd methods to get an existing stack 51/52351/15
Emma Foley [Mon, 19 Feb 2018 23:15:11 +0000 (23:15 +0000)]
Add methods to get an existing stack

* Add yardstick/orchestrator/heat.py:HeatStack.get
* Add yardstick/benchmark/contexts/heay.py:HeatContext.retrieve_existing_stack

JIRA: YARDSTICK-886
Change-Id: I6974b79a25f98066a49b1bc8ccd11383e7962091
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoUse context name instead of key_uuid 47/52347/14
Emma Foley [Fri, 16 Feb 2018 15:22:12 +0000 (15:22 +0000)]
Use context name instead of key_uuid

context.key_uuid is only used as a suffix for the key files.
There is already a unique ID associated with a the context;
the context name includes a task ID, which is a UUID.

This patch sets context.key_uuid to to context.name instead of
generating a separate UUID

As a result, get_short_key_uuid() is not needed.

JIRA: YARDSTICK-1028
Change-Id: Id175061d6cfe23a068bb3d12ce176c1f176e8236
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoAdd _create_new_stack method 79/52779/10
Emma Foley [Wed, 21 Feb 2018 12:36:15 +0000 (12:36 +0000)]
Add _create_new_stack method

The logic for creating a new stack in contexts/heat.py:HeatContext
can added to a method to make the code eassier to read and test.

This is in preparation for an update to deploy() that would allow
an existing stack to be reused, or a new stack created.
By having the create_new_stack logic in a new method, deploy()
becomes easier to read and test.

JIRA: YARDSTICK-886
Change-Id: I7af01e2209a3460658f8db0249b7c620743cced0
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoUpdate TaskParser to deal with qualified name in Context 51/52451/12
Emma Foley [Thu, 22 Feb 2018 19:11:12 +0000 (19:11 +0000)]
Update TaskParser to deal with qualified name in Context

The context name depends on the defined name in the testcase input
file, the task ID and the flags of the context.

If the context is going to be undeployed at the end of the test, the
task ID is suffixed to the name to avoid interferences with previous
deployments. If the context needs to be deployed at the end of the
test, the name assigned is kept.

This patch makes this process transparent to the developer.
This patch modifies how TaskParser determines the correct context name,
taking into account that the name might change based on context flags.

JIRA: YARDSTICK-886

Change-Id: I44da30dac562c1a4166e084645ae91c17798651d
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agovFW scale-up template. 87/51987/20
Mytnyk, Volodymyr [Sat, 10 Feb 2018 18:30:06 +0000 (20:30 +0200)]
vFW scale-up template.

Added topology and traffic profile templates
Added support for using JinJa2 templates in topology definition
Added support for static pipeline configs for SampleVNFs

JIRA: YARDSTICK-1043

Change-Id: Iab99fd5b5ad69ca32ee70b9fe47779387ad27e7f
Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoMerge "Module to manage pip packages"
Rodolfo Alonso Hernandez [Thu, 1 Mar 2018 13:23:54 +0000 (13:23 +0000)]
Merge "Module to manage pip packages"

6 years agoAdd qualified name to Context 55/52655/10
Emma Foley [Mon, 26 Feb 2018 11:43:51 +0000 (11:43 +0000)]
Add qualified name to Context

The context name depends on the defined name in the testcase input
file, the task ID and the flags of the context.

If the context is going to be undeployed at the end of the test, the
task ID is suffixed to the name to avoid interferences with previous
deployments. If the context needs to be deployed at the end of the
test, the name assigned is kept.

This patch makes base.Context use Flags when initialising contexts,
this sets the name property based on the no_setup and no_teardown
flags.

Since base.Context is an abstract class, it cannot be instantiated.
However, there are some non-abstract methods that need testing.
Since DummyContext does not override any of these methods, it can be used for
testing.

JIRA: YARDSTICK-886

Change-Id: I1447fb5ed447691eaeb0a97f928c0b3333799d07
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoAdd arguments to the traffic profile render 67/50767/17
Rodolfo Alonso Hernandez [Wed, 17 Jan 2018 18:04:53 +0000 (18:04 +0000)]
Add arguments to the traffic profile render

In order to render configurable traffic profiles in NSB test cases,
a new variable is introduced: "extra_arg". The content of
this variable is added to the VNFD render data, under a key called
"extra_args".

This will allow the user to define Jinja templates for traffic profiles.

E.g.:

  $ cat test_case_definition.yml
  scenarios:
  - type: NSPerf
    traffic_profile: traffic_profile.yml
    extra_args:
      vports: 10

  $ cat traffic_profile.yml
  {% set vports = get(extra_args, 'vports', '0') or 4 %}
  {% for vport in range(vports|int) %}
  uplink_{{vport}}:
    data...
  {% endfor %}

JIRA: YARDSTICK-946

Change-Id: Ib3c1f2d89efa012772edf2156e97d5f4742a6b80
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoImport "traffic_profile" modules only once 49/50849/16
Rodolfo Alonso Hernandez [Fri, 19 Jan 2018 15:46:11 +0000 (15:46 +0000)]
Import "traffic_profile" modules only once

"traffic_profile" modules should be imported only once. Every time
TrafficProfile.get is called, the modules under
"yardstick.network_services.traffic_profiles" are loaded [1]. Instead
of this, the modules should be registered only once the first time
"yardstick.network_services.traffic_profiles.base" is loaded. This
will reduce the execution time and will avoid unnecessary calls.

[1] https://github.com/opnfv/yardstick/blob/d2c7cc4e9768ed003257a95c92cdb278d516761b/yardstick/network_services/traffic_profile/base.py#L36-L37

JIRA: YARDSTICK-951

Change-Id: Ia3565378ba3a1377fcb0aea8bda50ef8189414fd
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoscenario: add opnfv_os-odl-bgpvpn-noha_daily.yaml 97/52897/3
Ross Brattain [Thu, 1 Mar 2018 02:30:48 +0000 (18:30 -0800)]
scenario: add opnfv_os-odl-bgpvpn-noha_daily.yaml

adding missing scenario for Apex.
https://gerrit.opnfv.org/gerrit/#/c/50045/

JIRA: YARDSTICK-998

Change-Id: I211f567587a4864b403ce321fea007a3c0963136
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoBugfix: yardstick env grafana fails when access the server 25/51225/3
rexlee8776 [Sat, 27 Jan 2018 06:15:51 +0000 (06:15 +0000)]
Bugfix: yardstick env grafana fails when access the server

current timeout is 10 seconds, it would fail if grafana
server start slow in some cases.

JIRA: YARDSTICK-972

Change-Id: I783206fdd1352e94ba7e498283f70b5a6834b6fa
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoMerge "fix pep8 problem in env.py"
Jack Chan [Thu, 1 Mar 2018 02:20:38 +0000 (02:20 +0000)]
Merge "fix pep8 problem in env.py"

6 years agoNSB: Rename traffic_profile to trex_traffic_profile 59/52059/13
Ross Brattain [Tue, 13 Feb 2018 01:34:40 +0000 (17:34 -0800)]
NSB: Rename traffic_profile to trex_traffic_profile

Make it more clear that traffic_profile is not a generic class.
Eventually we can refactor a more generic traffic profile.

Change-Id: I1bf44a8cafcdeb8d74efb9e85a34f6d7b526d036
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "Add Flags class to base.Context"
Emma Foley [Wed, 28 Feb 2018 18:01:24 +0000 (18:01 +0000)]
Merge "Add Flags class to base.Context"

6 years agoMerge "Mock TRex STL libraries globally"
Abhijit Sinha [Wed, 28 Feb 2018 17:49:00 +0000 (17:49 +0000)]
Merge "Mock TRex STL libraries globally"

6 years agoMerge "Deprecate authentication variable OS_TENANT_NAME"
Ross Brattain [Wed, 28 Feb 2018 15:52:57 +0000 (15:52 +0000)]
Merge "Deprecate authentication variable OS_TENANT_NAME"

6 years agoMock TRex STL libraries globally 17/52117/6
Rodolfo Alonso Hernandez [Tue, 13 Feb 2018 17:17:22 +0000 (17:17 +0000)]
Mock TRex STL libraries globally

TRex Python libraries are not going to be available during
unit testing execution. Those modules calling TRex libraries
should be mocked individually. This patch will remove the need
of mocking those libraries per module.

JIRA: YARDSTICK-1010

Change-Id: I4aa11d43ecf32a3dad78f869541b4afea4ec1d28
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Improve SampleVNF hugepages setup"
Abhijit Sinha [Wed, 28 Feb 2018 13:22:40 +0000 (13:22 +0000)]
Merge "Improve SampleVNF hugepages setup"

6 years agoMerge "Addition of storage of extra counters for Grafana"
Abhijit Sinha [Wed, 28 Feb 2018 13:05:45 +0000 (13:05 +0000)]
Merge "Addition of storage of extra counters for Grafana"

6 years agoAddition of storage of extra counters for Grafana 47/52247/3
DanielMartinBuckley [Fri, 16 Feb 2018 15:39:41 +0000 (15:39 +0000)]
Addition of storage of extra counters for Grafana

JIRA: YARDSTICK-1036
This stores a number of extra counters in influxdb for Prox test cases.
It also stores existing counters with a "succcess_" tag.
Previously throughput where stored without success or failure indication.
Also "Result_" counters are also stored.

These can now be used by Grafana to graph output.

Change-Id: Ie5636c14ecbab1b53a988bdfbd47ddd1fcdbd695
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoDeprecate authentication variable OS_TENANT_NAME 65/49265/7
Rodolfo Alonso Hernandez [Tue, 19 Dec 2017 09:08:05 +0000 (09:08 +0000)]
Deprecate authentication variable OS_TENANT_NAME

OS_TENANT_NAME was deprecated as authentication variable in Keystone
when moved from v2 to v3, in Icehouse (2014). Because this project doesn't
support oldest versions, by default the only identification API version
supported is v3.

JIRA: YARDSTICK-902

Change-Id: I273fb0151ba583f7c8a5a809e5e8864e92c27d31
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Yardstick real-time influxdb KPI reporting regressions"
Emma Foley [Wed, 28 Feb 2018 10:51:13 +0000 (10:51 +0000)]
Merge "Yardstick real-time influxdb KPI reporting regressions"

6 years agoAdd Flags class to base.Context 53/52653/3
Emma Foley [Mon, 26 Feb 2018 11:25:34 +0000 (11:25 +0000)]
Add Flags class to base.Context

This patch adds a Flags class to represent the flags passed to
the context.

Flags will contain a set of attributes (for example no_setup and no_teardown).
Those attributes will modify the behaviour of the context.

The attributes added initially are:
* 'no_setup' (default: False)
* 'no_teardown' (default: False)

In future patches, they will be used by the context to:
* Change the name of the context.
* Skip the deploy() and undeploy() methods to keep the VNFs active after the
  testcases are run.

JIRA: YARDSTICK-886
Change-Id: I734b082aa101649eea8991304a8b753ae96aa9de
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agofix pep8 problem in env.py 25/52725/2
rexlee8776 [Tue, 27 Feb 2018 09:27:48 +0000 (09:27 +0000)]
fix pep8 problem in env.py

JIRA: YARDSTICK-1033

Change-Id: Ia0f745b1a21845afc5c3b3ec58d7a79da56d6a6c
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoMerge "add egress security rules"
Ross Brattain [Wed, 28 Feb 2018 01:52:36 +0000 (01:52 +0000)]
Merge "add egress security rules"

6 years agoMerge "Addition of NSB Prox developer guide."
Abhijit Sinha [Tue, 27 Feb 2018 16:59:54 +0000 (16:59 +0000)]
Merge "Addition of NSB Prox developer guide."

6 years agoAddition of NSB Prox developer guide. 39/52239/3
DanielMartinBuckley [Fri, 16 Feb 2018 14:56:42 +0000 (14:56 +0000)]
Addition of NSB Prox developer guide.

JIRA: YARDSTICK-840

This provides a developer guide to enable the reader to add NSB
PROX tests based on existing PROX functionality.

This allows the user to configure, run and debug NSB Prox tests
for Baremetal and Openstack (both 2 and 4 ports)

Change-Id: I2cb9eb6064e501dd9abe0d2cc4ed8ce4b369f174
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
6 years agoadd egress security rules 11/52511/3
mting [Fri, 23 Feb 2018 10:01:44 +0000 (11:01 +0100)]
add egress security rules

JIRA: YARDSTICK-1034

Change-Id: I613b8bff669fceeb0c7c59dec2b6f1ab22b09247
Signed-off-by: mting <dianfeng.du@ericsson.com>
6 years agoMerge "Testcase to find storage bottlenecks using Yardstick for Multistack"
Rex Lee [Mon, 26 Feb 2018 04:21:13 +0000 (04:21 +0000)]
Merge "Testcase to find storage bottlenecks using Yardstick for Multistack"

6 years agoTestcase to find storage bottlenecks using Yardstick for Multistack 53/51753/3
Ace Lee [Tue, 6 Feb 2018 17:07:24 +0000 (17:07 +0000)]
Testcase to find storage bottlenecks using Yardstick for Multistack

JIRA: BOTTLENECK-217

This patch is used to find storage bottlenecks while using
yardstick for multistack.

Shubham is busy for his exam,
I can't change his patch, so I need to resubmit a patch.

Change-Id: Ibf8ec3ae7f19b27254eedabb1b95bd89a0993ed7
Signed-off-by: Ace Lee <liyin11@huawei.com>
6 years agoBugfix: ci error due to no heat client 05/52605/1
rexlee8776 [Mon, 26 Feb 2018 02:03:29 +0000 (02:03 +0000)]
Bugfix: ci error due to no heat client

JIRA: YARDSTICK-1030

Change-Id: I5d9e135449b2d35f051ecf5fa63a1ce11030de4a
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoMerge "Bug Fix: Move heat_timeout variable higher"
Jing Lu [Sat, 24 Feb 2018 02:31:41 +0000 (02:31 +0000)]
Merge "Bug Fix: Move heat_timeout variable higher"

6 years agoBug Fix: Move heat_timeout variable higher 29/52529/2
Dimitrios Markou [Fri, 23 Feb 2018 14:13:09 +0000 (16:13 +0200)]
Bug Fix: Move heat_timeout variable higher

If a heat_template was used by a testcase the heat_timeout
variable was not getting initialized and that caused errors
in the Shade python module.

JIRA: YARDSTICK-1027

Change-Id: Ic0bf15b13cc00595964a97f8150dc5e6f2b7e1d7
Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
6 years agoYardstick real-time influxdb KPI reporting regressions 51/51651/4
chenjiankun [Mon, 5 Feb 2018 08:13:07 +0000 (08:13 +0000)]
Yardstick real-time influxdb KPI reporting regressions

JIRA: YARDSTICK-989

We used to have real-time influxdb reporting of test KPIs. The user
could monitor using grafana and see the real-time output.
The record format was changed to now only report KPIs at the end of the
test.
This is a problem for test cases which run for a long duration,  we need
to wait until the end of the test execution to get any results from
influxdb. If the test fails in between or doesn't exit cleanly for some
reason, we are left with no results stored in influxdb which gives the
user no information.
This is also a regression from the previous behaviour.

Change-Id: I0f476dff9162a359f0286fb421f2e9c4befaa5cc
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
6 years agoAdd 'ANSIBLE_ROLES_PATH' in ansible_env 87/52487/2
JingLu5 [Fri, 23 Feb 2018 01:31:38 +0000 (01:31 +0000)]
Add 'ANSIBLE_ROLES_PATH' in ansible_env

JIRA: YARDSTICK-1024

This work adds the 'ANSIBLE_ROLES_PATH' as one of the Ansible environment variables.
This 'ANSIBLE_ROLES_PATH' is set to the roles directory in Yardstick repo.
By adding this variable, Ansible will first searches the local project for a role, then searches the roles_path.
It will let the users of the plugin projects to able to use a relative role path for any of their own roles in ansible playbook.

Change-Id: I0926ab0c0fa5c6cc37c4fe560db5b2aec3fe6eb1
Signed-off-by: JingLu5 <lvjing5@huawei.com>
6 years agoMerge "Fix error in address input format in "_ip_range_action_partial""
Rodolfo Alonso Hernandez [Thu, 22 Feb 2018 16:29:02 +0000 (16:29 +0000)]
Merge "Fix error in address input format in "_ip_range_action_partial""

6 years agoMerge "Make files pep8 compliant"
Emma Foley [Thu, 22 Feb 2018 12:54:27 +0000 (12:54 +0000)]
Merge "Make files pep8 compliant"

6 years agoMerge "Refactor benchmark.core.test_plugin.PluginTestCase"
Rodolfo Alonso Hernandez [Thu, 22 Feb 2018 11:36:37 +0000 (11:36 +0000)]
Merge "Refactor benchmark.core.test_plugin.PluginTestCase"

6 years agoMerge "Bug fix for Ansible destroy VM script"
Emma Foley [Wed, 21 Feb 2018 09:58:28 +0000 (09:58 +0000)]
Merge "Bug fix for Ansible destroy VM script"

6 years agoMerge "Proposing Abhijit Sinha as a committer in Yardstick"
Ross Brattain [Tue, 20 Feb 2018 18:55:04 +0000 (18:55 +0000)]
Merge "Proposing Abhijit Sinha as a committer in Yardstick"

6 years agoMake files pep8 compliant 45/52345/4
Emma Foley [Wed, 31 Jan 2018 17:41:33 +0000 (17:41 +0000)]
Make files pep8 compliant

This change makes files pep8 compliant before adding the
reuse context feature

JIRA: YARDSTICK-886
Change-Id: Iae7daaa159a9ddbb5809a9a7ac74f8a53683089a
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoMerge "Replace neutron network deletion with shade."
Emma Foley [Mon, 19 Feb 2018 13:34:47 +0000 (13:34 +0000)]
Merge "Replace neutron network deletion with shade."

6 years agoMerge "Make segmentation_id configurable for tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex"
Emma Foley [Mon, 19 Feb 2018 13:29:11 +0000 (13:29 +0000)]
Merge "Make segmentation_id configurable for tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex"

6 years agoMake segmentation_id configurable for tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex 35/52235/2
Mytnyk, Volodymyr [Fri, 16 Feb 2018 10:45:20 +0000 (10:45 +0000)]
Make segmentation_id configurable for tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex

Updated vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml TC to be
able to configure segmentation_id from commmand line.

E.g.:
  yardstick -d task start --task-args='{"provider": "sriov", \
    "segmentation_id": 100}' ...

JIRA: YARDSTICK-1004

Change-Id: I53daae3468de2a1f523bd96817143a5f809238a2
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoReplace neutron network deletion with shade. 67/51767/13
Shobhi Jain [Thu, 1 Feb 2018 16:31:55 +0000 (16:31 +0000)]
Replace neutron network deletion with shade.

Function delete_neutron_net now uses shade client instead of neutron
client.

JIRA: YARDSTICK-890

Change-Id: I99b6f3ecd37d6e3b783f7eb3da9f971398fc3bdd
Signed-off-by: Shobhi Jain <shobhi.jain@intel.com>
6 years agoFix error in address input format in "_ip_range_action_partial" 91/51691/7
Rodolfo Alonso Hernandez [Mon, 5 Feb 2018 17:52:10 +0000 (17:52 +0000)]
Fix error in address input format in "_ip_range_action_partial"

IP address format introduced in [1] should be unicode instead of string.
"ipaddress.IPv4Address(min_value)" doesn't parse correctly the input
parameter unless the parameter is in unicode format; this is valid both
for Python version 2 and 3.

Execution error if the parameter is a string:
>>> int(ipaddress.IPv4Address('10.0.3.2'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/ipaddress.py",
  line 1391, in __init__
    self._check_packed_address(address, 4)
  File "/usr/local/lib/python2.7/dist-packages/ipaddress.py",
  line 554, in _check_packed_address
    expected_len, self._version))
ipaddress.AddressValueError: '10.0.3.2' (len 8 != 4) is not permitted
as an IPv4 address. Did you pass in a bytes (str in Python 2) instead
of a unic

[1]https://github.com/opnfv/yardstick/blob/e5775e7efbc55f116b4d4ac11ff87b8d8553247e/yardstick/network_services/traffic_profile/traffic_profile.py#L87-L88

JIRA: YARDSTICK-996

Change-Id: Ic727a79044834b181c99789f0f5efc21c68f0ff2
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Workaround to fix Heat stack deletion bug in Shade"
Rodolfo Alonso Hernandez [Sat, 17 Feb 2018 19:01:36 +0000 (19:01 +0000)]
Merge "Workaround to fix Heat stack deletion bug in Shade"

6 years agoMerge "Bump pip packages installed in "nsb_setup.sh" script"
Rodolfo Alonso Hernandez [Fri, 16 Feb 2018 18:08:03 +0000 (18:08 +0000)]
Merge "Bump pip packages installed in "nsb_setup.sh" script"

6 years agoMerge "Fix "./docker/Dockerfile.aarch64.patch""
Rodolfo Alonso Hernandez [Fri, 16 Feb 2018 18:07:57 +0000 (18:07 +0000)]
Merge "Fix "./docker/Dockerfile.aarch64.patch""

6 years agoMerge "Remove tool provisioning from DpdkVnfSetupEnvHelper._setup_dpdk"
Rodolfo Alonso Hernandez [Fri, 16 Feb 2018 18:07:26 +0000 (18:07 +0000)]
Merge "Remove tool provisioning from DpdkVnfSetupEnvHelper._setup_dpdk"

6 years agoRefactor benchmark.core.test_plugin.PluginTestCase 57/47257/8
Rodolfo Alonso Hernandez [Wed, 15 Nov 2017 12:35:04 +0000 (12:35 +0000)]
Refactor benchmark.core.test_plugin.PluginTestCase

Tests in benchmark.core.test_plugin.PluginTestCase are not covering
correctly the functionality of the tested class.

YARDSTICK-851

Change-Id: I0b4cda7dbf109776a202167abfde8c9eb268db12
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoNSB: Ixia remove unused traffic_generator arg 37/52137/4
Ross Brattain [Wed, 14 Feb 2018 05:43:55 +0000 (21:43 -0800)]
NSB: Ixia remove unused traffic_generator arg

also fix pylint unittest warnings

JIRA: YARDSTICK-1014
Change-Id: I4ff9769fdca89c6fc1fb9ca62e9f8ea6b174681e
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "Update heat RFC2544 based TC to support SRIOV VF ports."
Emma Foley [Thu, 15 Feb 2018 15:03:27 +0000 (15:03 +0000)]
Merge "Update heat RFC2544 based TC to support SRIOV VF ports."

6 years agoWorkaround to fix Heat stack deletion bug in Shade 89/52189/2
Rodolfo Alonso Hernandez [Thu, 15 Feb 2018 11:42:11 +0000 (11:42 +0000)]
Workaround to fix Heat stack deletion bug in Shade

There is a bug in Shade, in the function "event_utils.poll_for_events".
While Shade is waiting for the Heat stack to be deleted, retrieves
periodically the status of this Heat stack. When the stack is deleted,
the function "get_stack" [1] returns None; then L90 tries to access to
a key of this None object.

[1]https://github.com/openstack-infra/shade/blob/e5bcb705c5b32334cdc1dd9807126298c87b0622/shade/_heat/event_utils.py#L89-L90
Shade function

JIRA: YARDSTICK-1015

Change-Id: I56ea853be41bfd7f63e094d7a3696d97ba078ba4
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoBugFix: Fix to pod context filename 51/52151/2
Xavier Simonart [Wed, 14 Feb 2018 14:32:53 +0000 (15:32 +0100)]
BugFix: Fix to pod context filename

The "context:file" of tc_prox_baremetal_lw_aftr
has an incorrect path, compaired to all other
prox test cases.

Change-Id: Id24e74905f36bc84fdf93d1a2966211ecad55c8b
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
6 years agoUpdate heat RFC2544 based TC to support SRIOV VF ports. 61/51961/4
Mytnyk, Volodymyr [Fri, 9 Feb 2018 14:55:04 +0000 (14:55 +0000)]
Update heat RFC2544 based TC to support SRIOV VF ports.

Updated vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml TC to be
able to run it in heat context with SRIOV VF ports configured in
OpenStack.

E.g.:
  yardstick task start --task-args='{"provider": "sriov"}' ...
  yardstick task start --task-args='{"provider": "sriov"}\
    physical_networks: ["physnet1", "physnet2"]}' ...

Console log: http://paste.openstack.org/show/666931/
Result output: http://paste.openstack.org/show/666930/

JIRA: YARDSTICK-1004

Change-Id: If0dabd0791e4e295782fee0f3de1536474218876
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoMerge "Fixed document for Grafana Port usage"
Rodolfo Alonso Hernandez [Wed, 14 Feb 2018 11:36:03 +0000 (11:36 +0000)]
Merge "Fixed document for Grafana Port usage"

6 years agoNSB: fix test_vpe_vnf.py 35/52135/3
Ross Brattain [Wed, 14 Feb 2018 05:43:10 +0000 (21:43 -0800)]
NSB: fix test_vpe_vnf.py

Change-Id: Ie1da572e2ac00e380b25f8e22f5aa9059dcb5238
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoNSB: fix pylint warnings in test_tg_ixload 33/52133/3
Ross Brattain [Wed, 14 Feb 2018 05:42:45 +0000 (21:42 -0800)]
NSB: fix pylint warnings in test_tg_ixload

Change-Id: Ieb58167d8b20e0de32578b3df105a141507869d5
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoNSB traffic_profile: fix pylint warnings and return partial 31/52131/3
Ross Brattain [Wed, 14 Feb 2018 05:11:11 +0000 (21:11 -0800)]
NSB traffic_profile: fix pylint warnings and return partial

also fix unittests

Change-Id: Ie3026f5a0df8e690271afb71fe36ac531cbae2a0
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoconstants: fix pylint warnings for OSError 39/52139/2
Ross Brattain [Wed, 14 Feb 2018 05:07:36 +0000 (21:07 -0800)]
constants: fix pylint warnings for OSError

IOError and OSError are the same in Python 3?
Anyway we want to ignore ENOENT, not EEXIST

JIRA: YARDSTICK-1012
Change-Id: Ia75e41fecce505685745e0099669e45f3e82a1be
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "Move tests: unit/benchmark"
Rodolfo Alonso Hernandez [Tue, 13 Feb 2018 11:02:45 +0000 (11:02 +0000)]
Merge "Move tests: unit/benchmark"

6 years agoBug fix for Ansible destroy VM script 89/52089/1
John O Loughlin [Mon, 12 Feb 2018 17:31:59 +0000 (17:31 +0000)]
Bug fix for Ansible destroy VM script

Modified the destroy script to ignore errors as 2 possible
problems can cause a play to fail.
1. A VM can be running but undefined.
2. A VM can be destroyed but still running.

JIRA: YARDSTICK-1006

Change-Id: Ib66f48401bfefb598f553cfef31bfa2682f5a584
Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
6 years agoBump pip packages installed in "nsb_setup.sh" script 53/52053/1
Rodolfo Alonso Hernandez [Mon, 12 Feb 2018 17:30:20 +0000 (17:30 +0000)]
Bump pip packages installed in "nsb_setup.sh" script

Bump pip package versions installed in "nsb_setup.sh" script to match
"./requirements.txt" versions.

JIRA: YARDSTICK-1008

Change-Id: Icb0f7e99c1be38848ee7122735ace7c1c715ef5b
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoFix "./docker/Dockerfile.aarch64.patch" 51/52051/1
Rodolfo Alonso Hernandez [Mon, 12 Feb 2018 17:24:33 +0000 (17:24 +0000)]
Fix "./docker/Dockerfile.aarch64.patch"

Patch "./docker/Dockerfile.aarch64.patch" doesn't apply on
"./docker/Dockerfile" due to the latest modification done in the
original file

The original reference lines must be changed.

JIRA: YARDSTICK-1007

Change-Id: I5cfc349fbaf34e8da59608e5b803818e2cd9d295
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMove tests: unit/benchmark 45/48245/15
Emma Foley [Wed, 29 Nov 2017 22:56:21 +0000 (22:56 +0000)]
Move tests: unit/benchmark

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

Change-Id: I30eb4b0aafe0575d0cddbc946108291f21a98ed8
Jira: YARDSTICK-837
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "Create host VMs configuration"
Emma Foley [Mon, 12 Feb 2018 16:57:15 +0000 (16:57 +0000)]
Merge "Create host VMs configuration"

6 years agoMerge "Heatstack create: fix str/bytes error in NamedTemporaryFile.write()"
Emma Foley [Mon, 12 Feb 2018 14:06:58 +0000 (14:06 +0000)]
Merge "Heatstack create: fix str/bytes error in NamedTemporaryFile.write()"

6 years agoMerge "Replace neutron get network id with shade."
Emma Foley [Mon, 12 Feb 2018 13:51:39 +0000 (13:51 +0000)]
Merge "Replace neutron get network id with shade."

6 years agoRemove tool provisioning from DpdkVnfSetupEnvHelper._setup_dpdk 55/51855/3
Rodolfo Alonso Hernandez [Wed, 7 Feb 2018 16:20:56 +0000 (16:20 +0000)]
Remove tool provisioning from DpdkVnfSetupEnvHelper._setup_dpdk

Both "uio" and "igb_uio" drivers are present in the VM image generated
using "./nsb_setup.sh" script. "igb_uio" driver is compiled along with
the DPDK library compilation.

Tool "nsb_setup.sh" provisioning and execution should be removed from
this function because there is not needed anymore.

In case "igb_uio" driver is not loaded, an exception should be raised.

JIRA: YARDSTICK-999

Change-Id: I89174f84ac36d8231587402c96751746cb18e290
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoHeatstack create: fix str/bytes error in NamedTemporaryFile.write() 19/52019/2
Ross Brattain [Mon, 12 Feb 2018 03:24:16 +0000 (19:24 -0800)]
Heatstack create: fix str/bytes error in NamedTemporaryFile.write()

we need to use jsonutils.dump_as_bytes for NamedTemporaryFile.write()

2018-02-11 19:15:01,366 [INFO] yardstick.orchestrator.heat heat.py:563 Creating stack 'yardstick-4359f33e' START
2018-02-11 19:15:01,935 [ERROR] yardstick.benchmark.contexts.heat heat.py:317 stack failed
Traceback (most recent call last):
  File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/heat.py", line 313, in deploy
    timeout=self.heat_timeout)
  File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/orchestrator/heat.py", line 567, in create
    stack.create(self._template, self.heat_parameters, block, timeout)
  File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/orchestrator/heat.py", line 56, in create
    template_file.write(jsonutils.dumps(template))
  File "/home/rbbratta/yardstick_venv3/lib/python3.5/tempfile.py", line 622, in func_wrapper
    return func(*args, **kwargs)
TypeError: a bytes-like object is required, not 'str'

JIRA: YARDSTICK-1005
Change-Id: I504f23b86119b62f3aea5b83a445b97bf810220c
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoCreate host VMs configuration 37/50937/11
John O Loughlin [Mon, 22 Jan 2018 13:57:05 +0000 (13:57 +0000)]
Create host VMs configuration

Using xml files deploy the VM's using the virt plugin.
This code generates the user-data network-config meta-data
and the vm's xml. It also deploys the vm

JIRA: YARDSTICK-963

Change-Id: I97f2d20d0ab75a4444b8b8e6fb65e9e463870af7
Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
6 years agoMerge "Create host networking configuration"
Emma Foley [Mon, 12 Feb 2018 10:12:49 +0000 (10:12 +0000)]
Merge "Create host networking configuration"

6 years agoMerge "vpe: convert DPKD_PORT to LINK ID"
Emma Foley [Mon, 12 Feb 2018 10:06:15 +0000 (10:06 +0000)]
Merge "vpe: convert DPKD_PORT to LINK ID"

6 years agoFixed document for Grafana Port usage 31/52031/1
Abhijit Sinha [Mon, 12 Feb 2018 09:54:41 +0000 (09:54 +0000)]
Fixed document for Grafana Port usage

JIRA: YARDSTICK-992

The Grafana port has been changed to 1948 from 3000, as the
port 3000 may be not free in some environments. The document
still says it uses port 3000, which leads to confusion.

Proposed Solution: The document needs to be updated with
correct Grafana port number "http://host_ip:1948"

Change-Id: I9078783849aec14602585f4d1b3bc293214ae737
Signed-off-by: Abhijit Sinha <abhijit.sinha@intel.com>
6 years agoCreate host networking configuration 27/50927/11
John O Loughlin [Mon, 22 Jan 2018 11:49:43 +0000 (11:49 +0000)]
Create host networking configuration

Create host networking configuration based on the input
configuration file. To define the networks using libvirt,
a temporary XML file is generated for each network.

JIRA: YARDSTICK-962

Change-Id: I24d1c4d2f2e62884675599deb7db255862283ea4
Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
6 years agoProposing Abhijit Sinha as a committer in Yardstick 85/51285/3
Ross Brattain [Tue, 23 Jan 2018 17:31:51 +0000 (09:31 -0800)]
Proposing Abhijit Sinha as a committer in Yardstick

Abhijit Sinha has contributed to the Yardstick project in many ways
Achievements/Contributions include:
- PROX VNF and TG for NFVi characterization tests
- Attending Yardstick meetings
- PROX Grafanfa dashboards

Change-Id: I455ff22b8cb68c6422fc864659668412bed3e9c4
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "cleanup: rm node_ID from yardstick prepare_env file"
Ross Brattain [Mon, 12 Feb 2018 03:05:09 +0000 (03:05 +0000)]
Merge "cleanup: rm node_ID from yardstick prepare_env file"

6 years agoMerge "Remove references to "dpdk_nic_bind" utility"
Emma Foley [Sun, 11 Feb 2018 22:08:55 +0000 (22:08 +0000)]
Merge "Remove references to "dpdk_nic_bind" utility"

6 years agoMerge "BugFIX: NSB-setup build script fix for BM"
Emma Foley [Sun, 11 Feb 2018 22:02:27 +0000 (22:02 +0000)]
Merge "BugFIX: NSB-setup build script fix for BM"

6 years agoMerge "Replace "python-heatclient" with "shade" client"
Rodolfo Alonso Hernandez [Sun, 11 Feb 2018 20:37:22 +0000 (20:37 +0000)]
Merge "Replace "python-heatclient" with "shade" client"

6 years agoModule to manage pip packages 23/51123/9
Rodolfo Alonso Hernandez [Thu, 25 Jan 2018 11:12:57 +0000 (11:12 +0000)]
Module to manage pip packages

This new module provides methods to manage Python PIP packages from
a URL, from a local directory or from a build PIP package.

The implemented commands are:
- Install package.
- Remove package.
- List all installed packages in the system.

JIRA: YARDSTICK-910

Change-Id: I8f7d1b77c0c384b801cc6f5e67d8b45ce7c6bfdf
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Prox vnf descriptor cleanup"
Rodolfo Alonso Hernandez [Fri, 9 Feb 2018 18:08:57 +0000 (18:08 +0000)]
Merge "Prox vnf descriptor cleanup"

6 years agoReplace neutron get network id with shade. 65/51765/10
Shobhi Jain [Mon, 29 Jan 2018 11:49:02 +0000 (11:49 +0000)]
Replace neutron get network id with shade.

Function get_network_id now uses shade client instead of shade client.
Removed redundant function: get_port_id.

JIRA: YARDSTICK-890

Change-Id: I6081477fee39fa78131187e65daf3d7d3a9e439f
Signed-off-by: Shobhi Jain <shobhi.jain@intel.com>
6 years agoMerge "Make 'Scenario' classes plugable"
Emma Foley [Fri, 9 Feb 2018 16:35:38 +0000 (16:35 +0000)]
Merge "Make 'Scenario' classes plugable"

6 years agoMerge "Change nsb_setup.sh to use NSB image instead of raw"
Emma Foley [Fri, 9 Feb 2018 16:18:34 +0000 (16:18 +0000)]
Merge "Change nsb_setup.sh to use NSB image instead of raw"

6 years agoMerge "Delete any previous configuration"
Emma Foley [Fri, 9 Feb 2018 15:49:01 +0000 (15:49 +0000)]
Merge "Delete any previous configuration"

6 years agoReplace "python-heatclient" with "shade" client 01/48401/11
Rodolfo Alonso Hernandez [Mon, 4 Dec 2017 16:24:13 +0000 (16:24 +0000)]
Replace "python-heatclient" with "shade" client

Replaced "python-heatclient" with "shade" client. The "python-heatclient"
library is removed from requirements file. "shade" client version 1.22.2
is the required one for OpenStack Pike release.

shade is the recommended client to interact with OpenStack clouds [1]

[1] https://github.com/openstack-infra/shade

JIRA: YARDSTICK-880

Change-Id: Ibbdfc42a543d2d9ed4cf70d78de3de8cbecd0076
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoDelete any previous configuration 19/50919/8
John O Loughlin [Mon, 22 Jan 2018 09:34:45 +0000 (09:34 +0000)]
Delete any previous configuration

Delete any previous configuration using the virsh module in ansible:
Networks, VMs and local IPs.

JIRA: YARDSTICK-950

Change-Id: Ie4963ab8210ce50ba2e9fa22642ec27a3b2031e1
Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
6 years agoProx vnf descriptor cleanup 01/51901/1
Abhijit Sinha [Thu, 8 Feb 2018 16:05:54 +0000 (16:05 +0000)]
Prox vnf descriptor cleanup

JIRA: YARDSTICK-1003

- There are redundant files for Prox vnf descriptors
tg and vnf for 1, 2 and 4 port set-up.
- We can use one vnf descriptor file per prox tg and prox vnf,
reducing the number of redundant files.
- Tested on Prox BM L2fwd 2 and 4 port test case.

Change-Id: I4c61fd11725121f19392443460ac2ad39c934e2d
Signed-off-by: Abhijit Sinha <abhijit.sinha@intel.com>
6 years agoChange nsb_setup.sh to use NSB image instead of raw 95/51895/1
Mytnyk, Volodymyr [Thu, 8 Feb 2018 13:22:46 +0000 (13:22 +0000)]
Change nsb_setup.sh to use NSB image instead of raw

The `ansible/roles/create_samplevnfs_image/tasks/main.yml` ansible role
uses raw NSB image (yardstick-xenial-server.raw) to upload it to
OpenStack instead QCOW2 NSB converted image (yardstick-nsb-image.img
which is converted from the raw). Changed the create_samplevnfs_image
role to to use NSB image.

JIRA: YARDSTICK-1000

Change-Id: Ieda9f14e6882ca854331b4eccc9a4cb390b6a475
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoMerge "Improve ovs_deploy.bash script"
Ross Brattain [Wed, 7 Feb 2018 19:04:23 +0000 (19:04 +0000)]
Merge "Improve ovs_deploy.bash script"

6 years agoMerge "Auto Generated INFO.yaml file"
Ross Brattain [Wed, 7 Feb 2018 19:04:03 +0000 (19:04 +0000)]
Merge "Auto Generated INFO.yaml file"