yardstick.git
6 years agoChange IxNextgen library load 71/58971/2
Rodolfo Alonso Hernandez [Mon, 25 Jun 2018 07:47:00 +0000 (08:47 +0100)]
Change IxNextgen library load

Changed IxNextgen library load in IXIA RFC2544 traffic generator call.

JIRA: YARDSTICK-1116

Change-Id: Icbdc5f1255dd2676584f3a1be767287e55e71f4a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Configure ACL via static file"
Abhijit Sinha [Fri, 22 Jun 2018 13:24:20 +0000 (13:24 +0000)]
Merge "Configure ACL via static file"

6 years agoConfigure ACL via static file 09/58109/9
Mytnyk, Volodymyr [Thu, 31 May 2018 09:03:10 +0000 (10:03 +0100)]
Configure ACL via static file

This patch allows user to configure ACL/vFW SampleVNF ACL
via configuration file provided in TC definition. The
Yardstick applies the rules to SampleVNF from specified config
file + rules generated by Yardstick (default rules).

The example of SampleVNF ACL CLI commands generated/applied
by Yardstick can be found at (using default ACL config file):

ACL VNF ACL CLI commands:
  acl/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
  http://paste.openstack.org/show/723303/

vFW VNF ACL CLI commands:
  vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
  http://paste.openstack.org/show/723304/

Change-Id: I76a630261a982083b628e3985fc3bec14ca495db
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoMerge "Cleanup EnvCommand test cases"
Rodolfo Alonso Hernandez [Thu, 21 Jun 2018 13:28:35 +0000 (13:28 +0000)]
Merge "Cleanup EnvCommand test cases"

6 years agoMerge "Add scale out TCs with availability zone support"
Rodolfo Alonso Hernandez [Thu, 21 Jun 2018 13:06:26 +0000 (13:06 +0000)]
Merge "Add scale out TCs with availability zone support"

6 years agoMerge "Document for Fraser test case results"
Jing Lu [Thu, 21 Jun 2018 02:35:29 +0000 (02:35 +0000)]
Merge "Document for Fraser test case results"

6 years agoNSB setup inventory name changed 53/58753/1
Rodolfo Alonso Hernandez [Wed, 20 Jun 2018 10:26:06 +0000 (11:26 +0100)]
NSB setup inventory name changed

NSB setup script [1] fails when retrieving the inventory file. This script
is going to be deprecated but we still support it.

[1] https://github.com/opnfv/yardstick/blob/master/nsb_setup.sh

JIRA: YARDSTICK-1247

Change-Id: I3798531652b0e3a1a007d1d9434998fdd54645d7
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Refactor RFC2455 TRex traffic profile injection"
Rodolfo Alonso Hernandez [Wed, 20 Jun 2018 07:37:34 +0000 (07:37 +0000)]
Merge "Refactor RFC2455 TRex traffic profile injection"

6 years agoMerge "Add "duration" parameter to test case definition"
Rodolfo Alonso Hernandez [Wed, 20 Jun 2018 07:37:29 +0000 (07:37 +0000)]
Merge "Add "duration" parameter to test case definition"

6 years agoMerge "Update pmd/lcore mask for OVS-DPDK context"
Emma Foley [Tue, 19 Jun 2018 08:58:27 +0000 (08:58 +0000)]
Merge "Update pmd/lcore mask for OVS-DPDK context"

6 years agoUpdate pmd/lcore mask for OVS-DPDK context 53/58653/2
Stepan Andrushko [Mon, 18 Jun 2018 11:23:16 +0000 (14:23 +0300)]
Update pmd/lcore mask for OVS-DPDK context

Added default values for pmd/lcore mask to the SampleVNF test case for
Trex TG.
lcore mask - specifies the CPU cores used for non-datapath OVS-DPDK
threads.
pmd mask - specifies the CPU cores used for data path packet processing.
Updated documentation for Standalone OVS-DPDK context.

JIRA: YARDSTICK-1246

Change-Id: I8d008a185c94d2047ffa4769d7974bc4d283fd7f
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
6 years agoCleanup EnvCommand test cases 21/58421/2
Rodolfo Alonso Hernandez [Mon, 11 Jun 2018 13:58:49 +0000 (14:58 +0100)]
Cleanup EnvCommand test cases

Refactor exception testing and remove console output.

JIRA: YARDSTICK-1235

Change-Id: I6376c3637ffb6476a2578ef963cd2420b63bab52
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoRefactor RFC2455 TRex traffic profile injection 29/57929/26
Rodolfo Alonso Hernandez [Thu, 24 May 2018 13:02:03 +0000 (14:02 +0100)]
Refactor RFC2455 TRex traffic profile injection

The TRex RFC2455 traffic profile configures the duration of the traffic
injection. Once the traffic is started, the traffic profile should poll
the traffic injector client for the injection status. When the injection
is finished, the traffic execution should end.

Instead of this, the traffic profile waits a fixed time (using time.sleep
method) and then stops the traffic. This approach is not accurate and may
cause shorter injection periods.

JIRA: YARDSTICK-1197

Change-Id: I5090df81ef4ec7945ff6c1aff070656b48e0fe77
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoAdd "duration" parameter to test case definition 59/57759/11
Rodolfo Alonso Hernandez [Wed, 23 May 2018 08:16:39 +0000 (09:16 +0100)]
Add "duration" parameter to test case definition

Add "duration" parameter to test case definition, in scenario:options
section. This parameter will be rendered in the traffic profile.

If the parameter is not present in the test case scenario options, the
default time written in the traffic profile options will be 30 seconds
(TrafficProfile.DEFAULT_DURATION = 30). If the traffic profile
injection time is not defined, the default injection time will be 30
seconds.

  testcase:scenario:options:duration (default = 30)
      render --> traffic_profile:duration
          parse --> TrafficProfile.duration (default = 30)

Target traffic profiles (RFC2544):
  - RFC2544Profile
  - IXIARFC2544Profile

JIRA: YARDSTICK-1194

Change-Id: I968922e6bb882d7ee15aa1c4db4037face7a3492
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge changes from topic 'convert-sla-asserts-to-raises'
Abhijit Sinha [Mon, 18 Jun 2018 10:54:49 +0000 (10:54 +0000)]
Merge changes from topic 'convert-sla-asserts-to-raises'

* changes:
  Replace MagicMocks with Mocks from SearchRunner UTs
  Add vsperf_dpdk SLA validation unit test cases
  Cleanup vsperf unit tests

6 years agoMerge "Convert SLA asserts to raises"
Abhijit Sinha [Mon, 18 Jun 2018 10:53:37 +0000 (10:53 +0000)]
Merge "Convert SLA asserts to raises"

6 years agoStop running tests from tests/unit 67/52367/14
Emma Foley [Tue, 20 Feb 2018 16:19:50 +0000 (16:19 +0000)]
Stop running tests from tests/unit

Since all tests have been moved into yardstick/tests,
temperary code to run tests in both tests and yardstick/tests
should be removed.

JIRA: YARDSTICK-837
Change-Id: I3bf3c9ece9bc8457d992e53ee8cc5cadb1dec8f0
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoMerge "Cleanup BaseMonitor unit tests"
Rodolfo Alonso Hernandez [Mon, 18 Jun 2018 08:42:12 +0000 (08:42 +0000)]
Merge "Cleanup BaseMonitor unit tests"

6 years agoCleanup BaseMonitor unit tests 75/58375/3
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 15:43:40 +0000 (16:43 +0100)]
Cleanup BaseMonitor unit tests

Stop multiprocessing queues to avoid BrokenPipe exception.
Assert correctly the raise of exceptions.

JIRA: YARDSTICK-1229

Change-Id: I1a1c92c49a439283a35c9a69b59d6866f35ec09a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoReplace MagicMocks with Mocks from SearchRunner UTs 23/57323/8
Miikka Koistinen [Fri, 11 May 2018 12:20:24 +0000 (15:20 +0300)]
Replace MagicMocks with Mocks from SearchRunner UTs

This patch replaces unneeded MagicMocks with Mocks from TestSearchRunner
and TestSearchRunnerHelper.

JIRA: YARDSTICK-1169

Change-Id: I30a7bbc71c0b1062e6c0dfe634a5a8f1ecb7c684
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoConvert SLA asserts to raises 61/57061/9
Miikka Koistinen [Fri, 8 Jun 2018 10:54:48 +0000 (13:54 +0300)]
Convert SLA asserts to raises

This commit converts Python assertions to a custom exception in all
places where SLA validation is checked with an assertion.

This commit also fixes all emerged pylint errors.

JIRA: YARDSTICK-966

Change-Id: If771ed03b2cbc0a43a57fcfb9293f18740b3ff80
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoAdd vsperf_dpdk SLA validation unit test cases 73/58373/4
Miikka Koistinen [Fri, 8 Jun 2018 14:53:29 +0000 (17:53 +0300)]
Add vsperf_dpdk SLA validation unit test cases

JIRA: YARDSTICK-1228

Change-Id: Ic32f2bcd7f8bdf718c9c266666409d32ecab4924
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoCleanup vsperf unit tests 71/58371/4
Miikka Koistinen [Fri, 8 Jun 2018 14:42:41 +0000 (17:42 +0300)]
Cleanup vsperf unit tests

Remove unneeded mock return values and improve the test execution.

Add unit test cases about SLA validation.

JIRA: YARDSTICK-1227

Change-Id: I90b8b9c4cb6a57ae4b0869e84af0da43ee22a670
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
6 years agoMerge "assert[Greater,Equal] -> assert_{,not_}called"
Rodolfo Alonso Hernandez [Fri, 15 Jun 2018 08:41:37 +0000 (08:41 +0000)]
Merge "assert[Greater,Equal] -> assert_{,not_}called"

6 years agoMerge "Add "host_name_separator" variable to Context class"
Rodolfo Alonso Hernandez [Thu, 14 Jun 2018 17:07:41 +0000 (17:07 +0000)]
Merge "Add "host_name_separator" variable to Context class"

6 years agoMerge "Install dependencies: bare-metal, standalone"
Emma Foley [Thu, 14 Jun 2018 14:50:01 +0000 (14:50 +0000)]
Merge "Install dependencies: bare-metal, standalone"

6 years agoInstall dependencies: bare-metal, standalone 89/57389/5
Stepan Andrushko [Tue, 15 May 2018 09:46:42 +0000 (12:46 +0300)]
Install dependencies: bare-metal, standalone

This will install all dependencies needed on the bare-metal and standalone
physical nodes.

JIRA: YARDSTICK-1177

Change-Id: I719ca64d4da098d6dd60597e5fc4ad6f4715f79e
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoStop time.sleep mocks in unit tests 33/58533/1
Rodolfo Alonso Hernandez [Thu, 14 Jun 2018 13:14:27 +0000 (14:14 +0100)]
Stop time.sleep mocks in unit tests

tVpeApproxVnf and TestProxSocketHelper classes mock "time.sleep" in
the setUp method without deleting it at the end of the test execution.

JIRA: YARDSTICK-1243

Change-Id: Iff31d9c7b400ad8a47f37792aeb0d20328b9d9e1
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoassert[Greater,Equal] -> assert_{,not_}called 93/53593/8
Emma Foley [Mon, 12 Mar 2018 15:07:59 +0000 (15:07 +0000)]
assert[Greater,Equal] -> assert_{,not_}called

assertEqual(mock_xxx.call_count, 1) -> mock_xxx.assert_called_once
assertEqual(mock_xxx.call_count, 0) -> mock_xxx.assert_not_called
assertGreater(mock.call_count, 0) -> mock.assert_called()
assertGreaterEqual(mock.call_count, 1) -> mock.assert_called()

JIRA: YARDSTICK-1069
Change-Id: I890084d120c8e78304e169e2a0e5d30011a41525
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoImprove IXIA IxNetwork library and traffic profile (4) 93/55593/10
Rodolfo Alonso Hernandez [Thu, 12 Apr 2018 14:08:58 +0000 (15:08 +0100)]
Improve IXIA IxNetwork library and traffic profile (4)

This patch implements an active wait for the traffic injection. Once the
traffic is started, the traffic generator class will poll periodically the
IXIA traffic generator chassis to retrieve the status of the traffic
("started", "stopped").

Now the latency statistics are retrieved and reported for each injection
period.

JIRA: YARDSTICK-1116

Change-Id: I4422e2c88b4fc97b7cac3de8a82b2d75467c4117
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoImprove IXIA IxNetwork library and traffic profile (3) 35/55435/10
Rodolfo Alonso Hernandez [Wed, 11 Apr 2018 17:32:52 +0000 (18:32 +0100)]
Improve IXIA IxNetwork library and traffic profile (3)

This patch modifies IP packet parameters.

"IxNextgen.update_ip_packet" modifies the L3 packet according to the
test case and setup the IP addresses.

JIRA: YARDSTICK-1116

Change-Id: I46ff75ab1989d0e6f5cc876418a015386717e06f
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoImprove IXIA IxNetwork library and traffic profile (2) 21/55321/14
Rodolfo Alonso Hernandez [Tue, 10 Apr 2018 08:15:55 +0000 (09:15 +0100)]
Improve IXIA IxNetwork library and traffic profile (2)

This patch modifies the way the packet frame and the flow is configured
before the injection.

"IxNextgen.update_frame" modifies the L2 frame according to the
test case and setup the frame rate, frame size, traffic injection
duration and MAC addresses.

JIRA: YARDSTICK-1116

Change-Id: Ife08f15a4eda24d7835c92c4172b450854d112ee
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoImprove IXIA IxNetwork library and traffic profile (1) 11/55311/15
Rodolfo Alonso Hernandez [Tue, 10 Apr 2018 07:39:16 +0000 (08:39 +0100)]
Improve IXIA IxNetwork library and traffic profile (1)

This patch introduces a new way to configure the TCL IxNetwork server.
All the configuration is done using the TCL API, removing the need of
using the pre-saved configuration file.

"IxNextgen.assign_ports" creates and assigns the virtual ports for each
physical port defined in the test case.

"IxNextgen.create_traffic_item" creates one traffic item and two flow
groups per pair of ports, in both directions.

"IxNextgen.gt_statistics" retrieves only the required statistics to
generate the samples blob in the traffic generator.

JIRA: YARDSTICK-1116

Change-Id: I8f1c0c55e99c274b2ed8276ed9a385c502e16d93
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
6 years agoMerge changes from topics 'YARDSTICK-1218', 'YARDSTICK-1216', 'YARDSTICK-1215', ...
Rodolfo Alonso Hernandez [Tue, 12 Jun 2018 07:27:51 +0000 (07:27 +0000)]
Merge changes from topics 'YARDSTICK-1218', 'YARDSTICK-1216', 'YARDSTICK-1215', 'YARDSTICK-1214'

* changes:
  Move IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptions
  Move ErrorClass definition to exceptions module
  Convert SSH custom exceptions to Yardstick exceptions
  Remove AnsibleCommon class method mock

6 years agoMove tests: unit/network_services/vnf_generic 71/56771/8
Emma Foley [Fri, 27 Apr 2018 10:30:20 +0000 (10:30 +0000)]
Move tests: unit/network_services/vnf_generic

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

JIRA: YARDSTICK-837

Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Change-Id: I3d95537a64bde923703421510d27bac0d6dd92bd

6 years agoMerge "Remove print out of logger exception in TestUtils"
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 13:11:44 +0000 (13:11 +0000)]
Merge "Remove print out of logger exception in TestUtils"

6 years agoMerge "Cleanup TestYardstickNSCli class"
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 13:11:38 +0000 (13:11 +0000)]
Merge "Cleanup TestYardstickNSCli class"

6 years agoMerge "Remove __init__ method overriding in HeatContextTestCase"
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 10:38:59 +0000 (10:38 +0000)]
Merge "Remove __init__ method overriding in HeatContextTestCase"

6 years agoMerge "Standalone XML machine type is not longer valid"
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 10:38:42 +0000 (10:38 +0000)]
Merge "Standalone XML machine type is not longer valid"

6 years agoMerge "Error when adding SR-IOV interfaces in SR-IOV context"
Rodolfo Alonso Hernandez [Fri, 8 Jun 2018 10:38:35 +0000 (10:38 +0000)]
Merge "Error when adding SR-IOV interfaces in SR-IOV context"

6 years agoRemove print out of logger exception in TestUtils 43/58343/3
Rodolfo Alonso Hernandez [Thu, 7 Jun 2018 16:50:30 +0000 (17:50 +0100)]
Remove print out of logger exception in TestUtils

Remove print out of logger exception while executing
"TestUtils.test_parse_ini_file_missing_section_header" test case.

JIRA: YARDSTICK-1225

Change-Id: I2a4e447c5d119fb9585b477d74f3d0a002c8278e
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoCleanup TestYardstickNSCli class 41/58341/3
Rodolfo Alonso Hernandez [Thu, 7 Jun 2018 16:13:09 +0000 (17:13 +0100)]
Cleanup TestYardstickNSCli class

Remove the console output during the test execution. Refactor the
"parse_arguments" test case.

JIRA: YARDSTICK-1224

Change-Id: I68954b986dbea70752adb121e0445e162bdfddcb
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Fix latency in l2 and l3 PROX generator config files"
Abhijit Sinha [Thu, 7 Jun 2018 12:06:13 +0000 (12:06 +0000)]
Merge "Fix latency in l2 and l3 PROX generator config files"

6 years agoMerge "Use TRex release v2.41 to support both x86 and aarch64"
Rodolfo Alonso Hernandez [Thu, 7 Jun 2018 11:58:07 +0000 (11:58 +0000)]
Merge "Use TRex release v2.41 to support both x86 and aarch64"

6 years agoMerge "Remove instantiated contexts in "test_task""
Rodolfo Alonso Hernandez [Thu, 7 Jun 2018 11:57:31 +0000 (11:57 +0000)]
Merge "Remove instantiated contexts in "test_task""

6 years agoFix latency in l2 and l3 PROX generator config files 35/58135/3
Xavier Simonart [Fri, 1 Jun 2018 08:55:33 +0000 (10:55 +0200)]
Fix latency in l2 and l3 PROX generator config files

JIRA: YARDSTICK-1211

Change-Id: I0899d470ca5a0ec7d42d36a9ff9c39cc7369310c
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
6 years agoMerge "Add NSB PROX Baremetal dashboard for L3"
Abhijit Sinha [Wed, 6 Jun 2018 12:44:03 +0000 (12:44 +0000)]
Merge "Add NSB PROX Baremetal dashboard for L3"

6 years agoMerge "Fix NSB NFVi metrics accuracy"
Abhijit Sinha [Wed, 6 Jun 2018 12:43:51 +0000 (12:43 +0000)]
Merge "Fix NSB NFVi metrics accuracy"

6 years agoMerge "Bump oslo.messaging version to 5.36.0"
Rodolfo Alonso Hernandez [Tue, 5 Jun 2018 16:31:08 +0000 (16:31 +0000)]
Merge "Bump oslo.messaging version to 5.36.0"

6 years agoFix NSB NFVi metrics accuracy 37/58137/7
Xavier Simonart [Fri, 1 Jun 2018 10:16:38 +0000 (12:16 +0200)]
Fix NSB NFVi metrics accuracy

JIRA: YARDSTICK-1212

Change-Id: Ic4d4a3c00e4e278f4de06cc176ff663892895569
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
6 years agoMove IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptions 57/58157/1
Rodolfo Alonso Hernandez [Fri, 1 Jun 2018 15:54:11 +0000 (16:54 +0100)]
Move IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptions

JIRA: YARDSTICK-1218

Change-Id: I804065e9bce3e728f5bf9e756a78df8fd28f74ac
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMove ErrorClass definition to exceptions module 49/58149/2
Rodolfo Alonso Hernandez [Fri, 1 Jun 2018 12:49:38 +0000 (13:49 +0100)]
Move ErrorClass definition to exceptions module

JIRA: YARDSTICK-1216

Change-Id: I82556e1d1b0c723221a58e188067cbce560b8338
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoConvert SSH custom exceptions to Yardstick exceptions 47/58147/1
Rodolfo Alonso Hernandez [Fri, 1 Jun 2018 11:21:20 +0000 (12:21 +0100)]
Convert SSH custom exceptions to Yardstick exceptions

JIRA: YARDSTICK-1215

Change-Id: I5ecfd3dccd91b07cd8de5309dfa1a372eff16ed0
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoRemove AnsibleCommon class method mock 41/58141/1
Rodolfo Alonso Hernandez [Fri, 1 Jun 2018 10:48:33 +0000 (11:48 +0100)]
Remove AnsibleCommon class method mock

In [1], several methods in "AnsibleCommon" class are mocked, but those
mocks are not removed after the test execution. Depending on the test
execution order, this affects other results.

[1] https://github.com/opnfv/yardstick/blob/stable/fraser/yardstick/tests/unit/service/test_environment.py#L20

JIRA: YARDSTICK-1214

Change-Id: I85ef702b3b5b2fda5cf453a21b9f0bec61b155f0
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoAdd NSB PROX Baremetal dashboard for L3 39/58139/1
Xavier Simonart [Fri, 1 Jun 2018 10:28:29 +0000 (12:28 +0200)]
Add NSB PROX Baremetal dashboard for L3

JIRA: YARDSTICK-1213

Change-Id: Id96edc5d23d3530110466e1cf69327c3e267e4c0
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
6 years agoRemove __init__ method overriding in HeatContextTestCase 33/58133/1
Rodolfo Alonso Hernandez [Fri, 1 Jun 2018 08:38:28 +0000 (09:38 +0100)]
Remove __init__ method overriding in HeatContextTestCase

Unit test case clases should implement setUp() method instead of overriding
__init__() method.

JIRA: YARDSTICK-1210

Change-Id: I0f9a10575f9aeebf76dfa84619e59589b3ea83f6
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoRemove instantiated contexts in "test_task" 23/58123/1
Rodolfo Alonso Hernandez [Thu, 31 May 2018 17:54:31 +0000 (18:54 +0100)]
Remove instantiated contexts in "test_task"

During unit testing, all contexts instantiated must be deleted at the end
of the test to avoid clashes with other tests.

JIRA: YARDSTICK-1209

Change-Id: I1994b915001b4a518e096833a0c5d33febbd8de6
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoReplace glance delete image with shade client. 75/55975/11
Shobhi Jain [Thu, 12 Apr 2018 12:38:21 +0000 (13:38 +0100)]
Replace glance delete image with shade client.

Remove get_image_id function.
Function delete_image now uses shade client.

JIRA: YARDSTICK-892

Change-Id: I6e8510dfa49aa14786ed7ac3382b85c4e699fb9e
Signed-off-by: Shobhi Jain <shobhi.jain@intel.com>
6 years agoMerge "Replace glance create image with shade client."
Rodolfo Alonso Hernandez [Thu, 31 May 2018 08:27:29 +0000 (08:27 +0000)]
Merge "Replace glance create image with shade client."

6 years agoMerge "Set "cmd2" library to version 0.8.6"
Rodolfo Alonso Hernandez [Wed, 30 May 2018 13:05:48 +0000 (13:05 +0000)]
Merge "Set "cmd2" library to version 0.8.6"

6 years agoReplace glance create image with shade client. 65/55965/11
Shobhi Jain [Wed, 11 Apr 2018 08:34:22 +0000 (09:34 +0100)]
Replace glance create image with shade client.

Function create_image now uses shade client.

JIRA: YARDSTICK-892

Change-Id: Ia41d9ce702a1f24031080f8a365c1b2bd9ac9faa
Signed-off-by: Shobhi Jain <shobhi.jain@intel.com>
6 years agoPTL change from Ross Brattain to Mingjiang Li (Rex) 07/58007/2
Ross Brattain [Tue, 29 May 2018 00:15:24 +0000 (17:15 -0700)]
PTL change from Ross Brattain to Mingjiang Li (Rex)

results of Yardstick PTL election

https://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_b99c79421ce70058&rkey=c39fb1d5405efc15

Change-Id: I1a71f4415a737baec4bd44085ac3115fb86bd560
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoBump oslo.messaging version to 5.36.0 01/58001/2
Rodolfo Alonso Hernandez [Mon, 28 May 2018 15:25:26 +0000 (16:25 +0100)]
Bump oslo.messaging version to 5.36.0

A decoding error was introduced in the AMQP libraries installed by
oslo.messaging version 5.30.2.

Error example:

File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/
        .tox/functional/local/lib/python2.7/site-packages/amqp/utils.py",
         line 89, in str_to_bytes
  return s.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd2 in position 3: ordinal not in range(128)

This problem is fixed in the library versions installed in version 5.36.0.

JIRA: YARDSTICK-1204

Change-Id: I4719d8e0a0891ebbb3b87c31d593d15f897ef53a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoSet "cmd2" library to version 0.8.6 03/58003/1
Rodolfo Alonso Hernandez [Mon, 28 May 2018 16:55:51 +0000 (17:55 +0100)]
Set "cmd2" library to version 0.8.6

Last "cmd2" version, 0.9.0, removed Python 2+ support [1]. In order to
support Python 2.7, version 0.8.6 (or earlier) must be installed.

[1] https://github.com/python-cmd2/cmd2/commit/e54e5a5b891548df4df0acb04b6e607ad8f871b1

JIRA: YARDSTICK-1205

Change-Id: Ic2fe57438720de5b21c75f9456e653c6facf3ae9
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "PROX: [WIP] Added scale up TCs."
Rodolfo Alonso Hernandez [Mon, 28 May 2018 07:10:49 +0000 (07:10 +0000)]
Merge "PROX: [WIP] Added scale up TCs."

6 years agoMerge "[release] Yardstick release note 6.1.0"
Ross Brattain [Fri, 25 May 2018 21:13:35 +0000 (21:13 +0000)]
Merge "[release] Yardstick release note 6.1.0"

6 years agoAdd scale out TCs with availability zone support 09/54909/6
Mytnyk, Volodymyr [Wed, 4 Apr 2018 15:30:08 +0000 (16:30 +0100)]
Add scale out TCs with availability zone support

Add PROX and vFW scale out TCs with availability zone
configuration. To be able to specify the availability
zone for specific VNF or TG the following example of
command line option shoud be used:

  yardstick -d task start --task-args='{
    "num_vnfs": 2, "availability_zone":
     {
      "vnf_0": "az_0", "tg_0": "az_0",
      "vnf_1": "az_1", "tg_1": "az_1",
      ... }}' <SCALE-OUT-TC-FILE>

The steps to run new scale-out TCs is documented in the
userguide.

JIRA: YARDSTICK-1203

Change-Id: Ib3878485c5483981fe9b48ab02995712bcc31ae5
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
6 years agoBugfix: Yardstick GUI build fails on Fraser 6.1.0 79/57879/1
rexlee8776 [Fri, 25 May 2018 07:49:43 +0000 (07:49 +0000)]
Bugfix: Yardstick GUI build fails on Fraser 6.1.0

Yardstick web GUI build fails due to angular version

JIRA: YARDSTICK-1202

Change-Id: Ie075d18cb2eac2070267759978cbdb7c784599ef
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoDockerfile: add ping for prepare_env.sh 21/57821/1
Ross Brattain [Thu, 24 May 2018 14:17:32 +0000 (07:17 -0700)]
Dockerfile: add ping for prepare_env.sh

INFO: Creating openstack credentials ..
Verifying connectivity to 10.20.0.2...
/home/opnfv/repos/yardstick//tests/ci/prepare_env.sh: line 56: ping: command not found
/home/opnfv/repos/yardstick//tests/ci/prepare_env.sh: line 56: ping: command not found

https://build.opnfv.org/ci/view/yardstick/job/yardstick-fuel-baremetal-daily-master/2400/console

JIRA: YARDSTICK-1201

Change-Id: Ib9e139046724ece4be7a85cc1ae74cbb0ec8e203
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoUse TRex release v2.41 to support both x86 and aarch64 25/57725/4
trevortao [Wed, 23 May 2018 04:42:28 +0000 (12:42 +0800)]
Use TRex release v2.41 to support both x86 and aarch64

Currently Yardstick uses Cisco's TRex release v2.28 which
can only support x86, for the consideration of
multi-arch support of Yardstick, the trex release v2.41
is now available and it can support multi-arch(both x86
and aarch64) now.

For the external lib of v2.41, for x86_64, it looks like:
external_libs/pyzmq-14.5.0/python2/intel/ucs4/64bit/zmq
for arm64, it looks like:
external_libs/pyzmq-14.5.0/python2/arm/ucs4/64bit/zmq
So replace the trex release v2.28 with v2.41.

JIRA: YARDSTICK-1191

Change-Id: I22dc3499283d48a89da1145cc480fd7d2a93a1cc
Signed-off-by: trevortao <trevor.tao@arm.com>
6 years agoTest case spec for SDN controller resilience in HA config 91/50991/7
Georg Kunz [Tue, 23 Jan 2018 12:43:24 +0000 (13:43 +0100)]
Test case spec for SDN controller resilience in HA config

This is the test case specification for verifying the HA capabilities
of a SDN controller running in a HA configuration.

JIRA: HA-33
JIRA: YARDSTICK-955

Change-Id: I016ec1ef7bb51b79f8c21440abf8f04c923ef9dc
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
6 years ago[release] Yardstick release note 6.1.0 13/57513/3
rexlee8776 [Thu, 17 May 2018 02:28:30 +0000 (10:28 +0800)]
[release] Yardstick release note 6.1.0

JIRA: YARDSTICK-1182

Change-Id: I3baca52e9bc0360b43f2ce61ea2f166a70e8c90a
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoStandalone XML machine type is not longer valid 71/57671/1
Rodolfo Alonso Hernandez [Tue, 22 May 2018 10:47:52 +0000 (11:47 +0100)]
Standalone XML machine type is not longer valid

Current XML machine type definition [1] is not longer valid. According to
[2], the default machine type should be "pc-i440fx-xenial".

NOTE: StandAlone physical nodes are deployed using Ubuntu OS.

[1]https://github.com/opnfv/yardstick/blob/master/yardstick/benchmark/contexts/standalone/model.py#L48
[2]https://wiki.ubuntu.com/QemuKVMMigration#Upgrade_machine_type

JIRA: YARDSTICK-1192

Change-Id: I77b1379aaf416b1225e91af2c3ac746cb101f774
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoError when adding SR-IOV interfaces in SR-IOV context 59/57659/4
Rodolfo Alonso Hernandez [Mon, 21 May 2018 16:38:50 +0000 (17:38 +0100)]
Error when adding SR-IOV interfaces in SR-IOV context

Method "add_sriov_interfaces" in Libvirt class throws an error when trying
to add SR-IOV interfaces in the XML file.

Function "setup_sriov_context" in SriovContext if passing to
"_enable_interfaces" method the XML file name instead of passing the XML
string.

JIRA: YARDSTICK-1189

Change-Id: If021c102311ecb98cb26934f78067dfc720e4291
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoBugfix: openrc api dump should be safe_dump 61/57261/4
rexlee8776 [Wed, 9 May 2018 09:52:27 +0000 (09:52 +0000)]
Bugfix: openrc api dump should be safe_dump

1. fix safe_dump
2. fix pep8 problem, use flask_restapi custom error handling [1]

[1] https://flask-restful.readthedocs.io/en/latest/extending.html#custom-error-handlers

JIRA: YARDSTICK-1165

RestApi openrc dump clouds.yaml error

it now use yarml.dump, should be yaml.safe_dump.

dump would gererate !!python/unicode and cause error when upload openrc file in gui

Change-Id: Id3e85f7ba7d4967277ef79109b07d7552179e5db
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoAdd "host_name_separator" variable to Context class 45/57645/1
Rodolfo Alonso Hernandez [Mon, 21 May 2018 11:53:03 +0000 (12:53 +0100)]
Add "host_name_separator" variable to Context class

This feature will provide Kubernetes context the ability to handle the
context name inside the class itself, providing to the developer an
abstraction of the possible naming limitations in Kubernetes. E.g.:
"dot" character in Pod names is no allowed [1].

[1] https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/issues/58

JIRA: YARDSTICK-1188

Change-Id: I82121f970b550170357a443b6340be7900602a57
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
6 years agoMerge "Test case spec for SDN Virtual Switch resilience."
Emma Foley [Sun, 20 May 2018 20:29:53 +0000 (20:29 +0000)]
Merge "Test case spec for SDN Virtual Switch resilience."

6 years agoMerge changes from topics 'YARDSTICK-1154', 'YARDSTICK-1160'
Rodolfo Alonso Hernandez [Thu, 17 May 2018 14:30:05 +0000 (14:30 +0000)]
Merge changes from topics 'YARDSTICK-1154', 'YARDSTICK-1160'

* changes:
  Kubernetes API "delete_service" missing parameter
  Bump Kubernetes Python client to version 6.0.0
  Avoid "volumeMounts" with "configMap" fixed permissions

6 years agoMerge "Parse "dispatcher" options correctly from InfluxDB client"
Rodolfo Alonso Hernandez [Thu, 17 May 2018 14:29:21 +0000 (14:29 +0000)]
Merge "Parse "dispatcher" options correctly from InfluxDB client"

6 years agoMerge "Fix PEP8 errors in prox_binseach"
Abhijit Sinha [Thu, 17 May 2018 10:47:49 +0000 (10:47 +0000)]
Merge "Fix PEP8 errors in prox_binseach"

6 years agoDocument for Fraser test case results 85/57385/9
JingLu5 [Tue, 15 May 2018 08:22:50 +0000 (08:22 +0000)]
Document for Fraser test case results

JIRA: YARDSTICK-1132

In B and C releases, Yardstick provides an overview of the results of test cases developed by Yardstick, executed on OPNFV community labs.

This work has not been done for the D and E releases, although result analysis for Euphrates is available at (https://wiki.opnfv.org/display/yardstick/Euphrates+release+test+results+analysis). We plan to restart to provide a basic test case results analysis for Fraser and compare performance data with Euphrates.

This JIRA ticket will be resolved into 3 commits: 1. Document for E release test case data; 2. Document for F release test case data; 3. Document for results analysis.

Change-Id: I3c59851c8878ba37145e6ad6958272a62bad3f52
Signed-off-by: JingLu5 <lvjing5@huawei.com>
6 years agoMerge "Add --hwlb options as a command line argument for SampleVNF"
Rodolfo Alonso Hernandez [Wed, 16 May 2018 10:26:02 +0000 (10:26 +0000)]
Merge "Add --hwlb options as a command line argument for SampleVNF"

6 years agoAdd --hwlb options as a command line argument for SampleVNF 01/56901/8
Chornyi, TarasX [Mon, 30 Apr 2018 14:17:26 +0000 (17:17 +0300)]
Add --hwlb options as a command line argument for SampleVNF

To enable HWLB queues in samplevnf on supported hardware
(e.g Intel  Fortville) --hwlb option needs to be passed
as a VNF command line argument.

JIRA: YARDSTICK-1159

Change-Id: I6e5c098dc71a711252b545c7622ee52085fa81f0
Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
6 years agoMerge "Move tests: unit/network_services/helpers"
Rodolfo Alonso Hernandez [Wed, 16 May 2018 08:40:48 +0000 (08:40 +0000)]
Merge "Move tests: unit/network_services/helpers"

6 years agoMerge "Move tests: unit/network_services/nfvi"
Rodolfo Alonso Hernandez [Wed, 16 May 2018 08:40:42 +0000 (08:40 +0000)]
Merge "Move tests: unit/network_services/nfvi"

6 years agoMerge "Move tests: unit/network_services/traffic_profile"
Rodolfo Alonso Hernandez [Wed, 16 May 2018 08:40:37 +0000 (08:40 +0000)]
Merge "Move tests: unit/network_services/traffic_profile"

6 years agoMerge "Test case description and configuration file for yardstick_tc088"
Jing Lu [Wed, 16 May 2018 08:14:06 +0000 (08:14 +0000)]
Merge "Test case description and configuration file for yardstick_tc088"

6 years agoMerge "Test case description and configuration file for yardstick_tc089"
Jing Lu [Wed, 16 May 2018 08:13:48 +0000 (08:13 +0000)]
Merge "Test case description and configuration file for yardstick_tc089"

6 years agoMove tests: unit/network_services/helpers 77/56777/4
Emma Foley [Fri, 27 Apr 2018 10:33:17 +0000 (10:33 +0000)]
Move tests: unit/network_services/helpers

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

JIRA: YARDSTICK-837
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Change-Id: If0b77a6e0b102071ecfb212362647c62a621e4f9

6 years agoMove tests: unit/network_services/nfvi 75/56775/4
Emma Foley [Fri, 27 Apr 2018 10:41:10 +0000 (10:41 +0000)]
Move tests: unit/network_services/nfvi

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

JIRA: YARDSTICK-837
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Change-Id: Id330ee1c89d54ef487edb7fe160e5a31d9a3094f

6 years agoMove tests: unit/network_services/traffic_profile 73/56773/4
Emma Foley [Fri, 27 Apr 2018 10:37:37 +0000 (10:37 +0000)]
Move tests: unit/network_services/traffic_profile

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

JIRA: YARDSTICK-837
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Change-Id: Id983a2e415d66633331e7fea96a377e2a7589980

6 years agoMerge "Start nginx and uwsgi servicies only in not container mode"
Rodolfo Alonso Hernandez [Tue, 15 May 2018 17:04:34 +0000 (17:04 +0000)]
Merge "Start nginx and uwsgi servicies only in not container mode"

6 years agoStart nginx and uwsgi servicies only in not container mode 09/57409/2
Stepan Andrushko [Tue, 15 May 2018 15:42:55 +0000 (18:42 +0300)]
Start nginx and uwsgi servicies only in not container mode

No need to execute nginx and uwsgi when running in container mode.

JIRA: YARDSTICK-1179

Change-Id: I253f3adab388a14886b70e6589936f6eb27ac4c0
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
6 years agoMerge "Fix PROX throughput result calculation"
Abhijit Sinha [Tue, 15 May 2018 15:46:39 +0000 (15:46 +0000)]
Merge "Fix PROX throughput result calculation"

6 years agoFix PEP8 errors in prox_binseach 61/57361/3
Abhijit Sinha [Mon, 14 May 2018 13:38:52 +0000 (14:38 +0100)]
Fix PEP8 errors in prox_binseach

There were some pep8 errors in the
prox_binsearch.py file. These are fixed here.

JIRA: YARDSTICK-1173

Change-Id: I817bba89c6ba298bb0f27b2538aa7585cb286754
Signed-off-by: Abhijit Sinha <abhijit.sinha@intel.com>
6 years agoFix PROX throughput result calculation 59/57359/8
Abhijit Sinha [Mon, 14 May 2018 13:31:52 +0000 (14:31 +0100)]
Fix PROX throughput result calculation

The success criteria needed a fix for throughput
calculation. The results on success criteria were
muliplied by 1000*1000 which is not correct and
not needed as this was leading to false results
in grafana.

JIRA: YARDSTICK-1172

Change-Id: I56b24a700f8a565db80897aeab796e7039cae5c9
Signed-off-by: Abhijit Sinha <abhijit.sinha@intel.com>
6 years agoMerge "Replace cinder detach volume with shade client."
Rodolfo Alonso Hernandez [Tue, 15 May 2018 08:07:47 +0000 (08:07 +0000)]
Merge "Replace cinder detach volume with shade client."

6 years agoMerge "Replace cinder delete volume with shade client."
Rodolfo Alonso Hernandez [Tue, 15 May 2018 08:03:46 +0000 (08:03 +0000)]
Merge "Replace cinder delete volume with shade client."

6 years agoMerge "Replace cinder create volume with shade client."
Rodolfo Alonso Hernandez [Tue, 15 May 2018 08:03:41 +0000 (08:03 +0000)]
Merge "Replace cinder create volume with shade client."

6 years agoMerge "Document for Euphrates test case results"
Jing Lu [Tue, 15 May 2018 06:56:01 +0000 (06:56 +0000)]
Merge "Document for Euphrates test case results"