yardstick.git
7 years agoupdate requirements.txt for OpenStack Ocata 03/35803/2
Ross Brattain [Wed, 22 Mar 2017 03:29:17 +0000 (20:29 -0700)]
update requirements.txt for OpenStack Ocata

add shade for ansible

update other versions to make OpenStack Ocata upper-contraints.txt

sort file using C.UTF-8 locale as OpenStack does.
C.UTF-8 locale sorts Upper-case before lower-case.

Change-Id: Ia20eaa2b024be5cc8595873b8cc5dcba5f25d10b
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
7 years agoMerge "utils: make module import safer"
Ross Brattain [Mon, 5 Jun 2017 08:31:46 +0000 (08:31 +0000)]
Merge "utils: make module import safer"

7 years agoMerge "Added arm condition in install.sh to avoid installation error on ubuntu."
Rex Lee [Mon, 5 Jun 2017 08:31:01 +0000 (08:31 +0000)]
Merge "Added arm condition in install.sh to avoid installation error on ubuntu."

7 years agoMerge "Pass parameters between scenarios"
Rex Lee [Mon, 5 Jun 2017 01:20:33 +0000 (01:20 +0000)]
Merge "Pass parameters between scenarios"

7 years agoutils: make module import safer 55/35755/1
Ross Brattain [Mon, 22 May 2017 04:08:49 +0000 (21:08 -0700)]
utils: make module import safer

If a module fails to import, then ignore it
and try the next module.

This can help if certain module depenencies
aren't installed, e.g. TREX.

Change-Id: I2bc4384429fabd61cd430817489cb2f3c86fe9b2
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
7 years agoMerge "Bugfix: AttributeError when run tc055"
Ross Brattain [Thu, 1 Jun 2017 15:25:07 +0000 (15:25 +0000)]
Merge "Bugfix: AttributeError when run tc055"

7 years agoAdd a new runner to do binary search for max PPS 73/33073/6
Jing Zhang [Thu, 6 Apr 2017 13:38:20 +0000 (09:38 -0400)]
Add a new runner to do binary search for max PPS

A run consists of multiple (configurable) iterations. The first iteration starts from a configured packet rate. The subsequent iterations start from the observed rate from the previous run.
An iteration is a binary search for maximum pps while not exceeding 10-6 packet loss rate. The upper rate is capped to the last pps when packet loss target is missed, the bottom rate is capped to the last pps when packet loss target is met. An iteration stops when the upper rate and the lower rate are close enough (configurable) or the received rate is well below the sending rate.
The output observed rate is set to the bottom rate.

Update-1: local run of run_tests.sh is good, but two lines are reported by Jekins as too long
Update-2: Minor fix to cope with "pps" is not defined in test case yaml file.
Update-3: Add pragma to skip unit test for this patch.

JIRA: YARDSTICK-613

Change-Id: I2411b173d18d928cc1cf08f883b08bc13a125ea2
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
7 years agoImporve monitor_process pass criteria 29/35329/15
JingLu5 [Thu, 25 May 2017 08:48:15 +0000 (08:48 +0000)]
Imporve monitor_process pass criteria

JIRA: YARDSTICK-660

The monitor func()'s criteria in the monitor_process.py now is whether at least
one process of the specific controller node service is recovered. But in reality
is more resonable to use whether processes have been recoverd to it's original
amount. This patch is aiming at improving the isssue

Change-Id: I950ce2a89555801b96092735b0d670e892049927
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoMerge "This is the vote for yardstick new PTL"
Kubi [Sat, 27 May 2017 06:22:25 +0000 (06:22 +0000)]
Merge "This is the vote for yardstick new PTL"

7 years agoBugfix: AttributeError when run tc055 89/35389/2
chenjiankun [Fri, 26 May 2017 06:52:07 +0000 (06:52 +0000)]
Bugfix: AttributeError when run tc055

JIRA: YARDSTICK-662

When I run tc055, I got an error, see log:
Traceback (most recent call last):
      File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in
      _bootstrap
          self.run()
        File "/usr/lib/python2.7/multiprocessing/process.py", line 114,
        in run
          self._target(*self._args, **self._kwargs)
        File
      "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/iteration.py",
      line 46, in _worker_process
          initial_rate = options_cfg.get("rate", 100)
      AttributeError: 'NoneType' object has no attribute 'get'

This is because in the former patch, we get 'options' by scenario_cfg['options'], it is
unsafe since some test case do not have 'options' field. For tc055, 'options' is None.

Change-Id: I18a4a7954c18c609f422da403fe65c4739c93648
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
7 years agoPass parameters between scenarios 33/35333/3
chenjiankun [Thu, 25 May 2017 09:19:04 +0000 (09:19 +0000)]
Pass parameters between scenarios

JIRA: YARDSTICK-641

Allowing parameters to pass between scenarios so that the one test case
can be combination of several scenarios.

Change-Id: I55a00855e77d5b719a27a069a3ea195d6bbd0ef8
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
7 years agoAdded arm condition in install.sh to avoid installation error on ubuntu. 87/35287/2
Maciej Skrocki [Tue, 16 May 2017 20:58:41 +0000 (13:58 -0700)]
Added arm condition in install.sh to avoid installation error on ubuntu.

Change-Id: Iaa07a8d68a7dede6f2b42440be44ed07f035b1ca
Signed-off-by: Maciej Skrocki <maciej.skrocki@intel.com>
7 years agoTerminate openstack service process using kill command in HA test cases 03/35303/2
JingLu5 [Thu, 25 May 2017 03:58:52 +0000 (03:58 +0000)]
Terminate openstack service process using kill command in HA test cases

JIRA: YARDSTICK-659

In some openstack environment, the service process cannot be killed by killall
command but can be terminate by kill command.
This patch is about to switch to use kill command in the
fault_process_kill.bash to kill processes.

Change-Id: Iec455ee56d3f31fb5c16de5994870d1acd33f41a
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agofix bug: multi monitor 51/35251/6
HuanLi [Wed, 24 May 2017 03:58:00 +0000 (11:58 +0800)]
fix bug: multi monitor

JIRA: YARDSTICK-657

Change-Id: I77223cc43d529828cf3f763529019590c35b2fcb
Signed-off-by: HuanLi <lihuansse@tongji.edu.cn>
7 years agoThis is the vote for yardstick new PTL 41/35141/1
kubi [Mon, 22 May 2017 12:43:41 +0000 (20:43 +0800)]
This is the vote for yardstick new PTL

Ross Brattain who is yardstick commtter has been nominated as
new PTL candidate before the nomination end date. As we only
have one candidate, we will use gerrit to vote instead of civs

all yardstick committers, please vote +2 or -2 before this friday
the vote will be closed at that time

Reference: https://wiki.opnfv.org/display/yardstick/Yardstick+PTL+Election

Change-Id: I561f70051ed2593581ab909f08bd8be88c5b0e77
Signed-off-by: kubi <jean.gaoliang@huawei.com>
7 years agoMerge "Added PyCharms files to gitignore."
Rex Lee [Mon, 22 May 2017 00:57:47 +0000 (00:57 +0000)]
Merge "Added PyCharms files to gitignore."

7 years agoMerge "increase monitor number in cases."
Rex Lee [Sat, 20 May 2017 02:01:11 +0000 (02:01 +0000)]
Merge "increase monitor number in cases."

7 years agoincrease monitor number in cases. 37/34837/4
HuanLi [Tue, 16 May 2017 10:23:29 +0000 (18:23 +0800)]
increase monitor number in cases.

change cases to multi-monitor mode and improve monitor accuracy.

JIRA: YARDSTICK-397

Change-Id: I6840423dd83899b769df76fbee808f9bf5dc4cfd
Signed-off-by: HuanLi <lihuansse@tongji.edu.cn>
7 years agoAdded PyCharms files to gitignore. 59/34859/2
Maciej Skrocki [Tue, 16 May 2017 20:44:18 +0000 (13:44 -0700)]
Added PyCharms files to gitignore.

Change-Id: I09e9aedd80262cf75b0c0549552fc90517a24bf0
Signed-off-by: Maciej Skrocki <maciej.skrocki@intel.com>
7 years agoBumped ansible version to 2.2.2. 57/34857/2
Maciej Skrocki [Tue, 16 May 2017 20:40:27 +0000 (13:40 -0700)]
Bumped ansible version to 2.2.2.

Change-Id: Ibdf633bd475cc295f53e8236b1938a3354cb4f99
Signed-off-by: Maciej Skrocki <maciej.skrocki@intel.com>
7 years agofix multi-monitor unitest hang 65/34865/1
Ross Brattain [Tue, 16 May 2017 22:39:42 +0000 (15:39 -0700)]
fix multi-monitor unitest hang

need to mock SSH.from_node().execute.return_value
because we switch most SSH calls to use from_node()
factory function

Change-Id: Ibea525c2fcc243d41de8ccbcd40bc98532d69006
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
7 years agoBugfix: change monitor command in tc019 17/34817/1
JingLu5 [Tue, 16 May 2017 05:21:13 +0000 (05:21 +0000)]
Bugfix: change monitor command in tc019

JIRA: YARDSTICK-655

The original openstack command used for monitoring high availability openstack
nova-api service in tc019 is "nova image-list", the "openstack image list" command
used in tc019 now may not correctly show the nova-api service status.
This is patch swtich the monitor command to "openstack server list".

Change-Id: I8810f28b4328ceb4d911a3041a24774ae9cae8f0
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoMerge "Add a new monitor type: MultiMonitor that can run any number of other monitors...
Ross Brattain [Tue, 16 May 2017 04:36:31 +0000 (04:36 +0000)]
Merge "Add a new monitor type: MultiMonitor that can run any number of other monitors at the same time."

7 years agoCreate flavor from heat context 01/33001/32
DanielMartinBuckley [Wed, 5 Apr 2017 10:20:46 +0000 (11:20 +0100)]
Create flavor from heat context

JIRA: YARDSTICK-582

Create a customizable flavor via heat context. All heat parameters
are configurable including Core Affinity. The default flavor name
is XXXX-flavor where XXXX is stackname. Flavor attributes are taken
from the heat context file. If a flavor attribute is not used it
takes default attribute value. If flavor name is not specified it
uses the server name + "-flavor" or stack-name + "-flavor".

Compute node specific attributes are configurable via "extra_specs"
attribute. See
https://docs.openstack.org/admin-guide/compute-flavors.html for
details.

Change-Id: If4015970b889b0b95bfa8eba9491ebf31e92f2c7
Signed-off-by: DanielMartinBuckley <daniel.m.buckley@intel.com>
7 years agoMerge "Extend TC008 to run pktgen-dpdk inside VM Need a fast path inside VM to verify...
Rex Lee [Thu, 11 May 2017 09:16:13 +0000 (09:16 +0000)]
Merge "Extend TC008 to run pktgen-dpdk inside VM Need a fast path inside VM to verify full throughput of SRIOV and OVS-dpdk."

7 years agoMerge "KVMFORNFV: Suppress tracing with breaktrace option "
Jing Lu [Thu, 11 May 2017 06:43:47 +0000 (06:43 +0000)]
Merge "KVMFORNFV: Suppress tracing with breaktrace option "

7 years agoMerge "pass user_data to heat template"
Ross Brattain [Wed, 10 May 2017 15:08:26 +0000 (15:08 +0000)]
Merge "pass user_data to heat template"

7 years agoKVMFORNFV: Suppress tracing with breaktrace option 57/34457/3
kalyanreddy [Mon, 8 May 2017 18:51:48 +0000 (11:51 -0700)]
KVMFORNFV: Suppress tracing with breaktrace option

This patch is used to suppress tracing while executing cyclictest with
breaktrace option as breaktrace parameter will enable the ftrace by
default.

Change-Id: I776953d8cd56722b7c0f697b12b09051021334fb
Signed-off-by: Gundarapu Kalyan Reddy <reddyx.gundarapu@intel.com>
7 years agoBugfix: Support HA test cases in TripleO 91/34391/2
JingLu5 [Sat, 6 May 2017 01:10:27 +0000 (01:10 +0000)]
Bugfix: Support HA test cases in TripleO

Change-Id: Ib1f6f45677e66ca88fb546ea0662f52588e9d336
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoMerge "Update release note for Danube.2.0"
Jing Lu [Thu, 4 May 2017 12:57:21 +0000 (12:57 +0000)]
Merge "Update release note for Danube.2.0"

7 years agoMerge "Bugfix: Local Openstack Operation in HA test frameworks"
Jing Lu [Thu, 4 May 2017 12:18:01 +0000 (12:18 +0000)]
Merge "Bugfix: Local Openstack Operation in HA test frameworks"

7 years agoUpdate release note for Danube.2.0 27/34227/2
JingLu5 [Thu, 4 May 2017 11:40:45 +0000 (11:40 +0000)]
Update release note for Danube.2.0

Change-Id: I573c156e86ce92eb82179a81d8106752e6421183
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoBugfix: Local Openstack Operation in HA test frameworks 15/34215/5
tjuyinkanglin [Thu, 4 May 2017 07:58:55 +0000 (15:58 +0800)]
Bugfix: Local Openstack Operation in HA test frameworks

JIRA: YARDSTICK-635

Change-Id: Ic27517714db9325e7a3b1ef623c49af61c36b2b5
Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
7 years agoMerge "Bugfix: fix HA test case tc046"
Jing Lu [Thu, 4 May 2017 09:36:54 +0000 (09:36 +0000)]
Merge "Bugfix: fix HA test case tc046"

7 years agoMerge "Modify HA test cases and HA test script to support other installers"
Jing Lu [Thu, 4 May 2017 09:31:25 +0000 (09:31 +0000)]
Merge "Modify HA test cases and HA test script to support other installers"

7 years agoModify HA test cases and HA test script to support other installers 41/34141/2
tjuyinkanglin [Wed, 3 May 2017 13:53:09 +0000 (21:53 +0800)]
Modify HA test cases and HA test script to support other installers

JIRA: YARDSTICK-633

Change-Id: I65fd1ba11504dc61485f83c3bcc93bec4d41883b
Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
7 years agoBugfix: fix HA test case tc046 07/34207/1
JingLu5 [Thu, 4 May 2017 06:46:04 +0000 (06:46 +0000)]
Bugfix: fix HA test case tc046

The original way to kill keystone service didn't work anymore.
this patch uses killall -u to kill keystone processes.

Change-Id: I553b716f17a5ab7e57630468517642a92f06dd27
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoMerge "Bugfix: KeyError when using http dispatcher"
Jing Lu [Thu, 4 May 2017 01:24:07 +0000 (01:24 +0000)]
Merge "Bugfix: KeyError when using http dispatcher"

7 years agoMerge "run ha test case in compass pod"
Jing Lu [Wed, 3 May 2017 15:27:41 +0000 (15:27 +0000)]
Merge "run ha test case in compass pod"

7 years agoMerge "fix ha issue when run tc050~tc054 in ci"
Jing Lu [Wed, 3 May 2017 15:27:14 +0000 (15:27 +0000)]
Merge "fix ha issue when run tc050~tc054 in ci"

7 years agofix ha issue when run tc050~tc054 in ci 11/34111/2
rexlee8776 [Wed, 3 May 2017 07:58:12 +0000 (07:58 +0000)]
fix ha issue when run tc050~tc054 in ci

JIRA: YARDSTICK-634

Change-Id: I46681c7e8afe391eef9c5309470028167e911950
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
7 years agoMerge "Point to user guide to get started on VNF testing"
Ross Brattain [Wed, 3 May 2017 14:33:01 +0000 (14:33 +0000)]
Merge "Point to user guide to get started on VNF testing"

7 years agorun ha test case in compass pod 47/34147/1
rexlee8776 [Wed, 3 May 2017 14:26:32 +0000 (14:26 +0000)]
run ha test case in compass pod

JIRA: YARDSTICK-634

Change-Id: I586d5177598efeda8ac3937d1eb04cd5b97b6ec6
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
7 years agoMerge "cachestat: use raw strings to escape \d"
Rex Lee [Wed, 3 May 2017 03:51:36 +0000 (03:51 +0000)]
Merge "cachestat: use raw strings to escape \d"

7 years agoBugfix: KeyError when using http dispatcher 91/34091/2
chenjiankun [Wed, 3 May 2017 01:05:01 +0000 (01:05 +0000)]
Bugfix: KeyError when using http dispatcher

JIRA: YARDSTICK-632

When we use http dispatcher to output yardstick result.
It can upload data, but when we query the data, it get a KeyError.

Change-Id: I5410c207c68cff2621ff8184ae17daa4c286cea5
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
7 years agoExtend TC008 to run pktgen-dpdk inside VM 45/33145/8
Jing Zhang [Sat, 8 Apr 2017 01:35:00 +0000 (21:35 -0400)]
Extend TC008 to run pktgen-dpdk inside VM
Need a fast path inside VM to verify full throughput of SRIOV and OVS-dpdk.

Update 1: Change newly added file names to avoid conflict
Update 2: Add more unit test cases
Update 3: Fix default parameter typo for testpmd
Update 4: Adapted to the pktgen-dpdk prompt change from "Pktgen>" to "Pktgen:/>", now just expect "Pktgen"
Update 5: Per comment, merge common functions between latency and throughput tests to utils.py
Update 6: Per comment, seperate the test case from TC008 to a new test case TC077

Change-Id: I1f7471d4ba77636a3a66c79c2652578321312185
JIRA: YARDSTICK-614
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
7 years agoMerge "bugfix: fix HA testcases"
Jing Lu [Thu, 27 Apr 2017 13:47:27 +0000 (13:47 +0000)]
Merge "bugfix: fix HA testcases"

7 years agobugfix: fix HA testcases 51/33951/1
JingLu5 [Thu, 27 Apr 2017 12:46:40 +0000 (12:46 +0000)]
bugfix: fix HA testcases

Change-Id: Ib38e94610a108ff7195cefdfaf048e0f4fd894e8
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoMerge "Bugfix: fix HA testcases"
Rex Lee [Thu, 27 Apr 2017 08:32:02 +0000 (08:32 +0000)]
Merge "Bugfix: fix HA testcases"

7 years agoBugfix: fix HA testcases 23/33923/1
JingLu5 [Thu, 27 Apr 2017 07:49:02 +0000 (07:49 +0000)]
Bugfix: fix HA testcases

Change-Id: Ic930c59fcf3d7e53d385016051596b6563dca0d7
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoMerge "Bugfix: yardstick env prepare cmd do not support other installer"
Jing Lu [Thu, 27 Apr 2017 06:04:18 +0000 (06:04 +0000)]
Merge "Bugfix: yardstick env prepare cmd do not support other installer"

7 years agoBugfix: yardstick env prepare cmd do not support other installer 11/33911/1
chenjiankun [Thu, 27 Apr 2017 02:27:39 +0000 (02:27 +0000)]
Bugfix: yardstick env prepare cmd do not support other installer

JIRA: YARDSTICK-629

Currently yardstick env prepare do not support other installer.
I will add follow support:
1. Environment variable already exists: then do not fetch openrc file
and do not add EXTERNAL_NETWORK variable.
2. Already have openrc file in /etc/yardstick/openstack.creds: only
source this file in API.
3.Environment variable not exists and openrc file not in
/etc/yardstick/openstack.creds: fetch openrc file and append
EXTERNAL_NETWORK variable.

Change-Id: I4ce98f2e17ef8e0a0a1c33c3862ca301c53bb6c2
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
7 years agoPoint to user guide to get started on VNF testing 29/32929/2
Deepak S [Tue, 4 Apr 2017 07:14:58 +0000 (12:44 +0530)]
Point to user guide to get started on VNF testing

Change-Id: Id050d06deff665e6eaa97876d50e1698697e61e5
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
7 years agoBugfix: fix multi-region support 63/33863/1
Ross Brattain [Wed, 26 Apr 2017 13:13:31 +0000 (15:13 +0200)]
Bugfix: fix multi-region support

When running with multiple regions we need to specify
the region when we select the endpoint.

the region is specified in the OS_REGION_NAME env variable

Change-Id: I37853dd42f6d9013d4475b5e3b2b9f97b6bdc8d0
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
7 years agoMerge "Update virtualenv installation step in userguide"
Kubi [Tue, 25 Apr 2017 12:48:28 +0000 (12:48 +0000)]
Merge "Update virtualenv installation step in userguide"

7 years agoBugfix: Make tc019 abd tc025 accept --task-args options to change pod.yaml 47/33747/1
xudan [Tue, 25 Apr 2017 07:01:23 +0000 (07:01 +0000)]
Bugfix: Make tc019 abd tc025 accept --task-args options to change pod.yaml

JIRA: YARDSTICK-628

Change-Id: I7d1b84232a7477a2854ef1625cc63521999eff65
Signed-off-by: xudan <xudan16@huawei.com>
7 years agoMerge "operation_general: fix logging to use %s"
Rex Lee [Mon, 24 Apr 2017 02:02:03 +0000 (02:02 +0000)]
Merge "operation_general: fix logging to use %s"

7 years agoMerge "standardize ssh auth"
Rex Lee [Mon, 24 Apr 2017 01:58:31 +0000 (01:58 +0000)]
Merge "standardize ssh auth"

7 years agoMerge "Add a new runner to test end-to-end fast data path"
Rex Lee [Mon, 24 Apr 2017 01:57:19 +0000 (01:57 +0000)]
Merge "Add a new runner to test end-to-end fast data path"

7 years agoMerge "Add SRIOV support"
Kubi [Sun, 23 Apr 2017 10:41:27 +0000 (10:41 +0000)]
Merge "Add SRIOV support"

7 years agoUpdate virtualenv installation step in userguide 07/33607/6
JingLu5 [Sat, 22 Apr 2017 02:28:44 +0000 (02:28 +0000)]
Update virtualenv installation step in userguide

Change-Id: I0b3e8f22b6878c3b2cf1a54f3c39f2888406c508
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoYardstick virtualenv support 87/33187/3
chenjiankun [Mon, 10 Apr 2017 11:01:57 +0000 (11:01 +0000)]
Yardstick virtualenv support

JIRA: YARDSTICK-620

In the patch, I separate python-setuptools from install.sh
and separate appdirs from requirements.txt because there will be
error when install them in virtualenv.
Also I will update the documentation when this patch merged;

Change-Id: I99a532d1851983775869e7c5e8e65fd0229e86df
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
7 years agoAdd SRIOV support 89/32789/11
Jing Zhang [Fri, 31 Mar 2017 13:20:46 +0000 (09:20 -0400)]
Add SRIOV support

A generic provider network solution is introduced. To identify whether a network specified in the test case is a provider network new attributes are introduced in the test case network section:

networks:
  test-net:
    cidr: '192.168.1.0/24'
    provider: "sriov"
    physical_network: 'physnet1'

If the "provider" attribute is present, the network is an existing provider network. If the value is "sriov", binding:vnic_type=direct is added to the interface in the heat deployment template. In orchestrator/heat.py, the interface creating functions are given a new parameter that tells if the network in use is a provider network. The benchmark/contexts/model.py is changed to store the value of the provider attribute from the test case and function calls to port creation is updated with the provider parameter. The same change is made in contexts/heat.py as well. Also calls for creating a new tenant network is replaced for creating a new provider network if the provider attribute is present.

Update-1: Change test_model.py
Update-2: Per comment, change comment style to """"
Update-3: Change test_heat.py
Update-4: Add unit test cases to pass coverage test
Update-5: Add SRIOV provider network example in opnfv_yardstick_tc008.yaml
Update-6: Per comment, remove empty line in orchestrator/test_heat.py
Update-7: Per comment, change comment lines in orchestrator/test_heat.py
Update-8: Add more unit test cases to pass coverage test
Update-9: Change to create SRIOV provider network on the fly so as to support co-current test runs
Update-10: Per comment, init physical_network to 'physnet1'

Change-Id: I76004c4fcc9bffcfd8ed021fd647e0cecb346ef4
JIRA: YARDSTICK-612
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
7 years agoKVMFORNFV:Passing breaktrace option to cyclictest. 07/33407/2
kalyanreddy [Mon, 17 Apr 2017 05:16:23 +0000 (10:46 +0530)]
KVMFORNFV:Passing breaktrace option to cyclictest.

This patch is used to add breaktrace option to monitor the latency
values while executing cyclictest test cases as a part of kvmfornfv
verify and daily jobs.

Change-Id: I035ea72a8946a9d3db37a6ee33ffbf3ca67834a2
Signed-off-by: Gundarapu Kalyan Reddy <reddyx.gundarapu@intel.com>
7 years agoMerge "Bugfix: fix typo"
Rex Lee [Thu, 13 Apr 2017 04:03:52 +0000 (04:03 +0000)]
Merge "Bugfix: fix typo"

7 years agoBugfix: fix typo 61/33261/3
JingLu5 [Mon, 27 Mar 2017 02:15:00 +0000 (02:15 +0000)]
Bugfix: fix typo

Change-Id: I9b7cd5f94b1663af3b807ce24c941868b10d85d1
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoAdd a new monitor type: MultiMonitor that can run any number of other monitors at... 35/33135/3
LiHuan [Fri, 7 Apr 2017 15:32:07 +0000 (23:32 +0800)]
Add a new monitor type: MultiMonitor that can run any number of other monitors at the same time.

JIRA: YARDSTICK-397

Change-Id: Ic5cb79f0820029e306373abead1ea43fac9abee2
Signed-off-by: HuanLi <lihuansse@tongji.edu.cn>
7 years agostandardize ssh auth 31/32931/7
Ross Brattain [Thu, 2 Mar 2017 01:28:46 +0000 (17:28 -0800)]
standardize ssh auth

we need to be following defautl paramiko rules,
first use pkey, then key_filenames (autodetecting ~/.ssh/ keys),
then password

We have too much boilerplate redudant code everywhere, we need
to standardize on a factory function that takes a node dict.

Using Python3 ChainMap we can layer overrides and defaults.

VNF descriptors have to default key_filename, password to Python None.
The only way to do this is to omit key values if the variable is not
defined, this way the dict will not have the value and it will
default to Python None

Add python2 chainmap backport

Updated unittest mocking to use ssh.SSH.from_node

Change-Id: I80b0cb606e593b33e317c9e5e8ed0b74da591514
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
7 years agoAdd a new runner to test end-to-end fast data path 71/33271/1
Jing Zhang [Tue, 11 Apr 2017 19:24:22 +0000 (15:24 -0400)]
Add a new runner to test end-to-end fast data path

This runner caters testing fast host data path (e.g. SRIOV) plus fast VM data path (VM running pktgen-dpdk).
It is implemented as a new flavor of the existing Iteration runner.
(1) Add a new SLA action rate-control
(2) While the SLA target is not met, reduce packet rate until packet loss target is met.

Change-Id: I633f391f1790f03ff31458458d79d3f272475ab5
JIRA: YARDSTICK-615
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
7 years agoKVMFORNFV: Grafana config for Packet Forwarding test cases. 83/33183/3
kalyanreddy [Mon, 10 Apr 2017 10:05:41 +0000 (15:35 +0530)]
KVMFORNFV: Grafana config for Packet Forwarding test cases.

This patch is used for adding configuration file to opnfv grafana
dashboard for kvmfornfv packet forwarding test cases as a part of
daily job.

Change-Id: If2f06d2c01a7f6863ae1aa25942c3197856d895b
Signed-off-by: Gundarapu Kalyan Reddy <reddyx.gundarapu@intel.com>
7 years agoMerge "cyclictest: use raw strings to escape \[8"
Jing Lu [Mon, 10 Apr 2017 00:53:12 +0000 (00:53 +0000)]
Merge "cyclictest: use raw strings to escape \[8"

7 years agoMerge "attacker_general: fix logging to use %s"
Jing Lu [Thu, 6 Apr 2017 07:23:17 +0000 (07:23 +0000)]
Merge "attacker_general: fix logging to use %s"

7 years agocyclictest: use raw strings to escape \[8 81/32981/2
Ross Brattain [Tue, 4 Apr 2017 06:46:51 +0000 (23:46 -0700)]
cyclictest: use raw strings to escape \[8

Change-Id: I36d93eacab2470f90af5653104ad5c07853411bf
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
7 years agopip install yardstick failed in Apex 35/33035/1
JingLu5 [Thu, 6 Apr 2017 05:02:12 +0000 (05:02 +0000)]
pip install yardstick failed in Apex

JIRA: YARDSTICK-625

In Apex, yardstick uninstall and install via pip failed.
This bug blocks yardstick CI jobs.
This patch enables pip install yardstick in editable mode.
So the install_yardstick() in yardstick_verify is unnecessary.

Change-Id: If290dac5fee5584601064ae9753ae2340643d3ca
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoBugfix: fix ubuntu image name typo 27/33027/1
JingLu5 [Thu, 6 Apr 2017 01:56:48 +0000 (01:56 +0000)]
Bugfix: fix ubuntu image name typo

Change-Id: I8722f2ddd096bc784f45d4bcf58ea6a84d319fb2
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoBugfix: fix query job status in TC074 53/32853/5
JingLu5 [Wed, 5 Apr 2017 06:11:16 +0000 (06:11 +0000)]
Bugfix: fix query job status in TC074

JIRA: YARDSTICK-621
The storperf job status query in TC074 didn' work properly. The cause is the
"type: status" is not passed in the URL. This patch also update storperf VM
image to xenial.

Change-Id: Idd9f501416b24612f6045a57ba2a95e2ed3a9572
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoattacker_general: fix logging to use %s 87/32987/1
Ross Brattain [Tue, 4 Apr 2017 06:43:56 +0000 (23:43 -0700)]
attacker_general: fix logging to use %s

Change-Id: Ib451d7883eb5df13cfe95477cea43c076ac0452a
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
7 years agooperation_general: fix logging to use %s 85/32985/1
Ross Brattain [Tue, 4 Apr 2017 06:45:57 +0000 (23:45 -0700)]
operation_general: fix logging to use %s

Change-Id: I1b37cbf07e8858ca3e75bb74c57fe84485ff4989
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
7 years agocachestat: use raw strings to escape \d 83/32983/1
Ross Brattain [Tue, 4 Apr 2017 06:46:21 +0000 (23:46 -0700)]
cachestat: use raw strings to escape \d

Change-Id: Iff40f7c938af0d7c148c708eed19095b140ccb8b
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
7 years agoMerge "Add opnfv_os-ovn-nofeature-noha_daily test suite"
Jing Lu [Wed, 5 Apr 2017 04:22:38 +0000 (04:22 +0000)]
Merge "Add opnfv_os-ovn-nofeature-noha_daily test suite"

7 years agoMerge "Yardstick virtualenv support"
Jing Lu [Wed, 5 Apr 2017 04:22:26 +0000 (04:22 +0000)]
Merge "Yardstick virtualenv support"

7 years agoAdd opnfv_os-ovn-nofeature-noha_daily test suite 73/32973/1
JingLu5 [Wed, 5 Apr 2017 01:43:50 +0000 (01:43 +0000)]
Add opnfv_os-ovn-nofeature-noha_daily test suite

JIRA: YARDSTICK-624

Change-Id: Ia783bec53e4d889736db7e510d28c32377d4f9b2
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoMerge "yardstick offline support"
Jing Lu [Sat, 1 Apr 2017 09:36:06 +0000 (09:36 +0000)]
Merge "yardstick offline support"

7 years agoBugfix: take test case modification into effect in load_images.sh 55/32855/1
JingLu5 [Sat, 1 Apr 2017 08:03:24 +0000 (08:03 +0000)]
Bugfix: take test case modification into effect in load_images.sh

JIRA: YARDSTICK-622

Bug log: https://build.opnfv.org/ci/view/armband/job/yardstick-fuel-armband-baremetal-daily-danube/24/console

This error is because the file changes done in the load_images.sh is not updated.
After file modification, pip install -U . is required to tack changes into effect.

Change-Id: I8be3ffc963e3d5cec861b68a33efc2fa5bdabbd2
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoYardstick virtualenv support 37/32837/2
chenjiankun [Sat, 1 Apr 2017 01:19:16 +0000 (01:19 +0000)]
Yardstick virtualenv support

JIRA: YARDSTICK-620

Currently we recommend using docker to run yardstick.
And it is hard to use virtualenv to install yardstick.
So I modify install.sh in yardstick root path.
It will support using virtualenv to install yardstick(including API) in
linux.
In this patch, I make yardstick support read yardstick configuration
have priority over constants.

Change-Id: I9ea1241b228532a6497451e6c8f232173ddb783e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
7 years agoRefine documentation 45/32745/3
JingLu5 [Fri, 31 Mar 2017 09:17:26 +0000 (09:17 +0000)]
Refine documentation

Change-Id: Ic9a84954fdb18ac97afde01541b4c84157df8441
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoMerge "Store packetsize in influxdb"
Rex Lee [Fri, 31 Mar 2017 06:52:23 +0000 (06:52 +0000)]
Merge "Store packetsize in influxdb"

7 years agoStore packetsize in influxdb 19/32519/3
Jing Zhang [Thu, 30 Mar 2017 13:26:55 +0000 (09:26 -0400)]
Store packetsize in influxdb

This allows calculating throughput in Gbps.

Add: test_pktgen.py

Change-Id: I7e2132ec71985c14570ef18b952b2264ebf7abbc
JIRA: YARKSTICK-611
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
7 years agoMerge "add yardstick_user_interface chapter in userguide"
Jing Lu [Fri, 31 Mar 2017 01:40:59 +0000 (01:40 +0000)]
Merge "add yardstick_user_interface chapter in userguide"

7 years agoMerge "Bugfix: cacert should suppot insecure mode if needed"
Jing Lu [Fri, 31 Mar 2017 01:40:34 +0000 (01:40 +0000)]
Merge "Bugfix: cacert should suppot insecure mode if needed"

7 years agoadd yardstick_user_interface chapter in userguide 77/32477/1
JingLu5 [Thu, 30 Mar 2017 11:50:02 +0000 (11:50 +0000)]
add yardstick_user_interface chapter in userguide

JIRA: YARDSTICK-618

Change-Id: I690c24d665016a381ae1ed7d8fa94d5a34bc1b1b
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoMerge "Bugfix: Add test suite for opnfv_os-odl_l3-ovs-ha_daily scenario"
Jing Lu [Thu, 30 Mar 2017 11:36:18 +0000 (11:36 +0000)]
Merge "Bugfix: Add test suite for opnfv_os-odl_l3-ovs-ha_daily scenario"

7 years agoMerge "Refine documentation"
Jing Lu [Thu, 30 Mar 2017 09:40:07 +0000 (09:40 +0000)]
Merge "Refine documentation"

7 years agoRefine documentation 59/31959/28
JingLu5 [Tue, 28 Mar 2017 01:43:21 +0000 (01:43 +0000)]
Refine documentation

Change-Id: I0d3387a520e3decf51bb0f7db53996f148a611d0
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoBugfix: Add test suite for opnfv_os-odl_l3-ovs-ha_daily scenario 81/32381/2
JingLu5 [Thu, 30 Mar 2017 06:36:29 +0000 (06:36 +0000)]
Bugfix: Add test suite for opnfv_os-odl_l3-ovs-ha_daily scenario

JIRA: YARDSTICK-617

Apex has a new scenario 'opnfv_os-odl_l3-ovs-ha_daily' in CI.
This patch add yardstick test suite for this scenario.

Change-Id: I50eb84fb506e7cf87cbd6bcb67f3df6d185d8b6b
Signed-off-by: JingLu5 <lvjing5@huawei.com>
7 years agoBugfix: cacert should suppot insecure mode if needed 59/32359/1
rexlee8776 [Thu, 30 Mar 2017 02:37:02 +0000 (10:37 +0800)]
Bugfix: cacert should suppot insecure mode if needed

currently, we use env variable "OS_CACERT" to specify the location of
cert file. but if there's no cacert and use for personal verification,
it's better to also support insecure mode by set OS_CACERT=False

JIRA: YARDSTICK-616

Change-Id: I73b0860863bf6386f9114328c52594ef87e02fa3
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
7 years agoMerge "Set work directory in Yardstick container"
Jing Lu [Wed, 29 Mar 2017 03:21:58 +0000 (03:21 +0000)]
Merge "Set work directory in Yardstick container"

7 years agoBugfix: improper license header in swagger docs 59/32059/1
chenjiankun [Tue, 28 Mar 2017 13:51:42 +0000 (13:51 +0000)]
Bugfix: improper license header in swagger docs

JIRA: YARDSTICK-609

In the patch:
https://gerrit.opnfv.org/gerrit/#/c/28833/

It update the license header, but it is improper to swagger yaml file.
The license information will show in swagger docs.
So I remove it.

Change-Id: Ib87f0f43a95e3b2145c4b59059c4573000f1997c
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
7 years agoyardstick offline support 05/32005/3
chenjiankun [Tue, 28 Mar 2017 07:58:49 +0000 (07:58 +0000)]
yardstick offline support

JIRA: YARDSTICK-597

Currently yardstick can not run offline.
In some case this will block the process, causing yardstick job failed
The reason is yardstick will pull the latest code when run in CI.
Actually we need not pull the latest code because the code in yardstick
is already the latest code.
So I remove them.
Also I check the environment variable and yardstick-image when fetch
openrc file and load images.

Change-Id: I797da272932ab5292aa40585c82c119aa71b8849
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
7 years agoMerge "Bugfix: Error validating value 'cirros-0.3.5'"
Jing Lu [Tue, 28 Mar 2017 08:48:42 +0000 (08:48 +0000)]
Merge "Bugfix: Error validating value 'cirros-0.3.5'"