yardstick.git
6 years agoDRAFT: update yardstick generic test cases dashboard 63/39563/6
rexlee8776 [Fri, 18 Aug 2017 01:13:44 +0000 (01:13 +0000)]
DRAFT: update yardstick generic test cases dashboard

current dashboard is obselete and hard to understand
add local grafana dashboard automatically import by
"yardstick env grafana"
update grafana version to latest steady

Change-Id: Iffae9170231d3e0867a52a1f6b5e99c4650b7326
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoFix import error in IxNetwork 97/39497/4
Deepak S [Thu, 17 Aug 2017 04:20:55 +0000 (21:20 -0700)]
Fix import error in IxNetwork

This patch fixes import error related to ixnetwork and do not wait for
TG server to start as IXIA is always running :)

Change-Id: I49d6b100eadafad75431cb8974605a6faa496f23
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agosamplevnf: fix make_ip_addr 55/39455/2
Ross Brattain [Wed, 9 Aug 2017 00:56:42 +0000 (17:56 -0700)]
samplevnf: fix make_ip_addr

we get TypeError sometimes, so
we need to catch them

Change-Id: I5fc58006f18acf84f62d8f4a21077d71b2fb0ed8
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "utils: create TASK_LOG_DIR if it doesn't exist"
Ross Brattain [Thu, 17 Aug 2017 01:58:52 +0000 (01:58 +0000)]
Merge "utils: create TASK_LOG_DIR if it doesn't exist"

6 years agoMerge "Verify the route list before parsing the data key"
Ross Brattain [Thu, 17 Aug 2017 01:56:47 +0000 (01:56 +0000)]
Merge "Verify the route list before parsing the data key"

6 years agoMerge "run_traffic: capture and exit gracefully if crash in trex run_traffic"
Ross Brattain [Thu, 17 Aug 2017 01:56:04 +0000 (01:56 +0000)]
Merge "run_traffic: capture and exit gracefully if crash in trex run_traffic"

6 years agoMerge "YAML fixes"
Ross Brattain [Wed, 16 Aug 2017 16:19:56 +0000 (16:19 +0000)]
Merge "YAML fixes"

6 years agoVerify the route list before parsing the data key 83/39183/4
Deepak S [Sat, 12 Aug 2017 22:13:21 +0000 (15:13 -0700)]
Verify the route list before parsing the data key

Exception:
line 502, in get_route_data
return next((route[data_key] for route in route_list if route['if'] ==
port), None)
TypeError: 'NoneType' object is not iterable

Change-Id: Ia45d90a4e9a61cb917b23efd03b86eafb249b747
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
6 years agoMerge "Remove redundancy file and do relative modification"
Ross Brattain [Wed, 16 Aug 2017 03:37:59 +0000 (03:37 +0000)]
Merge "Remove redundancy file and do relative modification"

6 years agoYAML fixes 09/39209/10
Ross Brattain [Mon, 7 Aug 2017 21:32:45 +0000 (14:32 -0700)]
YAML fixes

There are multiple issues wiht YAML loading.

1. Jinja2 renders None values as a string 'None'.  This is not valid YAML
we need to render None values to '~' or 'null' which is the native YAML
None value.

2. Jinja2 renders dict and lists that contain unicode with
u'foo' values.  This is not value YAML syntax.
Because we are serializing dict and lists into YAML, we
need to encode them as valid YAML.  We can override Jinja2 finalize to
use yaml.dump to dump inline YAML.

We use yaml.safe_dump(elem, default_flow_style=True).replace('\n', '')
to generate valid single-line YAML dict and list values.

But this problem highlights the general difficulties with templating and
loading files.
We could avoid this Python->Jinja2->YAML->Python issue by directly
injecting the list or dict after the YAML is loaded.

I'm not sure of the real utility of these templates.

3. On Python 2 YAML loader is rendering all strings
as unicode.  This does not work for Trex because Trex is broken
and badly coded.  Trex does type checking against str() which
is different for Python 2 and Python 3.
The default YAML loader will return native string types, str() or unicode()
for Python 2 and Python 3 respectively.

The bad Trex codes is in convert_val:

https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_packet_builder_scapy.py#L674

def convert_val (val):
    if is_integer(val):
        return val
    if type(val) == str:
        return ipv4_str_to_num (is_valid_ipv4(val))
    raise CTRexPacketBuildException(-11,("init val invalid %s ") % val  );

This code is doing type(val) == str.  This is bad and broken.
We can't fix Trex, so we have to render all strings as native str() types

The bug here was that the Heat template loader template_format.py
was overriding the global YAML loader to always return unicode.
We don't want this global override.

To fix this we have to use local subclasses of the yaml.SafeLoader
class.
But in order to dynamically subclass from CSafeLoader or SafeLoader
we have to use the type() builtin to define a new class at runtime.

Once we have new classes defined, we can safely isolate different
YAML constructors and return unicode or not depending on the case.

To be consistent we implement a new yaml_loader.py module to centralize
all non-Heat template yaml loading to ensure correct uncode/str
conversion

Change-Id: Iebf9cf78fbda390977c390436b0869e7bbf503eb
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agorun_traffic: capture and exit gracefully if crash in trex run_traffic 39/39339/7
Deepak S [Mon, 14 Aug 2017 10:09:53 +0000 (03:09 -0700)]
run_traffic: capture and exit gracefully if crash in trex run_traffic

new non-blocking code of run_traffic has raise condition which causes
trex client to exit with STLError. This patches captures the exception
and exit gracefully

Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in
_bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114,
in run
self._target(*self._args, **self._kwargs)
File
"/opt/nsb_bin/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/network_services/vnf_generic/vnf/sample_vnf.py",
line 945, in _traffic_runner
self.resource_helper.run_traffic(traffic_profile)
File
"/opt/nsb_bin/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/network_services/vnf_generic/vnf/sample_vnf.py",
line 506, in run_traffic
self._run_traffic_once(traffic_profile)
File
"/opt/nsb_bin/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/network_services/vnf_generic/vnf/tg_rfc2544_trex.py",
line 63, in _run_traffic_once
self.client.stop(self.my_ports)
File
"/opt/nsb_bin/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py",
line 1202, in wrap2
raise STLError("'{0}' - connection to the server had been lost: '{1}'".format(func_name, client.conn.get_disconnection_cause()))
STLError:

Change-Id: Ie5741339451e0a3f9c4bb48f64fd35d86d18e5d4
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agohtml_template: can't use iteritems, use items 69/39369/1
Ross Brattain [Wed, 16 Aug 2017 00:50:32 +0000 (17:50 -0700)]
html_template: can't use iteritems, use items

iteritems is not supported in Python 3
we have to use items()

Change-Id: Ib923dfea4220eeab311949c9f6f420fef378e98a
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoEnable ixnet traffic generator to run traffic 43/39343/4
Deepak S [Mon, 14 Aug 2017 22:19:54 +0000 (15:19 -0700)]
Enable ixnet traffic generator to run traffic

Change-Id: I8d7cea75aa864c37a058a154a38c9828ea5da434
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMoving trex to latest version (v2.28) to support new NICs 45/39345/2
Deepak S [Tue, 15 Aug 2017 04:46:29 +0000 (21:46 -0700)]
Moving trex to latest version (v2.28) to support new NICs

Change-Id: I81e119deec36ae90749c2eb1d555f75475ce2474
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
6 years agoMerge "Download the right dpdk bind tool before starting the test case"
Ross Brattain [Tue, 15 Aug 2017 15:25:11 +0000 (15:25 +0000)]
Merge "Download the right dpdk bind tool before starting the test case"

6 years agoMerge "deepgetitem: try string index before int"
Ross Brattain [Tue, 15 Aug 2017 15:24:30 +0000 (15:24 +0000)]
Merge "deepgetitem: try string index before int"

6 years agoMerge "gitignore: ignore .tox/"
Ross Brattain [Tue, 15 Aug 2017 13:41:58 +0000 (13:41 +0000)]
Merge "gitignore: ignore .tox/"

6 years agoDownload the right dpdk bind tool before starting the test case 35/39335/2
Deepak S [Sat, 12 Aug 2017 22:01:07 +0000 (15:01 -0700)]
Download the right dpdk bind tool before starting the test case

Change-Id: Id8901aabef402d5743e2575e275b5a57046e897c
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
6 years agogitignore: ignore .tox/ 11/39311/1
Ross Brattain [Mon, 7 Aug 2017 21:31:54 +0000 (14:31 -0700)]
gitignore: ignore .tox/

Change-Id: If021ed0f099c81492fd7509317efc31f020e6d18
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agodeepgetitem: try string index before int 07/39307/2
Ross Brattain [Tue, 15 Aug 2017 03:30:39 +0000 (20:30 -0700)]
deepgetitem: try string index before int

Change-Id: I3b353c5887db0ebbe33e37db505e72b85167b54e
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "Update release note for Danube.3.2"
Ross Brattain [Tue, 15 Aug 2017 05:09:02 +0000 (05:09 +0000)]
Merge "Update release note for Danube.3.2"

6 years agoMerge "ACL: remove failing testing"
Ross Brattain [Tue, 15 Aug 2017 05:01:01 +0000 (05:01 +0000)]
Merge "ACL: remove failing testing"

6 years agoMerge "task: initialize result"
Ross Brattain [Tue, 15 Aug 2017 04:44:26 +0000 (04:44 +0000)]
Merge "task: initialize result"

6 years agotask: initialize result 05/39305/1
Ross Brattain [Tue, 15 Aug 2017 04:32:52 +0000 (21:32 -0700)]
task: initialize result

Change-Id: I8d6b9bed24c9afa4be54296515e6aa91e84fe650
Signed-off-by: Martin Banszel <martinx.banszel@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agodynamictp: fix flake8 warning 91/38491/3
Ross Brattain [Mon, 31 Jul 2017 20:21:08 +0000 (13:21 -0700)]
dynamictp: fix flake8 warning

flake8 warns about indent

Change-Id: Ib7a57dcb8462de24a2ccb1fdbc60db995cc60d33
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoACL: remove failing testing 97/39297/1
Ross Brattain [Tue, 15 Aug 2017 03:21:06 +0000 (20:21 -0700)]
ACL: remove failing testing

test_instantiate_panic fails randomly sometimes
in Jenkins, the test doesn't really do anything
useful, so remove it

Change-Id: Ia44ebbe778b812ae1b4660d64e88bf029fed3701
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "KVMFORNFV: Update Grafana config for Packet Forwarding test cases."
Rex Lee [Tue, 15 Aug 2017 01:13:40 +0000 (01:13 +0000)]
Merge "KVMFORNFV: Update Grafana config for Packet Forwarding test cases."

6 years agoIntegrate vsperf in Tgen mode 79/35379/10
Jing Zhang [Thu, 25 May 2017 22:41:00 +0000 (18:41 -0400)]
Integrate vsperf in Tgen mode

Problem:
Running Vsperf in Tgen mode is supported but the integration is not complete at the code level
i.e. not ready-to-use, and dpdk loopback is not supported inside the VM.

Solution:
(1) Completely automates VM image generation and supports 1G huge pages.
(2) Adds a new test scenario VsperfDPDK for testpmd based loopback inside the VM.

Update 1-2: Fixed "line too long" issues not reported by local run_tests.sh (why?)
Update 3: Per comment change to use SSH.from_node() and add unit test cases
Update 4: Add more unit test cases for coverage and ready the code for merge

JIRA: YARDSTICK-661

Change-Id: Iea3014d4c83e1b0c079019a4ed27771d40a7eed8
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
6 years agoUpdate release note for Danube.3.2 93/39293/1
JingLu5 [Tue, 15 Aug 2017 00:58:05 +0000 (00:58 +0000)]
Update release note for Danube.3.2

Note: This patch should be merger to stable branch.

Change-Id: I7f07a7c367fc387494f2313143a300fef85a8fd1
Signed-off-by: JingLu5 <lvjing5@huawei.com>
6 years agoMerge "Add yardstick report for each task"
Ross Brattain [Tue, 15 Aug 2017 00:56:55 +0000 (00:56 +0000)]
Merge "Add yardstick report for each task"

6 years agoAnsible typo fix, whitespace 41/39241/3
Ross Brattain [Wed, 15 Mar 2017 18:50:29 +0000 (11:50 -0700)]
Ansible typo fix, whitespace

Change-Id: I3639ab84ab5b9e802e1bbd23674b0fbda46fdc66
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoPROX VNF and TG 51/38251/9
Ross Brattain [Mon, 24 Apr 2017 21:25:48 +0000 (23:25 +0200)]
PROX VNF and TG

PROX was added to samplevnf project
https://git.opnfv.org/samplevnf/tree/VNFs/DPPD-PROX

JIRA: YARDSTICK-638

Change-Id: If9875b1130c6bed87deb8720b0d8b28ede9289d9
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "cli: typo, s/dispath/dispatch/g"
Ross Brattain [Mon, 14 Aug 2017 15:41:10 +0000 (15:41 +0000)]
Merge "cli: typo, s/dispath/dispatch/g"

6 years agoAdd yardstick report for each task 17/39217/1
chenjiankun [Mon, 14 Aug 2017 02:47:29 +0000 (02:47 +0000)]
Add yardstick report for each task

JIRA: YARDSTICK-784

Currently we have yardstick report in GUI.
But if users do not use GUI, they can't see this report.
So we need generate a report each task.

After run each test case, we can see report:
http://ip:port/report/report.html

Change-Id: Ic76cf57f55aa6680b91272e210135136f0225373
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
6 years agoutils: create TASK_LOG_DIR if it doesn't exist 13/39213/1
Ross Brattain [Sun, 13 Aug 2017 20:36:57 +0000 (13:36 -0700)]
utils: create TASK_LOG_DIR if it doesn't exist

Change-Id: I10630599065e6c159b137bd4d6ee89353a1c411b
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agocli: typo, s/dispath/dispatch/g 11/39211/1
Ross Brattain [Sun, 13 Aug 2017 19:09:53 +0000 (12:09 -0700)]
cli: typo, s/dispath/dispatch/g

Change-Id: I3df6bc606f2ac78baf611aee22b9247f6f9d5136
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMove the comment above the config parameter line to avoid the expection 95/39195/1
Deepak S [Sat, 12 Aug 2017 23:57:06 +0000 (16:57 -0700)]
Move the comment above the config parameter line to avoid the expection

Traceback (most recent call last):
File "/opt/nsb_bin/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/cmd/commands/task.py", line 54, in do_start
result = Task().start(param, **kwargs)
File "/opt/nsb_bin/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/benchmark/core/task.py", line 148, in start
self._do_output(output_config, result)
File "/opt/nsb_bin/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/benchmark/core/task.py", line 227, in _do_output
dispatchers = DispatcherBase.get(output_config)
File "/opt/nsb_bin/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/dispatcher/base.py", line 46, in get
for out_type in config['DEFAULT']['dispatcher']]
File "/opt/nsb_bin/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/dispatcher/base.py", line 38, in get_cls
raise RuntimeError("No such dispatcher_type %s" % dispatcher_type)
RuntimeError: No such dispatcher_type Http # setup multiple dipatcher with comma deperted e.g. file

Change-Id: I497d0d9ce7fdb7ea2913af6ebeb4bb9008344fed
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
6 years agoAdd Ansible executor class for node context 11/33411/13
Ross Brattain [Tue, 14 Mar 2017 05:08:06 +0000 (22:08 -0700)]
Add Ansible executor class for node context

import the AnsibleCommon class to execute Ansible playbooks

Update node context support to use AnsibleCommon

needs unittests

We must call ansible-playbook as an executable, so we must create temp
files for inventory, and for the playbooks.

AnsibleCommon has evolved to be quite flexible, it auto-generates the
inventory from the context['nodes'] and generates groups from the node
Role.

We also support either a single playbook filename, or a list of
filenames.

If given a list we dynamically generate a playbook that includes the
other playbooks.

We support adding any number of extra_vars using a temp JSON file.

Also designed to be extended by subclassing.

Change-Id: I5bd0a2b4547feaadd70b7e2b8801f19371b99df0
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoixia: VNF id must match class name 55/39155/1
Ross Brattain [Fri, 11 Aug 2017 21:38:28 +0000 (14:38 -0700)]
ixia: VNF id must match class name

The class name is IxiaTrafficGen

Change-Id: Ie18be44c7b58d50f13e9a5df50afa3e02fc0cce3
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "yardstick setup ansible, including load_images"
Ross Brattain [Fri, 11 Aug 2017 17:47:29 +0000 (17:47 +0000)]
Merge "yardstick setup ansible, including load_images"

6 years agoKVMFORNFV: Update Grafana config for Packet Forwarding test cases. 27/39127/3
kalyanreddy [Fri, 11 Aug 2017 10:45:24 +0000 (03:45 -0700)]
KVMFORNFV: Update Grafana config for Packet Forwarding test cases.

JIRA: YARDSTICK-783

This patch is used for updating the configuration file of opnfv grafana
dashboard for kvmfornfv packet forwarding to publish the results of new
test cases added as part of kvmfornfv daily job.

Also used for updating the config file of opnfv grafana dashboard for
kvmfornfv cyclictest test cases to correct the alias colours.

Change-Id: I34aa155e2782e58147e55f974134673a892eb93b
Signed-off-by: Gundarapu Kalyan Reddy <reddyx.gundarapu@intel.com>
6 years agoRemove redundancy file and do relative modification 69/38969/4
chenjiankun [Wed, 9 Aug 2017 09:51:31 +0000 (09:51 +0000)]
Remove redundancy file and do relative modification

Change-Id: If6f672a2cbe218a20e3d8f3d093d31f6887d7ca3
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
6 years agoAdd real time log view in GUI 53/38953/3
chenjiankun [Wed, 9 Aug 2017 03:23:58 +0000 (03:23 +0000)]
Add real time log view in GUI

JIRA: YARDSTICK-775

We have GUI now, but we can't see real time log in GUI view.
So I add real time log view in GUI.

Change-Id: Ie83f327ef0a94302afa6b3def764fec6ef5818d1
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
6 years agoMerge "Add common openstack opertation scenarios: image & volume"
Jing Lu [Fri, 11 Aug 2017 09:16:42 +0000 (09:16 +0000)]
Merge "Add common openstack opertation scenarios: image & volume"

6 years agoMerge "bugfix: tc038 ssh default wait 3600s"
Kubi [Fri, 11 Aug 2017 07:55:43 +0000 (07:55 +0000)]
Merge "bugfix: tc038 ssh default wait 3600s"

6 years agoMerge "Add common openstack opertation scenarios: flavor & server"
Kubi [Fri, 11 Aug 2017 07:54:37 +0000 (07:54 +0000)]
Merge "Add common openstack opertation scenarios: flavor & server"

6 years agobugfix: tc038 ssh default wait 3600s 11/39111/2
rexlee8776 [Fri, 11 Aug 2017 07:04:35 +0000 (07:04 +0000)]
bugfix: tc038 ssh default wait 3600s

JIRA: YARDSTICK-771

Change-Id: Ibcd2228505d341feb09b0d477e5f4ed6062c1e89
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoMerge "UDP relay"
Ross Brattain [Fri, 11 Aug 2017 04:26:07 +0000 (04:26 +0000)]
Merge "UDP relay"

6 years agoMerge "New storage test case: Bonnie++"
Jing Lu [Fri, 11 Aug 2017 03:47:23 +0000 (03:47 +0000)]
Merge "New storage test case: Bonnie++"

6 years agoAdd common openstack opertation scenarios: image & volume 95/39095/4
JingLu5 [Fri, 11 Aug 2017 02:58:20 +0000 (02:58 +0000)]
Add common openstack opertation scenarios: image & volume

JIRA: YARDSTICK-781

This patch adds some common openstack opertation scenarios

Change-Id: I3de7dbb30eaebac4feebcf07dd6a0d2bdcf428d9
Signed-off-by: JingLu5 <lvjing5@huawei.com>
6 years agoUDP relay 43/38243/8
Deepak S [Tue, 20 Jun 2017 21:21:24 +0000 (14:21 -0700)]
UDP relay

Change-Id: I598f6f98f94f70334139966cc170832c90ea9aa6
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "Sample CGNAPT VNF"
Ross Brattain [Fri, 11 Aug 2017 02:52:24 +0000 (02:52 +0000)]
Merge "Sample CGNAPT VNF"

6 years agoSample VFW VNF 39/38239/11
Deepak S [Tue, 20 Jun 2017 21:15:50 +0000 (14:15 -0700)]
Sample VFW VNF

Change-Id: I32ca166cd6d818b57bbcfaf7c7e5a65b7147ba8d
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoSample CGNAPT VNF 37/38237/16
Deepak S [Tue, 20 Jun 2017 21:15:18 +0000 (14:15 -0700)]
Sample CGNAPT VNF

Change-Id: I0f5555a1674a5ecb7e2afb508d495b7872c90757
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoAdd common openstack opertation scenarios: flavor & server 37/39037/19
JingLu5 [Thu, 10 Aug 2017 06:00:58 +0000 (06:00 +0000)]
Add common openstack opertation scenarios: flavor & server

JIRA: YARDSTICK-781

This patch adds some common openstack opertation scenarios

Change-Id: I9e84a8894fe9b9c1754a45a0ddfdf93739164b9a
Signed-off-by: JingLu5 <lvjing5@huawei.com>
6 years agoMerge "NSB fixes: docstring, whitespace, etc."
Ross Brattain [Thu, 10 Aug 2017 19:53:49 +0000 (19:53 +0000)]
Merge "NSB fixes: docstring, whitespace, etc."

6 years agoMerge "bugfix: tc037/38 multiprocessing failed sometimes"
Kubi [Thu, 10 Aug 2017 06:18:58 +0000 (06:18 +0000)]
Merge "bugfix: tc037/38 multiprocessing failed sometimes"

6 years agoMerge "pip_license: quick hack script to add license info to requirements.txt"
Ross Brattain [Thu, 10 Aug 2017 06:14:50 +0000 (06:14 +0000)]
Merge "pip_license: quick hack script to add license info to requirements.txt"

6 years agoMerge "requirements.txt: add license info"
Ross Brattain [Thu, 10 Aug 2017 06:14:29 +0000 (06:14 +0000)]
Merge "requirements.txt: add license info"

6 years agoNSB fixes: docstring, whitespace, etc. 35/39035/1
Ross Brattain [Thu, 10 Aug 2017 05:30:25 +0000 (22:30 -0700)]
NSB fixes: docstring, whitespace, etc.

remove unused docstring params
remove whitespace
fix sudo vnf_build command
ignore stdout

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Change-Id: I6fcdbb0ed4bc6b7fc2f557529f5ff9fd960c05e9

6 years agoMerge "Collectd"
Ross Brattain [Thu, 10 Aug 2017 03:57:47 +0000 (03:57 +0000)]
Merge "Collectd"

6 years agoMerge "constants: cache YAML config values"
Ross Brattain [Thu, 10 Aug 2017 03:56:50 +0000 (03:56 +0000)]
Merge "constants: cache YAML config values"

6 years agoMerge "IXIA traffic generator"
Ross Brattain [Thu, 10 Aug 2017 03:56:37 +0000 (03:56 +0000)]
Merge "IXIA traffic generator"

6 years agoMerge "Sample ACL VNF"
Ross Brattain [Thu, 10 Aug 2017 03:56:29 +0000 (03:56 +0000)]
Merge "Sample ACL VNF"

6 years agobugfix: tc037/38 multiprocessing failed sometimes 09/39009/5
rexlee8776 [Thu, 10 Aug 2017 01:35:58 +0000 (01:35 +0000)]
bugfix: tc037/38 multiprocessing failed sometimes

set runner.join timeout

JIRA: YARDSTICK-771

Change-Id: Iec7d04549a45c0fbd0eb48dcaad7770fd5ca1f4e
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoMerge "bugfix: tc025 and tc054 fail"
Kubi [Thu, 10 Aug 2017 01:18:55 +0000 (01:18 +0000)]
Merge "bugfix: tc025 and tc054 fail"

6 years agoMerge "bugfix: hosts should be clean if update_hosts twice"
Kubi [Thu, 10 Aug 2017 01:18:33 +0000 (01:18 +0000)]
Merge "bugfix: hosts should be clean if update_hosts twice"

6 years agorequirements.txt: add license info 77/38877/2
Ross Brattain [Tue, 8 Aug 2017 01:38:53 +0000 (18:38 -0700)]
requirements.txt: add license info

Change-Id: I9de6a51e7efb9e49da5bbd541c466f106c3a52e3
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoBugfix: remove ping package for it is GPL license 61/38961/2
chenjiankun [Wed, 9 Aug 2017 07:37:22 +0000 (07:37 +0000)]
Bugfix: remove ping package for it is GPL license

JIRA: YARDSTICK-777

In migrate test case, we use ping to count downtime.
But ping package is GPL license, so we need remove it.
Since ping command tool can not set timeout less then 1 second, and the
downtime is less then 1 second, so we need find another package to replace.
For now, I find most ping tool is GPL license, so maybe later we need to
implement ping ourselves.
And the migrate case can not calculate downtime unless user install ping
themselves.

Change-Id: I78cba5e4af9de2e3c0bb16b27570bed951caf440
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
6 years agoconstants: cache YAML config values 43/36343/9
Ross Brattain [Thu, 22 Jun 2017 02:25:22 +0000 (19:25 -0700)]
constants: cache YAML config values

don't reparse the yardstick.yaml file each
time we lookup an option.

Since it is global data, just cache it in a global
var

Use pkg_resources.resource_filename
to lookup the path of yardstick tests, intsead of using __file__

pkg_resources is slightly more proper than __file__
at least for packages

Change-Id: I05d9748390a37bd45c53013fc084d23069ab7c51
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agobugfix: tc025 and tc054 fail 83/38683/6
rexlee8776 [Thu, 3 Aug 2017 12:39:44 +0000 (12:39 +0000)]
bugfix: tc025 and tc054 fail

also modify tc025 an tc054 to be able to pass paras

JIRA: YARDSTICK-772

Change-Id: Ibeba931804cccfd74fc70fcf4fdb6af9d5c8ab77
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agobugfix: hosts should be clean if update_hosts twice 89/38889/2
rexlee8776 [Tue, 8 Aug 2017 08:09:41 +0000 (08:09 +0000)]
bugfix: hosts should be clean if update_hosts twice

JIRA: YARDSTICK-774

Change-Id: Id610d43d236ab3a7d1aa4b8cfd230a40958fac38
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
6 years agoyardstick setup ansible, including load_images 77/37977/3
Malanik Jan [Tue, 18 Jul 2017 16:54:10 +0000 (12:54 -0400)]
yardstick setup ansible, including load_images

also update to cirros 0.3.5
added PROX compilation
add create_node_pod_yaml role

JIRA: YARDSTICK-639

Change-Id: If5999841287a54c7e5c64a7cc487c6394df90424
Signed-off-by: Malanik Jan <janx.malanik@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoCollectd 45/38245/4
Deepak S [Tue, 20 Jun 2017 21:24:26 +0000 (14:24 -0700)]
Collectd

Change-Id: I15e4ac38b347a08350b71c68469e2793eeed92ab
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoSample ACL VNF 23/38223/11
Deepak S [Tue, 20 Jun 2017 21:14:06 +0000 (14:14 -0700)]
Sample ACL VNF

Change-Id: I33de47ac6ca353d6c69f0d166809b4c95d3fd90f
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoIXIA traffic generator 91/38291/7
Deepak S [Tue, 20 Jun 2017 21:18:57 +0000 (14:18 -0700)]
IXIA traffic generator

Change-Id: I09bcb3f2c4b945283070d442589d3bf00468abbc
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoNSB update 37/38137/40
Deepak S [Tue, 20 Jun 2017 21:31:19 +0000 (14:31 -0700)]
NSB update

Refactored main NSB VNF classes accroding to class diagram

https://wiki.opnfv.org/display/yardstick/NSB+class+diagram

All the SampleVNFs have been separated and placed under
the SampleVNF class.

Added AutoConnectSSH to automatically create SSH conneciton on demand.

Added VnfdHelper class to wrap the VNFD dictionary in prepartion for
class-based modeling.

Extracted DpdkVnfSetupEnvHelper for DPDK based VNF setup.

Extracted Stats and other client config to ResourceHelper

Had to replace dict_key_flatten with deepgetitem due to Python 2.7
Jinja2 infinite recursion.

Change-Id: Ia8840e9c44cdbdf39aab6b02e6d2176b31937dc9
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agopip_license: quick hack script to add license info to requirements.txt 75/38875/1
Ross Brattain [Tue, 8 Aug 2017 01:39:28 +0000 (18:39 -0700)]
pip_license: quick hack script to add license info to requirements.txt

Change-Id: Ic41772cb32cd53d405b16880bd43cf89307b4716
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agofix heatclient import 01/38701/1
Ross Brattain [Thu, 3 Aug 2017 20:18:00 +0000 (13:18 -0700)]
fix heatclient import

for some reason, maybe apexlake, we didn't do the correct import

we need to import heatclient.client directly

Change-Id: I682c88ae780845adb0b5aa898390697197e3af5d
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "DRAFT: remove apexlake"
Ross Brattain [Thu, 3 Aug 2017 15:09:29 +0000 (15:09 +0000)]
Merge "DRAFT: remove apexlake"

6 years agoNew storage test case: Bonnie++ 85/38585/14
JingLu5 [Wed, 2 Aug 2017 07:16:27 +0000 (07:16 +0000)]
New storage test case: Bonnie++

JIRA: YARDSTICK-770

Bonnie++ is a disk and file system benchmarking tool for measuring I/O performance.
With Bonnie++ you can quickly and easily produce a meaningful value to represent
your current file system performance.

This work is add new storage test case using Bonnie++.

Change-Id: I752fee156707cda730962c68d17fda4d4e9cd472
Signed-off-by: JingLu5 <lvjing5@huawei.com>
6 years agoMerge "replace yaml.load with yaml.safe_load"
Ross Brattain [Thu, 3 Aug 2017 01:54:40 +0000 (01:54 +0000)]
Merge "replace yaml.load with yaml.safe_load"

6 years agoMerge "Fix adding right deb repo based on the distro we are running on"
Ross Brattain [Wed, 2 Aug 2017 19:38:33 +0000 (19:38 +0000)]
Merge "Fix adding right deb repo based on the distro we are running on"

6 years agoMerge "Add task-args(from yaml file) candidates in /api/v2/yardstick/testcases API"
Jing Lu [Wed, 2 Aug 2017 01:20:22 +0000 (01:20 +0000)]
Merge "Add task-args(from yaml file) candidates in /api/v2/yardstick/testcases API"

6 years agoAdd task-args(from yaml file) candidates in /api/v2/yardstick/testcases API 91/38391/2
chenjiankun [Mon, 31 Jul 2017 02:49:12 +0000 (02:49 +0000)]
Add task-args(from yaml file) candidates in /api/v2/yardstick/testcases API

JIRA: YARDSTICK-766

We have the demand to get all task-args candidates in test case yaml
file,
so that user get easily change it.
The response will like:
{
    'status': 1,
    'result': {
        'testcase': case content,
        'args': {
            'image': {'description': '', 'type': 'String'}
        }
    }
}

In this patch, I add jinja2schema 0.1.4 in requirements.txt.

Change-Id: I450082402370add5ab29090286f026fe3cc8c36e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
6 years agoMerge "Add spec cpu2006 test case"
Jing Lu [Wed, 2 Aug 2017 00:46:00 +0000 (00:46 +0000)]
Merge "Add spec cpu2006 test case"

6 years agotest_env_action: speedup unittest, don't sleep 25/37425/2
Ross Brattain [Thu, 13 Jul 2017 18:21:46 +0000 (11:21 -0700)]
test_env_action: speedup unittest, don't sleep

Change-Id: Ib662032e5133b8fead1c6858905bd13ca40f4dd6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoreplace yaml.load with yaml.safe_load 17/38017/4
Ross Brattain [Sat, 22 Jul 2017 22:15:13 +0000 (15:15 -0700)]
replace yaml.load with yaml.safe_load

yaml.safe_load is safer, obviously.

anteater will check for this

template_format use specialized constructor based on yaml.SafeLoader

JIRA: YARDSTICK-760

Change-Id: Ia3b0b3aa0765385a0ee472a4d83f49d424b5a77f
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoBugfix: port_security_enabled issue 11/38511/2
JingLu5 [Tue, 1 Aug 2017 08:24:01 +0000 (08:24 +0000)]
Bugfix: port_security_enabled issue

JIRA: YARDSTICK-765

When port_security_enabled is not set, VMs are assigned with security group 'default'.
When using 'default' security group, all egress traffic and intercommunication in the
default group are allowed and all ingress from outside of the default group is dropped
by default (in the default security group). This causes yardstick cannot ssh into VMs.

If port_security_enabled is not set, we should still add the security group that created
by yardstick to the VMs.

Change-Id: Ifd22fb452e0077581b6900f8f51c4e3c342a30aa
Signed-off-by: JingLu5 <lvjing5@huawei.com>
6 years agoDRAFT: remove apexlake 99/38499/3
Ross Brattain [Tue, 1 Aug 2017 03:10:20 +0000 (20:10 -0700)]
DRAFT: remove apexlake

apexlake is unmaintained, so remove it

From some reason orchestrator/heat.py started failing
so fixup those unittests

Change-Id: Ie06508b5ab7c9dcf9fdfca83e173a188a894d564
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "bugfix: enable vlan and physical netwok able to set"
Jing Lu [Tue, 1 Aug 2017 03:36:34 +0000 (03:36 +0000)]
Merge "bugfix: enable vlan and physical netwok able to set"

6 years agoMerge "Add more parameters in iperf3 taml"
Jing Lu [Tue, 1 Aug 2017 03:34:07 +0000 (03:34 +0000)]
Merge "Add more parameters in iperf3 taml"

6 years agocover.sh: delete .testrepository before running coverage 95/38495/1
Ross Brattain [Tue, 1 Aug 2017 02:34:16 +0000 (19:34 -0700)]
cover.sh: delete .testrepository before running coverage

When running py27 and py3 test ran into problems
with .testrepository already exists, but testr
thinking it was corrupt

running testr
No repository found in /home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master. Create one by running "testr init".
error: testr failed (3)

The fix seems to be to delete .testrepository before
running testr coverage

Change-Id: Ib8cd3ab9d3384935380ac29ce365439c6464adc3
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
6 years agoMerge "Bugfix: yardstick always report 'PASS' to DB"
Rex Lee [Tue, 1 Aug 2017 02:18:33 +0000 (02:18 +0000)]
Merge "Bugfix: yardstick always report 'PASS' to DB"

6 years agoMerge "Add yardstick logo in GUI"
Rex Lee [Tue, 1 Aug 2017 02:17:22 +0000 (02:17 +0000)]
Merge "Add yardstick logo in GUI"

6 years agoMerge "Add container ping VM test case"
Jing Lu [Tue, 1 Aug 2017 02:02:57 +0000 (02:02 +0000)]
Merge "Add container ping VM test case"

6 years agoAdd yardstick logo in GUI 95/38395/3
chenjiankun [Mon, 31 Jul 2017 03:05:36 +0000 (03:05 +0000)]
Add yardstick logo in GUI

JIRA: YARDSTICK-767

We have yardstick logo in opnfv community.
We have the demand to display it in yardstick GUI.

Change-Id: Ief5807c918754e679e95ae1d1f3a769ab2d6d4b2
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
6 years agoMerge "add opnfv_os-odl-sfc-ha_daily.yaml test suite"
Jing Lu [Tue, 1 Aug 2017 01:09:31 +0000 (01:09 +0000)]
Merge "add opnfv_os-odl-sfc-ha_daily.yaml test suite"

6 years agoMerge "run_tests: don't write python bytecode"
Ross Brattain [Mon, 31 Jul 2017 21:26:15 +0000 (21:26 +0000)]
Merge "run_tests: don't write python bytecode"

6 years agocover: another 'db type could not be determined' workaround 83/38483/1
Ross Brattain [Mon, 31 Jul 2017 21:13:58 +0000 (14:13 -0700)]
cover: another 'db type could not be determined' workaround

Change-Id: Ib94ff2dfc86725e5367908296b5160f9565442b8
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>