Merge "remove trex install from run_tests.sh."

Merge "heat: convert open to context manager"

Merge "heat: close file before parsing template"

Merge "vnf_generic: replace list comprehension with generator expression"

Merge "Bugfix: write_json_to_file: use json.dump with files"

Merge "nsb: move NSB root to constant, add unittest"

Merge "vnf_generic: convert sshmanager to class"

Merge "test_tg_text: fix indentation"

Merge "vnfdgen: replace six.iteritems() with .items()"

Merge "test_vnf_generic: fix unitests when files are present"

Merge "Bugfix: apex_lake: sort json keys for git"

Merge "test_env_action: use dict literal"

Merge "Push yardstick debug log into the artifacts"

Merge "KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk+barometer scenarios."

KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk+barometer scenarios.

This patch implements test suites for kvmfornfv os-nosdn-kvm_ovs_dpdk_bar-ha
and os-nosdn-kvm_ovs_dpdk_bar-noha scenarios.

Change-Id: Idb7d360aa5b91a15d1c69566418a9da4b68db7ab
Signed-off-by: Navya Bathula <navyax.bathula@intel.com>

Merge "uwsgi: move init_db to uwsgi entry point"

Fix: yardstick-docker-build-push-master failure

JIRA: YARDSTICK-552

Change-Id: I5c006b2fcbffb6a3904c378ab0557b5ce437d9ed
Signed-off-by: JingLu5 <lvjing5@huawei.com>

remove trex install from run_tests.sh.

We have mocked out all the Trex imports, so we don't need to download
the Trex library

It doesn't really make sense to down external libraries
each time we run unittests.  Trex is a large stand-alone package
and shouldn't really be used during unittests

Importing Trex caused some odd pyzmq import errors on Python 3

File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master
  /trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18
  /64bit/zmq/backend/cython/__init__.py", line 6, in <module>
    from . import (constants, error, message, context,
ImportError: cannot import name 'constants'

Change-Id: I2ed06c0f88061baeff9ec96534cc59cca81924cc
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

uwsgi: move init_db to uwsgi entry point

uwsgi will call the 'callable' function which should normally be
Flask.__call__().

But we need to init the db first, so make a wrapper function that inits
the db and then calls app()

Also refactor add_resource into for loop instead of reduce.  reduce is
not really approriate here since we aren't consuming the return value

JIRA: YARDSTICK-543

Change-Id: I692d6d42de09f7d6ecf1a67a22e3019d97a4f3ca
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

test_env_action: use dict literal

use literal forms when possible

Change-Id: I2d815f40007aa2259f3f3bcb2b9cd84a11027422
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Bugfix: apex_lake: sort json keys for git

experiment_1.yaml.json are in git
and everytime the unnitests are run the
json file is re-generated with a slightly different
key order so it make the git directory dirty.

use sort_keys=True to try to maintain a consistent
json file so git doesn't detect a difference

Change-Id: I7859664e00561e8db032849a6664fb933a36702b
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

nsb: move NSB root to constant, add unittest

add unittest for non-None default
fixup get_nsb_option to return default correctly.
use NSB_ROOT for all paths

Change-Id: Idd1951a8d436bc49c30d59d84ca12a5f26e9148f
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

test_vnf_generic: fix unitests when files are present

replace paths with empty strings so unittests see
the Exception they expect when the files are not present

Otherwise unitests are dependent on local filesystem state

fix pylint issues, adjust formatting
removed duplicate key

Change-Id: I7857988c6e6bf586b0eb403fb1d3a3da7f170cbf
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

vnfdgen: replace six.iteritems() with .items()

Openstack python3 style guide prefers to not use six.iteritems() but
instead use data.items() for all cases.

We should follow the Openstack guides since OPNFV is aligned with
Openstack

https://wiki.openstack.org/wiki/Python3#Port_Python_2_code_to_Python_3

Common patterns:
* Replace "for key in dict.iterkeys()" with "for key in dict"
* Replace dict.iteritems() with dict.items()
* Replace dict.itervalues() with dict.values()

Change-Id: I99b0505199a568771341658caf8b71c1896db9cd
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

vnf_generic: replace list comprehension with generator expression

no need to build a list, just use next

Change-Id: I8a899ac538849b765f12bebda4fb8c89c84f333e
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

vnf_generic: convert sshmanager to class

@contextmanager have an issue with respect to exceptions that makes
them not suitable for real usage.

@contextmanager uses yield to create a generator and then uses
generator.throw() to raise any exceptions.  Exceptions thrown from
generators loose their call stack due to the way generators work, so any
exception inside a context manager is harder to debug. For this reason
we don't use @contextmanager and instead always define a new class with
__enter__ and __exit__.

There is sample code that demonstrates the
issue with @contextmanager and generator.throw() here
https://gist.github.com/rbbratta/e28b6e64a4551522c3ac9815ca7f25f0

Change-Id: I5383c01f40a63e33680112f39b5bd9c858e328f1
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Bugfix: write_json_to_file: use json.dump with files

In python3 file objects automatically handle encoding to utf-8 and expect
string, not bytes, so use regular json.dump() with the file object

We should only use dump_as_bytes to replace json.dumps(), not
to replace json.dump()

This fixes Python3 issue:

Traceback (most recent call last):
  File "yardstick/main.py", line 52, in <module>
    main()
  File "yardstick/main.py", line 49, in main
    YardstickCLI().main(sys.argv[1:])
  File "yardstick/yardstick/cmd/cli.py", line 167, in main
    self._dispath_func_notask()
  File "yardstick/yardstick/cmd/cli.py", line 145, in _dispath_func_notask
    func(CONF.category)
  File "yardstick/yardstick/cmd/commands/task.py", line 48, in do_start
    self._init_result_file()
  File "yardstick/yardstick/cmd/commands/task.py", line 57, in _init_result_file
    write_json_to_file(consts.DEFAULT_OUTPUT_FILE, data)
  File "yardstick/yardstick/common/utils.py", line 152, in write_json_to_file
    write_file(path, jsonutils.dump_as_bytes(data), mode)
  File "yardstick/yardstick/common/utils.py", line 157, in write_file
    f.write(data)
TypeError: write() argument must be str, not bytes

Change-Id: I573419be25d8fa1f015e1507730ba66c05f86686
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

test_tg_text: fix indentation

flake8 error

Change-Id: Ia4c90c2703335123ea48c8421a84d7288126dcf6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Proposing Trevor Cooper as a commiter in yardstick

Trevor Cooper has contributed greatly to yardstick D release.
Achievement/Contributions include:
Vsperf integration with yardstick.
NSB integration coordination.
Host US slot yardstick weekly meeting.
Draft Stress Test proposal in testperf.
Draft new Testing Projects Ecosystem graph.

Change-Id: I783335a77cfe3926c0c8ff76174ef3cc4f203fd6
Signed-off-by: kubi <jean.gaoliang@huawei.com>

Proposing Ross Brattain as a commiter in yardstick

Ross has contributed greatly to yardstick D release.
Achievement/Contributions include:
Provided quite many high-quality review comments.
Prepare Yardstick for Python3
Support Vsperf integration with yardstick.
Support NSB integration.
Jenkins verify test evolvement.
Yardstick Architecture Improvement.
LOG system unified.
https://gerrit.opnfv.org/gerrit/#/q/project:yardstick+owner:%22Ross+Brattain+%253Cross.b.brattain%2540intel.com%253E%22

Change-Id: Ided9c95e856269277ff09572e92e9fda7288734e
Signed-off-by: kubi <jean.gaoliang@huawei.com>

Pass environment variables of proxy to tox

When a development environment is under a proxy, tox is failed even if
environment variables of the proxy are set.

Change-Id: Ia6f8f2a1f215fc4c861d7dbc7a61dcd6973fb3cb
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

fix unittests failures due to py3

Traceback (most recent call last):
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_vpe_vnf.py", line 306, in test_instantiate
    self.context_cfg))
AssertionError: 0 != None

Traceback (most recent call last):
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_base.py", line 55, in test_clear
    self.assertEqual(queue_file_wrapper.q_out.empty(), True)
  File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.5/unittest/case.py", line 813, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: False != True

Traceback (most recent call last):
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 321, in test_run_traffic
    self.assertEqual(True, result)
  File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.5/unittest/case.py", line 813, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: True != False

Traceback (most recent call last):
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_rfc2544_trex.py", line 369, in test_run_traffic
    self.assertEqual(True, result)
  File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual\
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.5/unittest/case.py", line 813, in _baseAssertEqual\
    raise self.failureException(msg)
AssertionError: True != False

Change-Id: I68340196b8cc9f0fc2e4e0ef1022e8098fc860f0
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

fix unittests, mock trex_stl_lib, fix raw_input, division

replace raw_input with six.moves.raw_input
fix raw_input mock
force float division in python 2

re-added trex download to try to workaround coverage failing
try installing pyzmq=14.5.0 and see if that helps trex compatibility

======================================================================
ERROR: test__fill_traffic_profile (benchmark.scenarios.networking.test_vnf_generic.TestNetworkServiceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/benchmark/scenarios/networking/test_vnf_generic.py", line 399, in test__fill_traffic_profile
    self.context_cfg))
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/benchmark/scenarios/networking/vnf_generic.py", line 144, in _fill_traffic_profile
    return TrafficProfile.get(traffic_profile)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/base.py", line 35, in get
    "yardstick.network_services.traffic_profile")
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 86, in import_modules_from_package
    try_append_module(module_name, sys.modules)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 70, in try_append_module
    modules[name] = importutils.import_module(name)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/oslo_utils/importutils.py", line 73, in import_module
    __import__(import_str)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/rfc2544.py", line 19, in <module>
    from yardstick.network_services.traffic_profile.traffic_profile \
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/traffic_profile.py", line 24, in <module>
    from stl.trex_stl_lib.trex_stl_client import STLStream
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module>
    from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module>
    import zmq
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module>
    from zmq import backend
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module>
    reraise(*exc_info)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise
    raise value
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module>
    _ns = select_backend(first)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend
    mod = __import__(name, fromlist=public_api)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module>
    from . import (constants, error, message, context,
ImportError: cannot import name 'constants'

======================================================================
ERROR: network_services.vnf_generic.vnf.test_tg_trex (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: network_services.vnf_generic.vnf.test_tg_trex
Traceback (most recent call last):
  File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 24, in <module>
    from stl.trex_stl_lib.trex_stl_client import STLClient
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module>
    from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module>
    import zmq
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module>
    from zmq import backend
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module>
    reraise(*exc_info)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise
    raise value
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module>
    _ns = select_backend(first)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend
    mod = __import__(name, fromlist=public_api)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module>
    from . import (constants, error, message, context,
ImportError: cannot import name 'constants'

Change-Id: I832bf8c912dea6d85131ee6603b408b3198cef2f
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk scenarios."

Merge "Adding simple cmdline to run the Network service testcases"

Merge "Setup yardstick_virtual environments for Network Service Testing"

Merge "Adding sample Thoughput Test case for vPE."

Merge "Adding vPE VNF class aligned with IETS per-deploy senarios"

Merge "Adding Trex rfc2544 VNF class to initate Traffic for throughput"

Merge "Adding trex trafficgen example."

Merge "Adding ping testcase to demonstrate the isb Generic framework"

Merge "Adding ping based sample VNF appliance"

Merge "Adding generic traffic profiles for trex traffic generator"

KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk scenarios.

This patch implements test suites for kvmfornfv os-nosdn-kvm_ovs_dpdk-ha
and os-nosdn-kvm_ovs_dpdk-noha scenarios.

Change-Id: Ia7b17dda262df5d81cac87372b70510192044a40
Co-Authored-By: Navya <b.navya4@tcs.com>
Signed-off-by: KalyanReddy <reddyx.gundarapu@intel.com>

Push yardstick debug log into the artifacts

the log directory should be specific yardstick so
it can be easily pushed into artifacts with the
directory.

JIRA: YARDSTICK-507

Change-Id: If28b0c3938e123c8a55272cb98291fb76aaf6f9e
Signed-off-by: rexlee8776 <limingjiang@huawei.com>

nfvi: disable failing AMQP unittest

Failures:

test_connect (network_services.nfvi.test_collectd.TestAmqpConsumer) ... ERROR:pika.adapters.base_connection:Connection to 1.1.1.1:5672 failed: timeout
WARNING:pika.connection:Could not connect, 0 attempts left
ERROR:pika.callback:Calling <bound method SelectConnection._on_connection_error of <pika.adapters.select_connection.SelectConnection object at 0x7fe7e2333710>> for "0:_on_connection_error" failed
Traceback (most recent call last):
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py27/local/lib/python2.7/site-packages/pika/callback.py", line 236, in process
    callback(*args, **keywords)
  File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py27/local/lib/python2.7/site-packages/pika/connection.py", line 1265, in _on_connection_error
    self.params.connection_attempts)
AMQPConnectionError: Connection to 1.1.1.1:5672 failed: timeout
ok

Firstly, 1.1.1.1 is not an approriate fake address, use 127.0.0.1 so we don't try
to connect to anything external

But 127.0.0.1 won't work anyway, so disable test_connect

replace 152.16.0.0 with 172.16.0.0

Remove network_services.nfvi.test_resource.TestResourceProfile since it
also fails due to same error

Remove test_amqp_collect_nfvi_kpi_exception

Change-Id: I00bb1729658e18b4651129661ad9dd9c0dedcf37
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Fix: network_services: decode stdout instead of encode

Fixes unittest on Python 3

stdout is bytes, so we need to decode bytes to unicode using utf-8

bytes.decode(); str.encode()

Change-Id: I4b19d237139f338a648d04aa44cd4f199bb901c3
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "Add infrastructure to add the NFVi KPI collections"

Merge "Adding sample yardstick.conf & pod files to help describe the topology"

Merge "Introducing Generic framework to do pre-deployment VNF & Network service testing"

Merge "basic tox.ini for python 2.7 and python 3 unittests"

Merge "Generic helper function to provision and get path from config"

Merge "Record task status if running via CLI"

Record task status if running via CLI

JIRA: YARDSTICK-542

Currently we do not record task status when using CLI to run task.
So I add this function.
If status=0, task is not done.
if status=1, task is done.
if status=2, there is an error.

Change-Id: Ib9b3c8abd233909c04f792115199250419fa8d7a
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Create API to get a list of all test cases

JIRA: YARDSTICK-456

Currently we do not have a API to get a list of all test cases;
Currently the test case info is from the comment;
So I create a API to get a list of all test cases;
And create a 'description' attribute to record info of a test case;
And use the CLI call this API;

Change-Id: Ife800600446683664097835c7b9f11899c85771d
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Bugfix:can not run a test suite if not under yardstick root path"

Merge "Bugfix: task_id parameter from API can not pass to yardstick core"

basic tox.ini for python 2.7 and python 3 unittests

Nothing fancy for now, just call run_tests.sh

If we want to expand we could add pep8, coverage
to tox.  See dovetail tox.ini for example

Change-Id: I8b5897779ed9622dd1b38612cb393fc214ec6300
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Adding simple cmdline to run the Network service testcases

JIRA: YARDSTICK-522
Change-Id: I5000c0ae9cf128f09b273afd85fd797068516484
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Setup yardstick_virtual environments for Network Service Testing

JIRA: YARDSTICK-521
Change-Id: Ib91d231120067e58d2e1df7f1b42db216d15c468
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Adding sample Thoughput Test case for vPE.

TestCases:
    - 64B TC
    - 1518B TC
    - IMIX TC

JIRA: YARDSTICK-520
Change-Id: Ic7842de8afb0f5c222de42f99bf70af29442c94a
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Adding vPE VNF class aligned with IETS per-deploy senarios

VNF life cycle
- instantiate
- collect_kpi
- terminate

JIRA: YARDSTICK-520
Change-Id: Ied6ac79870fa35cc3dd14fd7a99b6bade3b77c81
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Adding Trex rfc2544 VNF class to initate Traffic for throughput

JIRA: YARDSTICK-520
Change-Id: I1c683236a7fb946873418fb67f63500e1ba8fc91
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Adding trex trafficgen example.

This patch uses trex trafficgen example to define dynamic traffic profiles
and how it can be mapped to real world traffic.

JIRA: YARDSTICK-492
Change-Id: Ica24957ebf43315a8d81adabd4745c27d3c7c36a
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Adding ping testcase to demonstrate the isb Generic framework

This patch defines
  - 2tg-topology-baremetal.yaml --> Define topology
  - pod.yaml -> sample pod.yaml describing TG & VNF unit details
  - tc.yaml --> test case in yardstick format (scenarios/context)
  - tg_ping_tpl.yaml --> VNF descriptor explaining how units are connected,
eg Host (Xe0) --> Target (Xe0)
   Host (Xe1) --> Target (Xe1)

JIRA: YARDSTICK-491
Change-Id:I41b69f457a6caa58d806cac9af8e831752ad314d
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Adding ping based sample VNF appliance

This patch defines
 - Generic VNF APIs to test Network service
--> instantiate
--> collect_kpi
--> run_traffic
--> listen_traffic
--> terminate
 - vnf Descriptor to map the physical NFVi topology of the Test unit.

JIRA: YARDSTICK-491
Change-Id: I6b7e09972fc536977b65d8a19d635a220815e5f3
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Adding generic traffic profiles for trex traffic generator

This patch defines Generic traffic profiles
 - rfc2544, http etc

JiRA: YARDSTICK-489
Change-Id:I0d8270b4d5f5f2d3415b98182990d8649099dbe3
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Add infrastructure to add the NFVi KPI collections

 This patches added common function to collect NFVi KPIs for given usecases
 - Core KPIs like memory/LLC/IPC etc
 - OVS stats
 - memory stats etc.

JIRA: YARDSTICK-488
Change-Id: Iab41146392efc47b7313b1846a67728a44d0f1d6
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Adding sample yardstick.conf & pod files to help describe the topology

JIRA: YARDSTICK-487
Change-Id:I23dabcf6b4b7d49b8eb0f18778cddd50a1195a05
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Introducing Generic framework to do pre-deployment VNF & Network service testing

This patch introduces the framework which is aligned with ETSI-TST001

This patch adds:
1. NetworkServiceTestCase introduces following functions
      --> setup
            --> Verify if infrastructure mapping can meet topology
            --> Load VNF models
            --> Fill traffic profile with information from topology
            --> Provision VNFs
            --> Run experiment (traffic)
      --> run -> Yardstick calls run() at intervals defined in the yaml
                 and produces timestamped samples
      --> teardown --> Stop  VNFs

2. TrafficProfile is a generic class to get traffic profile for a given
   testcase and select the traffic generator for testcase.

3. QueueFileWrapper is a class to send/recive cmds to vnf

4. GenericVNF is a generic class to instantiate VNF

5. GenericTrafficGen is a generic class to run/listen/verify traffic.

JIRA: YARDSTICK-483
Change-Id: Ic453c917d34dcb508a7f3afb459011da85f6402e
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Generic helper function to provision and get path from config

This patch adds, generic helper function to provision the tools
and get required fields from yardstick.conf

v2: Added unit tests to keep test coverage :)

JIRA: YARDSTICK-484
Change-Id: Id6701924e3488c7f38f29c82e55c27fba67c0d76
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Introducing the collector to subscribe/publish the KPIs for yardstick plot

v2: Added unit tests to keep test coverage :)

JIRA: YARDSTICK-482
Change-Id: I9281b00a4b619cc04550cb623c027ee5765c4974
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Introducing Standalone context for running test in non-managed environment.

This patch introduces standalone virtualization context to
deploy/undeploy NFVi infrastructure to run the VNF

Supported NFVi Type:
  - vswitch
   - ovs
   - ovs-dpdk
   - sr-iov
   - testpmd
   - linuxbridge

This patches inits the function stubs to enable the standalone context.
Actual deploy/undeploy code will be added in later check-in

v2: Added unit tests to keep test coverage :)

JIRA: YARDSTICK-479
Change-Id: I6ab3ac3335f40eabc4efb0af7d5addc20c122d65
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Merge "Verify stdin data before doing encodeutils safe_encode"

Merge "Bugfix: command "openstack image list" failed"

Bugfix: command "openstack image list" failed

JIRA: YARDSTICK-537

Now the openstackclient version is set to 3.3.0, but it cannot support
osc-lib 1.3.0 whose version is automatic set.
So I set openstackclient version to 3.7.0 and osc-lib version to 1.2.0

Change-Id: If4ccd517f6a572f6c463270596d0e1019ba9a1e6
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Bugfix: context not found for server ''node1.LF''"

Bugfix: context not found for server ''node1.LF''

JIRA: YARDSTICK-540

We are adding dynamically generated suffix to server name in the
previous patch.
But in that patch we just change the 'target', 'host' in nodes, but not
'node1', 'node2' ...
This patch will change all this.

Change-Id: Ic7c82c323dea24a40f1fe30871603d30c2689f67
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Re-factor Node.py to use better python inbuilt functions"

Merge "Add unittest framework for Yardstick API"

Verify stdin data before doing encodeutils safe_encode

Process running via ssh can return "None" or emtpy data from the
application. To avoid encodutils raise NoneType issue. Check the data before
encode.

JIRA: YARDSTICK-539

Change-Id: I7e86e6a17c0adc95d41714f6fec463dfadc2b81b
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Re-factor Node.py to use better python inbuilt functions

1. don't use sys.exit() to exit when there is an exception, it will hide
the underlying error
2. use the Abstract Base Classes for type checking.
3. don't have to build list, can use next

JIRA: YARDSTICK-541

Change-Id: Id4485acb21e7e02bbc22d3e689cbf0699363098a
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Merge "Use """ to replace ''' in docstring"

heat: close file before parsing template

we don't need to keep the file open once
we have read it

Change-Id: I7234fe91d2080d357c7a6506543cef067a9c2aac
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

heat: convert open to context manager

Change-Id: I5ce6c77839f4104d305d66238bb2c6cbdc39a28c
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Add unittest framework for Yardstick API

JIRA: YARDSTICK-538

Currently it is hard to test API, So I add a base class as flask
document do.
In this framework I will mock a temp sqlite database and a server.

Change-Id: If881233cb22655617c07ad018201b8ee08492d06
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Bugfix: command "openstack image list" failed

JIRA: YARDSTICK-537

Now the openstackclient version is set to 3.3.0, but it cannot support
osc-lib 1.3.0 whose version is automatic set.
So I set openstackclient version to 3.3.0 and osc-lib version to 1.3.0

Change-Id: Ia480feb3621be97a4a97bb027685cc438e01c53e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Adding python package requirement for VNF testing."

Use """ to replace ''' in docstring

JIRA: YARDSTICK-525

For consistency, we always use """triple double quotes""" around
docstrings.

Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "remove failing influx testcases"

Bugfix:can not run a test suite if not under yardstick root path

JIRA: YARDSTICK-530

Currently we can not run a test suite if we are not in yardstick root
path.
The reason is that the file in test suite config file use relative path.
So I change it to absolute path when run.

Change-Id: I62758bc67f466ac794d339b597562b3be05574fb
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add support for Python 3

Porting to Python3 using Openstack guidelines:
https://wiki.openstack.org/wiki/Python3

This passes unittests on Python 3.5 and passes opnfv_smoke suite

Updates:
   use six for urlparse and urlopen
   fix exception.message attribute removal
   run unittests on python3
   use unitest.mock on python 3
   fix open mock for vsperf
   fix float division by using delta/eplison comparison
   use unicode in StringIO
   use plugin/sample_config.yaml relative path from test case
   fixed apexlake unittests
   upgraded to mock 2.0.0 to match python3 unittest.mock features
   fixed flake8 issues
   implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes()
   implement safe unicode encode/decode with oslo_utils.encodeutils

heat: convert pub key file from bytes to unicode
    pkg_resources returns raw bytes, in python3
    we have to decode this to utf-8 unicode
    so JSON can encode it for heat template

JIRA: YARDSTICK-452

Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

heat: fix key_uuid format

The .format() won't work, since it can't slice the uuid

We have to convert the uuid to string before we slice it.
I thought .format() would implicitly call __str__() before
applying width, but that is not the case.

'files/yardstick_key-{:.{width}}'.format(self.key_uuid, width=8))

We also need to define a constant short uuid lenght, we can't
hardcode the length to 8 everywhere.

Create a helper function to standardize the generation
of the short key uuid and use that helper function everywhere

Change-Id: I59e051bfe697587e967f93f5b8f209e0e7daa5c7
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

remove failing influx testcases

when influx was refactored these test cases must not
have been removed

>       influx._write_data(measurement, field, timestamp, tags)
E       AttributeError: 'module' object has no attribute '_write_data'

Change-Id: I78814266fae04bd3b0bc06fe1fe41317ba8aced4
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Bugfix: task_id parameter from API can not pass to yardstick core

JIRA: YARDSTICK-531

Now in API entry the task_id parameter will not pass to yardstick core.
I fix it by pass task_id to yardstick.benchmark.core.task.start() method.

Change-Id: I66439660ff116d83104e5ba4f040106ca73142e6
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Create API to run test suite"

Merge "associate an uuid to yardstick_key and yardstick_key.pub"

Create API to run test suite

JIRA: YARDSTICK-475

This API will be used to run test suite files in the test/opnfv/test_suites directory

Change-Id: I208edf9abed62fd6436de988ac85bfe99c4d01bd
Signed-off-by: JingLu5 <lvjing5@huawei.com>

associate an uuid to yardstick_key and yardstick_key.pub

JIRA: YARDSTICK-527

This work is to support yardstick parallel tasks in the future.
Currently, the RSA key we generated and used to access the VM is named
'yardstick_key'.
If more than two tasks are running paralleled, the later 'yardstick_key' will
cover the former.
We want associate an uuid to identify differnets for each tasks. So the key
files won't conflict.
The first 8 digits will be used, as there is no need to used a full-length uuid.

Change-Id: If8eaf47ae527cf9b3bd50f37ab3051fbdccf5f03
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Merge "Yardstick framework concurrent support"