Merge "Move tests: unit/network_services/vnf_generic" into stable/fraser

Merge "Standalone XML machine type is not longer valid" into stable/fraser

Move tests: unit/network_services/vnf_generic

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

JIRA: YARDSTICK-837

Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Change-Id: I3d95537a64bde923703421510d27bac0d6dd92bd
(cherry-picked from commit 98294f2c8153f663c3b3a4e4cb98910e9d5cb602)

Merge "Remove instantiated contexts in "test_task"" into stable/fraser

Remove instantiated contexts in "test_task"

During unit testing, all contexts instantiated must be deleted at the end
of the test to avoid clashes with other tests.

JIRA: YARDSTICK-1209

Change-Id: I1994b915001b4a518e096833a0c5d33febbd8de6
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit 5e4d95ba299ce92588701d9481657c343b8ef609)

Merge "Move tests: unit/network_services/nfvi" into stable/fraser

Merge "Add "duration" parameter to test case definition" into stable/fraser

Merge "Remove print out of logger exception in TestUtils" into stable/fraser

Merge "Cleanup TestYardstickNSCli class" into stable/fraser

Merge "Document for Fraser test case results" into stable/fraser

Move tests: unit/network_services/nfvi

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

JIRA: YARDSTICK-837
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Change-Id: Id330ee1c89d54ef487edb7fe160e5a31d9a3094f
(cherry-picked from commit e12f98c2bbed5191714bdc553e7b269b1437cecd)

Add "duration" parameter to test case definition

Add "duration" parameter to test case definition, in scenario:options
section. This parameter will be rendered in the traffic profile.

If the parameter is not present in the test case scenario options, the
default time written in the traffic profile options will be 30 seconds
(TrafficProfile.DEFAULT_DURATION = 30). If the traffic profile
injection time is not defined, the default injection time will be 30
seconds.

  testcase:scenario:options:duration (default = 30)
      render --> traffic_profile:duration
          parse --> TrafficProfile.duration (default = 30)

Target traffic profiles (RFC2544):
  - RFC2544Profile
  - IXIARFC2544Profile

JIRA: YARDSTICK-1194

Change-Id: I968922e6bb882d7ee15aa1c4db4037face7a3492
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit 1517319fd210d71db72c5e4bfa00fc49ae4fa877)

Improve IXIA IxNetwork library and traffic profile (4)

This patch implements an active wait for the traffic injection. Once the
traffic is started, the traffic generator class will poll periodically the
IXIA traffic generator chassis to retrieve the status of the traffic
("started", "stopped").

Now the latency statistics are retrieved and reported for each injection
period.

JIRA: YARDSTICK-1116

Change-Id: I4422e2c88b4fc97b7cac3de8a82b2d75467c4117
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
(cherry picked from commit a3399d07b83ce0e50d9c0144d00a7ba83a73390f)

Improve IXIA IxNetwork library and traffic profile (3)

This patch modifies IP packet parameters.

"IxNextgen.update_ip_packet" modifies the L3 packet according to the
test case and setup the IP addresses.

JIRA: YARDSTICK-1116

Change-Id: I46ff75ab1989d0e6f5cc876418a015386717e06f
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
(cherry picked from commit e5c78ae3ec1bcf0cb44646c6031fb9c04dd63e4a)

Improve IXIA IxNetwork library and traffic profile (2)

This patch modifies the way the packet frame and the flow is configured
before the injection.

"IxNextgen.update_frame" modifies the L2 frame according to the
test case and setup the frame rate, frame size, traffic injection
duration and MAC addresses.

JIRA: YARDSTICK-1116

Change-Id: Ife08f15a4eda24d7835c92c4172b450854d112ee
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
(cherry picked from commit 8d10a6a8447fc2798491a6f3856e21b85fa309ba)

Improve IXIA IxNetwork library and traffic profile (1)

This patch introduces a new way to configure the TCL IxNetwork server.
All the configuration is done using the TCL API, removing the need of
using the pre-saved configuration file.

"IxNextgen.assign_ports" creates and assigns the virtual ports for each
physical port defined in the test case.

"IxNextgen.create_traffic_item" creates one traffic item and two flow
groups per pair of ports, in both directions.

"IxNextgen.gt_statistics" retrieves only the required statistics to
generate the samples blob in the traffic generator.

JIRA: YARDSTICK-1116

Change-Id: I8f1c0c55e99c274b2ed8276ed9a385c502e16d93
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
(cherry picked from commit ee721fd3e8b77ccfe4252a107a9af8dc41ccc389)

Merge "Add active wait function" into stable/fraser

Merge "Add IxNetwork API Python Binding package" into stable/fraser

Merge "Use TRex release v2.41 to support both x86 and aarch64" into stable/fraser

Add active wait function

Added function "wait_until_true". This function will make an active
wait until the predicate passed as an argument returns True.

If the timeout expires, the function will raise a generic exception
or a user defined one passed as an argument.

This function will be used in YARDSTICK-1127.

JIRA: YARDSTICK-1128

Change-Id: I9854e465ac6b586bf4be39ab4b266d5625b39e30
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from 099108aab37d1fae1b27f4e2e20136c234df1a52)

Add IxNetwork API Python Binding package

Add IxNetwork python package to the requirements list.

This module is needed for the "NSPerf" scenarios using IXIA as traffic
generator, when IxNetwork is the program used to handle the IXIA
generator.

The latest version of this module is 8.40.1124.9 [1]. The license type
is MIT.

[1] https://pypi.python.org/pypi/ixnetwork

JIRA: YARDSTICK-1106

Change-Id: I279933c95994f5120930256a104c4192a0b9900d
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Merge "Configure ACL via static file" into stable/fraser

Merge "Move tests: unit/network_services/traffic_profile" into stable/fraser

Merge "Cleanup BaseMonitor unit tests" into stable/fraser

Move tests: unit/network_services/traffic_profile

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

JIRA: YARDSTICK-837
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Change-Id: Id983a2e415d66633331e7fea96a377e2a7589980
(cherry-picked from commit cfc40ad1bb36889e54bc99cb779cfcfa1f3b04dd)

Merge "MQ startup process refactor" into stable/fraser

Merge "Add "host_name_separator" variable to Context class" into stable/fraser

Merge "Move IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptions" into stable/fraser

Use TRex release v2.41 to support both x86 and aarch64

Currently Yardstick uses Cisco's TRex release v2.28 which
can only support x86, for the consideration of
multi-arch support of Yardstick, the trex release v2.41
is now available and it can support multi-arch(both x86
and aarch64) now.

For the external lib of v2.41, for x86_64, it looks like:
external_libs/pyzmq-14.5.0/python2/intel/ucs4/64bit/zmq
for arm64, it looks like:
external_libs/pyzmq-14.5.0/python2/arm/ucs4/64bit/zmq
So replace the trex release v2.28 with v2.41.

JIRA: YARDSTICK-1191

Change-Id: I22dc3499283d48a89da1145cc480fd7d2a93a1cc
Signed-off-by: trevortao <trevor.tao@arm.com>
(cherry picked from commit 13f19d35a04ffb46ca673e81385779bc46c48781)

Configure ACL via static file

This patch allows user to configure ACL/vFW SampleVNF ACL
via configuration file provided in TC definition. The
Yardstick applies the rules to SampleVNF from specified config
file + rules generated by Yardstick (default rules).

The example of SampleVNF ACL CLI commands generated/applied
by Yardstick can be found at (using default ACL config file):

ACL VNF ACL CLI commands:
  acl/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
  http://paste.openstack.org/show/723303/

vFW VNF ACL CLI commands:
  vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
  http://paste.openstack.org/show/723304/

Change-Id: I76a630261a982083b628e3985fc3bec14ca495db
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
(cherry picked from commit 3fdca97ff5053770161059a34a95fd39463eaecd)

MQ startup process refactor

JIRA: YARDSTICK-1112

Change-Id: Id30236e190bae9fb0f5b5ebd4c5e04798e4fd443
Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
(cherry picked from commit d38bd854fb2c3eb6c0c4dd34ace2a52af7cf892d)

Merge "Do not start collectd twice when SampleVNF is running on Baremetal" into stable/fraser

Add scale out TCs with availability zone support

Add PROX and vFW scale out TCs with availability zone
configuration. To be able to specify the availability
zone for specific VNF or TG the following example of
command line option shoud be used:

  yardstick -d task start --task-args='{
    "num_vnfs": 2, "availability_zone":
     {
      "vnf_0": "az_0", "tg_0": "az_0",
      "vnf_1": "az_1", "tg_1": "az_1",
      ... }}' <SCALE-OUT-TC-FILE>

The steps to run new scale-out TCs is documented in the
userguide.

JIRA: YARDSTICK-1203

Change-Id: Ib3878485c5483981fe9b48ab02995712bcc31ae5
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
(cherry picked from commit 0156b5c37e08f91a37dda08edb854758bbba7413)

Merge "Add --hwlb options as a command line argument for SampleVNF" into stable/fraser

Standalone XML machine type is not longer valid

Current XML machine type definition [1] is not longer valid. According to
[2], the default machine type should be "pc-i440fx-xenial".

NOTE: StandAlone physical nodes are deployed using Ubuntu OS.

[1]https://github.com/opnfv/yardstick/blob/master/yardstick/benchmark/contexts/standalone/model.py#L48
[2]https://wiki.ubuntu.com/QemuKVMMigration#Upgrade_machine_type

JIRA: YARDSTICK-1192

Change-Id: I77b1379aaf416b1225e91af2c3ac746cb101f774
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit 6414a61a988a73ee1b939e2d5de2f0afdde4319a)

Move IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptions

JIRA: YARDSTICK-1218

Change-Id: I804065e9bce3e728f5bf9e756a78df8fd28f74ac
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit dc0e675f47f4dbf1b54ce9c22878e2e876bc26e8)

Move ErrorClass definition to exceptions module

JIRA: YARDSTICK-1216

Change-Id: I82556e1d1b0c723221a58e188067cbce560b8338
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit 2bdda85878ed1507d1a91f69c3489bb9ba9d447d)

Merge "Convert SSH custom exceptions to Yardstick exceptions" into stable/fraser

Merge "Remove AnsibleCommon class method mock" into stable/fraser

Merge "Error when adding SR-IOV interfaces in SR-IOV context" into stable/fraser

Merge "Remove __init__ method overriding in HeatContextTestCase" into stable/fraser

Merge "Bump oslo.messaging version to 5.36.0" into stable/fraser

Merge "Set "cmd2" library to version 0.8.6" into stable/fraser

Error when adding SR-IOV interfaces in SR-IOV context

Method "add_sriov_interfaces" in Libvirt class throws an error when trying
to add SR-IOV interfaces in the XML file.

Function "setup_sriov_context" in SriovContext if passing to
"_enable_interfaces" method the XML file name instead of passing the XML
string.

JIRA: YARDSTICK-1189

Change-Id: If021c102311ecb98cb26934f78067dfc720e4291
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit 72df0eaac4a2a1f23439af7f249c2b5d18d1b819)

Remove __init__ method overriding in HeatContextTestCase

Unit test case clases should implement setUp() method instead of overriding
__init__() method.

JIRA: YARDSTICK-1210

Change-Id: I0f9a10575f9aeebf76dfa84619e59589b3ea83f6
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit 619bed631f064a94ab20ac296802d467ebe32318)

Correctly set PYTHONPATH in Dockerfile

JIRA: YARDSTICK-1144

Change-Id: I8c0fe229ed620417a85356e017b171260373c43f
Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
(cherry picked from commit a114dc39f003beb1bb1ed12ed17fd3a6d0e7deaa)

Document for Fraser test case results

JIRA: YARDSTICK-1132

In B and C releases, Yardstick provides an overview of the results of test cases developed by Yardstick, executed on OPNFV community labs.

This work has not been done for the D and E releases, although result analysis for Euphrates is available at (https://wiki.opnfv.org/display/yardstick/Euphrates+release+test+results+analysis). We plan to restart to provide a basic test case results analysis for Fraser and compare performance data with Euphrates.

This JIRA ticket will be resolved into 3 commits: 1. Document for E release test case data; 2. Document for F release test case data; 3. Document for results analysis.

Change-Id: I3c59851c8878ba37145e6ad6958272a62bad3f52
Signed-off-by: JingLu5 <lvjing5@huawei.com>
(cherry picked from commit 3f0df920c71529024a69ad65c2bebabbcbde6022)

Convert SSH custom exceptions to Yardstick exceptions

JIRA: YARDSTICK-1215

Change-Id: I5ecfd3dccd91b07cd8de5309dfa1a372eff16ed0
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit dde8fa628483deec352e176cea8decdbf6bed7e8)

Add --hwlb options as a command line argument for SampleVNF

To enable HWLB queues in samplevnf on supported hardware
(e.g Intel  Fortville) --hwlb option needs to be passed
as a VNF command line argument.

JIRA: YARDSTICK-1159

Change-Id: I6e5c098dc71a711252b545c7622ee52085fa81f0
Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
(cherry picked from commit 1c8bfeb11a620c63ffd9fcd7f8735ded4521e077)

Do not start collectd twice when SampleVNF is running on Baremetal

JIRA: YARDSTICK-1167

Change-Id: I7591bbb2a84ee4039a20c5da2914f1e374299015
Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
(cherry picked from commit ba2ca2819a8b0312627742d126030c3dac15052f)

Cleanup BaseMonitor unit tests

Stop multiprocessing queues to avoid BrokenPipe exception.
Assert correctly the raise of exceptions.

JIRA: YARDSTICK-1229

Change-Id: I1a1c92c49a439283a35c9a69b59d6866f35ec09a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit 94d56c9f7cd99baea08629eaf3e580d020494383)

Add "host_name_separator" variable to Context class

This feature will provide Kubernetes context the ability to handle the
context name inside the class itself, providing to the developer an
abstraction of the possible naming limitations in Kubernetes. E.g.:
"dot" character in Pod names is no allowed [1].

[1] https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/issues/58

JIRA: YARDSTICK-1188

Change-Id: I82121f970b550170357a443b6340be7900602a57
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit a2ee61016721386b40caa4e2d72f61be8adec94a)

Remove print out of logger exception in TestUtils

Remove print out of logger exception while executing
"TestUtils.test_parse_ini_file_missing_section_header" test case.

JIRA: YARDSTICK-1225

Change-Id: I2a4e447c5d119fb9585b477d74f3d0a002c8278e
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit 99d51230daac3452f5295b4ea39890d8920e6ca3)

Cleanup TestYardstickNSCli class

Remove the console output during the test execution. Refactor the
"parse_arguments" test case.

JIRA: YARDSTICK-1224

Change-Id: I68954b986dbea70752adb121e0445e162bdfddcb
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit a8ddeef62ee1f9795881994380f49ce88a760102)

Bump oslo.messaging version to 5.36.0

A decoding error was introduced in the AMQP libraries installed by
oslo.messaging version 5.30.2.

Error example:

File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/
        .tox/functional/local/lib/python2.7/site-packages/amqp/utils.py",
         line 89, in str_to_bytes
  return s.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd2 in position 3: ordinal not in range(128)

This problem is fixed in the library versions installed in version 5.36.0.

JIRA: YARDSTICK-1204

Change-Id: I4719d8e0a0891ebbb3b87c31d593d15f897ef53a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry-picked from commit bcc841699c8072a74378774593729931e81b4398)

Set "cmd2" library to version 0.8.6

Last "cmd2" version, 0.9.0, removed Python 2+ support [1]. In order to
support Python 2.7, version 0.8.6 (or earlier) must be installed.

[1] https://github.com/python-cmd2/cmd2/commit/e54e5a5b891548df4df0acb04b6e607ad8f871b1

JIRA: YARDSTICK-1205

Change-Id: Ic2fe57438720de5b21c75f9456e653c6facf3ae9
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit c4d821c58858418ce8dfec95426f06ae11b25c7b)

Remove AnsibleCommon class method mock

In [1], several methods in "AnsibleCommon" class are mocked, but those
mocks are not removed after the test execution. Depending on the test
execution order, this affects other results.

[1] https://github.com/opnfv/yardstick/blob/stable/fraser/yardstick/tests/unit/service/test_environment.py#L20

JIRA: YARDSTICK-1214

Change-Id: I85ef702b3b5b2fda5cf453a21b9f0bec61b155f0
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
(cherry picked from commit e204c5063ab86d4ed2bff296959afae43866890f)

Move tests: unit/network_services/{lib/,collector/,*.py}

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

JIRA: YARDSTICK-837
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Change-Id: Idc2e70291b6d9b4cf7e7249a10c4a290b999206c
(cherry picked from commit 2e75e6d27240945ec985998bbe9ec6241d11626a)

Update pmd/lcore mask for OVS-DPDK context

Added default values for pmd/lcore mask to the SampleVNF test case for
Trex TG.
lcore mask - specifies the CPU cores used for non-datapath OVS-DPDK
threads.
pmd mask - specifies the CPU cores used for data path packet processing.
Updated documentation for Standalone OVS-DPDK context.

JIRA: YARDSTICK-1246

Change-Id: I8d008a185c94d2047ffa4769d7974bc4d283fd7f
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
(cherry picked from commit 779a868348c3361f2a3054f06002807fd69a6549)

Merge "Bugfix: openrc api dump should be safe_dump" into stable/fraser

Merge "[release] Yardstick release note 6.1.0" into stable/fraser

Merge "Get HA test case results on failure" into stable/fraser

Merge "Bugfix: Yardstick GUI build fails on Fraser 6.1.0" into stable/fraser

Merge "update docker image tag to 6.1.0" into stable/fraser

[release] Yardstick release note 6.1.0

JIRA: YARDSTICK-1182

Change-Id: I3baca52e9bc0360b43f2ce61ea2f166a70e8c90a
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
(cherry picked from commit 23d911a604f5686ade4fbbf80e0e046a2160dc8c)

Bugfix: Yardstick GUI build fails on Fraser 6.1.0

Yardstick web GUI build fails due to angular version

JIRA: YARDSTICK-1202

Change-Id: Ie075d18cb2eac2070267759978cbdb7c784599ef
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
(cherry picked from commit 81059ba8c7293a50aee72251afee89eb65547562)

update docker image tag to 6.1.0

Change-Id: Ia7a0668c1c2461fceb97425bcb6ade5bdb105c79
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Dockerfile: add ping for prepare_env.sh

INFO: Creating openstack credentials ..
Verifying connectivity to 10.20.0.2...
/home/opnfv/repos/yardstick//tests/ci/prepare_env.sh: line 56: ping: command not found
/home/opnfv/repos/yardstick//tests/ci/prepare_env.sh: line 56: ping: command not found

https://build.opnfv.org/ci/view/yardstick/job/yardstick-fuel-baremetal-daily-master/2400/console

JIRA: YARDSTICK-1201

Change-Id: Ib9e139046724ece4be7a85cc1ae74cbb0ec8e203
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Get HA test case results on failure

Now, yardstick HA test cases (which have 'sla action: monitor') do
not output results if the test case SLA validation fails.
This patch modifies the task runner and the serviceHA scenario so
that during an SLA validation failure the test result data is
captured correctly.

Do similar changes in the general scenario.

fix missing variable k
s/envrioment/environment

JIRA: YARDSTICK-1115

Change-Id: I648a8b229600c0ad089320ac3f803698f73aa800
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
(cherry picked from commit dd87e15c9c4f0aff3577c1bcd68bd86dd2d64898)
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Test case spec for SDN controller resilience in HA config

This is the test case specification for verifying the HA capabilities
of a SDN controller running in a HA configuration.

JIRA: HA-33
JIRA: YARDSTICK-955

Change-Id: I016ec1ef7bb51b79f8c21440abf8f04c923ef9dc
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>

Bugfix: openrc api dump should be safe_dump

1. fix safe_dump
2. fix pep8 problem, use flask_restapi custom error handling [1]

[1] https://flask-restful.readthedocs.io/en/latest/extending.html#custom-error-handlers

JIRA: YARDSTICK-1165

RestApi openrc dump clouds.yaml error

it now use yarml.dump, should be yaml.safe_dump.

dump would gererate !!python/unicode and cause error when upload openrc file in gui

Change-Id: Id3e85f7ba7d4967277ef79109b07d7552179e5db
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
(cherry picked from commit f1a71b2889da3b49358b10b9a12ea5a512dd2743)

Test case spec for SDN Virtual Switch resilience.

This is a test case for verifying the proper functionality of
SDN Vswitches which are located in the Compute nodes of an OpenStack
environment.
This test case is unaware of the configuration (non-HA,HA)

JIRA: HA-34
JIRA: YARDSTICK-955

Change-Id: I701bf1f9527b1a31b9cb52e102734e10c4162766
Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
(cherry picked from commit 4cc2882c73a64e243f98c0448ca2d1f9a677953c)

Merge "Update samples/test_suite.yaml" into stable/fraser

Merge "Fix log typos in benchmark/scenarios/availability" into stable/fraser

Merge "Remove unused methods in SampleVNF" into stable/fraser

Fix log typos in benchmark/scenarios/availability

This commit fixes multiple log message typos and all the emerged pylint
errors.

MonitorProcess/MonitorOpenstackCmd.verify_SLA() repeat already logged
information, so these prints are removed.

JIRA: YARDSTICK-1145

Change-Id: Ifef26e4b4ff7766089caec24785511969c2d663e
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
(cherry picked from commit 6fa51bdcc7ad7e4d407f2803fc05b1cfaf66c115)

Merge "Update the dummy-scenario-heat-context testcase" into stable/fraser

Merge "Document for Euphrates test case results" into stable/fraser

Document for Euphrates test case results

JIRA: YARDSTICK-1132

In B and C releases, Yardstick provides an overview of the results of test cases
developed by Yardstick, executed on OPNFV community labs.

This work has not been done for the D and E releases, although result analysis
for Euphrates is available at (https://wiki.opnfv.org/display/yardstick/Euphrates+release+test+results+analysis).
We plan to restart to provide a basic test case results analysis for Fraser and compare performance data with Euphrates.

This JIRA ticket will be resolved into 3 commits: 1. Document for E release test case data; 2. Document for F release test case data;
3. Document for results analysis.

Change-Id: I372a60807ac3dfaab87e70cdb956c1059b019f7a
Signed-off-by: JingLu5 <lvjing5@huawei.com>
(cherry picked from commit 4c20fb57684bc21e7ae0461341088100e518307d)

Update samples/test_suite.yaml

The sample test suite at sample/test_suite.yaml contained incorrect
syntax. It has been updated to add the "default" parameter to
"task_args" when running a testcase.

JIRA: YARDSTICK-1125
Change-Id: I53409168e47d969317145623df1e2c789396cc32
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
(cherry picked from commit 7b00d64494bdd61f2ab37005de76fa5f6496d45d)

Implement Virtual Switch resilience test case

This testcase tests the resilience of the Virtual
Switch in an OpenStack environment. It can only run in a
non Ubuntu distro because it is limited by this bug [0]

[0]: JIRA: YARDSTICK-1140

Change-Id: Ia1480fba9f3549d1144c80699056b501d3b07a0f
Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
(cherry picked from commit 53f5951a47fcaa5a3f8bcd6ea1020e54a1b285d3)

Bug Fix: Start openvswitch service in Centos distro

JIRA: YARDSTICK-1142

Change-Id: I82c479c95e10de37416081eacf2d8bee396e50dc
Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
(cherry picked from commit d00660e583ef89fe07149c7f0cffcced7cd72133)

Add support for restarting a service

JIRA: YARDSTICK-1138

Change-Id: Id57defb574eb7e0f5be81b52803cdb05f46796ab
Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
(cherry picked from commit 131b9f04a360ccc894510a275bd33c57634608fc)

Add ODL HA testcase

This testcase confirms the resiliency of Opendaylight
in a HA deployment

SPEC: https://gerrit.opnfv.org/gerrit/#/c/50991

JIRA: HA-33
JIRA: YARSTICK-955

Change-Id: I4cb39d1970ed0333a43fbb58f8c7138908379dcb
Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
(cherry picked from commit d16b2e14fe420a646d4399986bd9f4d1ae3a606d)

Add pod.yaml files for Apex

JIRA: YARDSTICK-1097

Change-Id: If116174c562bc08c0b7602324599bc85d8c60be0
Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
(cherry picked from commit 6f374141466640655cc57277b4b49d6deffa9092)

Update the dummy-scenario-heat-context testcase

The testcase is included in the docs to demonstrate definining a heat context.
It previously used cirros, but this is not included by default.
Set-up instructions/scripts add yardstick-image and yardstick-flavor, so these
have been added to the testcase instead.

A default value for context_name was also added.

Change-Id: I5f642086971487b334ae48a5cf5c9b71e9b99e2e
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
(cherry picked from commit f37e4dcf771caec3be52e95047997aa22cbd7a88)

Remove unused methods in SampleVNF

The following private methods are unused in
yardstick/network_services/vnf_generic/vnf/sample_vnf.py:SampleVNF

* _build_ports(self)
* _get_route_data(self, route_index, route_type)
* _get_port0localip6(self)
* _get_port1localip6(self)
* _get_port0prefixlen6(self)
* _get_port1prefixlen6(self)
* _get_port0gateway6(self)
* _get_port1gateway6(self)

They are being removed as:
* _get_port* functions are already implemented and used in MultiPortConfig
* _get_route_data is an auxillary functions used by the previous ones
* _build_ports is implemented in the helpers

JIRA: YARDSTICK-1122
Change-Id: Ie7a706cd87b610ea56f89214118a02e612a70dc0
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
(cherry picked from commit 50bcd3e477bc29abb43ed73643d551ff55a73403)

[release] Yardstick Fraser 6.0.0 Release notes

JIRA: YARDSTICK-1129

Change-Id: I5adb2e9547c58b90d3eafb0e2b90015ce5472cfa
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
(cherry picked from commit a324c1b922ae2598dbe25e9b2885620ef09f4901)

Bugfix: ansible can't not run without config

JIRA: YARDSTICK-1135

Ansible need ansible.cfg when run in the remote node.
However the file name is wrong. Need:
setup.cfg -> ansible.cfg

Change-Id: Ifa674fb624ca83d90c3ac9ad3bfe148013bf8bfd
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
(cherry picked from commit 170842f20426ad244f3e49b3780b0256302b0be3)

nsb_setup: pin docker image version to release tag

The install scripts pull tags for stable releases.

We need to temporarily pin the image tag to the unreleased tag
because we have to update the code before we actually tag the image.

Or maybe we should make this a parameter to the script

JIRA: YARDSTICK-1137

Change-Id: I2e5f13ffe19d1c19fd7b8e071df28fd225be3d79
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "extra loopback cleanup" into stable/fraser

Merge "[docs] Updated baremetal instalation instructions" into stable/fraser

Merge "Bugfix: Can't get image list in API" into stable/fraser

remove more old apexlake testcases

fix fraser failures

Change-Id: I8bb12a762054a4c3274cc6e0297e948b92700e7e
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

[docs] Updated baremetal instalation instructions

JIRA: YARDSTICK-1111

Change-Id: Iefab24f74c87a0855c2d70090ac34256ffa9200c
Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
(cherry picked from commit 1c4a5e82aec771aba616879a9a351bade6d6d8d0)

extra loopback cleanup

try to cleanup in the shell script in case ansible errors

maybe it is hopeless because of docker

JIRA: YARDSTICK-1096
JIRA: YARDSTICK-1054

Change-Id: I759ded4731a7710a194a81530cfb35376eb7800d
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

disable NSB (DPDK) image load for ARM

we skip the building now but we also need to skip the image loading

JIRA: YARDSTICK-1124

Change-Id: I27c40d6319e74374282203132cfc2c7ebff9b094
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
(cherry picked from commit 15cec319ab06f161c239ead78da9ecdf1673b7ce)

disable DPDK images for ARM

We are trying to compile DPDK for the os-nosdn-ovs-ha scenario because we added TC042  "Measure network latency using testpmd and pktgen-dpdk".

However it looks like TC042 is only enabled for compass in opnfv_os-nosdn-kvm_ovs_dpdk-ha

https://gerrit.opnfv.org/gerrit/#/c/54659/1/tests/opnfv/test_suites/opnfv_os-nosdn-kvm_ovs_dpdk-ha_daily.yaml

-
  file_name: opnfv_yardstick_tc042.yaml
  constraint:
      installer: compass
      pod: huawei-pod1

We don't actually need TC042 and DPDK or os-nosdn-ovs-ha.

But in order to run TC042 we have to know beforehand that we are going to use DPDK, so we have a conditional in the load_images.sh script that tries to guess when we will need a DPDK image.

The conditional only checks for *[_-]ovs[_-]*
if [[ $DEPLOY_SCENARIO == *[_-]ovs[_-]* ]]; then
    ansible-playbook \
             -e img_property="nsb" \
             -e YARD_IMG_ARCH=${YARD_IMG_ARCH} \
             -vvv -i inventory.ini build_yardstick_image.yml
    if [ ! -f "${QCOW_NSB_IMAGE}" ]; then
        echo "Failed building QCOW NSB image"
        exit 1
    fi
fi

For this release we won't be able to get DPDK images to compile on ARM, we should fix this conditional and not try to compile.

Also change the shell glob to match *[_-]ovs_dpdk[_-]* to only enable DPDK
images for ovs_dpdk scenarios

JIRA: YARDSTICK-1124

Change-Id: I03142e6b8aa00dc9cdf9b446868fb723cd10198b
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
(cherry picked from commit 50154694f8dc37726d456cfb8efddc6d3e126dbd)

remove apex lake tc007 from test suites

fixes opnfv_os-nosdn-fdio-noha_daily.yaml failures due
to missing testcase

Change-Id: I5f050d72334da46b94de5fa3662047bf694f7e3e
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
(cherry picked from commit 01ef85fc8234c8ddeaae65eceaf5bbb9545b9e98)

Merge "[userguide] Use literalinclude directive in 05-operation" into stable/fraser