KVMFORNFV: To avoid livemigration result update to json

This patch is used to remove the livemigration testcase result loaded
to json as there is no dashboard implemented for the testcase

Change-Id: I7a9589a0bbc5f2a28587c2878da042fc50af18e0
Signed-off-by: Navya Bathula <navyax.bathula@intel.com>

Removing trailing whitespace

Change-Id: I0380deec3e41a6dc02571ed9c4fa16e304f7f85e
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Add "How to work with test case" into devguide

JIRA: YARDSTICK-73

add a new chapter about how to work with test case, now just add three sub chapter,

sample test case, release test case, test case description file, will update more for

how to write a new test case.

Change-Id: I44906917e76ad6d4890f345c963b9e4440f6be39
Signed-off-by: kubi <jean.gaoliang@huawei.com>

Add vfw ixload testcase for heat

    - concurrency
    - connections/second
    - throughput

Change-Id: Idb977462952fba2e6a9684e916dfeca810f1ee94
Signed-off-by: Jennifer Li <jennifer.li@intel.com>

NSB: update vfw tc_heat_external tests

rename xe0, xe1 to uplink_0, downlink_0
enumerate tgs/vnfs starting from 0.
add disable_dhcp

Change-Id: Ia4f08b3ad5ab298192c249d04b10a6efbe4f210e
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

BugFix: Fix to pod filename ext and prox path

As per defect:
- the "context:" stanza of every tc_prox_baremetal*.yaml"
 file has an incorrect pathname for the config file

-  tc_prox_baremetal_binsearch.yaml and
tc_prox_baremetal_ramp.yaml refer to
"prox-baremetal.yml", which idoes not exist

- incorrect prox bin path

Resolution:
- Unify the pod file name and path for all BM tests
- Prox bin path changed to "/opt/nsb_bin/prox"

Change-Id: I16b25189104ad5585ddc1c2c0dd71b3c4c2aecd3
Signed-off-by: Abhijit Sinha <abhijit.sinha@intel.com>

Merge "Fix Malformed Table in User Guide docs"

Merge "Added NSB sample Test Case documentation."

Fix Malformed Table in User Guide docs

Change-Id: I2f98b75e9bbcd3d0cd4666c097a3280dd89eb02a
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>

Added NSB sample Test Case documentation.

Update:
Added Prox NSB test case documentation

Change-Id: Ia2f47d823581fc7f193d3123f080dd87f29d4b99
Signed-off-by: Maciej Skrocki <maciej.skrocki@intel.com>
Signed-off-by: Abhijit Sinha <abhijit.sinha@intel.com>

Docs: add yardstick gui descrption in userguide

Change-Id: I09bc55e296d2c641f10819972ec0d48a289432b8
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Merge "Docs: update new test cases in release note"

Docs: update new test cases in release note

Change-Id: Ie5ff6fe4f867ce68a33a3a857d8d051ba0020395
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Merge "nsb_installation: updates"

nsb_installation: updates

JIRA: YARDSTICK-500

Change-Id: I6eef884ef6262abe49fc13bc353ca14a72a5b648
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Martin Banszel <martinx.banszel@intel.com>
Signed-off-by: Abhijit Sinha <abhijit.sinha@intel.com>
Signed-off-by: Maciej Skrocki <maciej.skrocki@intel.com>

Merge "NSB: fix trex config to use dpdk port number"

Merge "BugFix: Negative dropped packets in Prox tests"

BugFix: Negative dropped packets in Prox tests

Removed the abs function which can potentially mask
negative dropped packets.

Dropped packets in Prox workload VNF = rx_packets - tx_packets

Change-Id: I510a351e899cdf9a1f366d632b9f0528b1d9dcce
Signed-off-by: Abhijit Sinha <abhijit.sinha@intel.com>

Merge "Adding sample trex example"

Merge "Add missing RFC2544 Ethernet frame sizes"

Merge "DRAFT: update release notes for Euphrates"

Bugfix: move opnfv_os-odl-fdio_dvr-noha_daily.yaml to 'test_suites' directory

The opnfv_os-odl-fdio_dvr-noha_daily.yaml test suite file is wrongly put in Yardstick root directory,
this patch moves this file to the 'test_suites' directory.

Change-Id: Ib3f75845c7af658dc6f96ac2de737cfbd1faf15a
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Merge "Docs: update new implemented APIs"

Docs: update new implemented APIs

Change-Id: If6b70791213099d53c82993bd22ac9eed6000c32
Signed-off-by: JingLu5 <lvjing5@huawei.com>

DRAFT: update release notes for Euphrates

Change-Id: I7e437612ca52b75b71e38962a84d0dcdb453d5dd
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml updated"

Merge "Create the ovs-vswitchd logging directory"

NSB: update vfw scale-out testcases

Change-Id: I6bd8a87f70954f288975e3e73e3cfbd36ae561dc
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "add CheckConnectivity scenario"

add CheckConnectivity scenario

Change-Id: I9d246828790467c2a57ba410826ee9751fff89c5
Signed-off-by: JingLu5 <lvjing5@huawei.com>

NSB: fix trex config to use dpdk port number

From some reason Heat/Neutron is now creating
interfaces such that xe0 and xe1 are swapped.

xe0  fa:16:3e:38:c7:66  0000:00:05.0
xe1  fa:16:3e:2f:f3:e2  0000:00:03.0

this causes the DPDK port numbering to be swapped.

xe0 is DPDK port 1 because it has higher PCI address
xe1 is DPDK port 0 because it has lower PCI address.

The VNF is configured correctly because it uses DPDK port numbers,
whereas TRex was using interface list ordering.

Modify trex_cfg.yaml to use DPDK port ordering.  This also
requires running generate_cfg() after setup() in instantiate()

+------------------------------------+-------------------+------------------------------+--------+
| Name                               | MAC Address       | Fixed IP Addresses           | Status |
+------------------------------------+-------------------+------------------------------+--------+
| vnf_0.yardstick-af5ccb47-xe0-port  | fa:16:3e:66:a5:e4 | ip_address='10.1.0.7',       | ACTIVE |
| vnf_0.yardstick-af5ccb47-mgmt-port | fa:16:3e:fa:98:fe | ip_address='10.0.1.10',      | ACTIVE |
| tg_0.yardstick-af5ccb47-xe1-port   | fa:16:3e:2f:f3:e2 | ip_address='10.1.1.9',       | ACTIVE |
| vnf_0.yardstick-af5ccb47-xe1-port  | fa:16:3e:f3:1d:f5 | ip_address='10.1.1.4',       | ACTIVE |
|                                    | fa:16:3e:e3:8c:65 | ip_address='10.0.1.1',       | ACTIVE |
|                                    | fa:16:3e:ff:d1:b7 | ip_address='11.191.14.110',  | N/A    |
| tg_0.yardstick-af5ccb47-xe0-port   | fa:16:3e:38:c7:66 | ip_address='10.1.0.8',       | ACTIVE |
|                                    | fa:16:3e:ff:53:5f | ip_address='11.191.14.101',  | DOWN   |
|                                    | fa:16:3e:23:5d:2c | ip_address='10.0.1.2',       | ACTIVE |
| tg_0.yardstick-af5ccb47-mgmt-port  | fa:16:3e:7a:df:4e | ip_address='10.0.1.5',       | ACTIVE |
|                                    | fa:16:3e:22:65:36 | ip_address='11.191.14.109',  | N/A    |
+------------------------------------+-------------------+------------------------------+--------+

/sys/devices/pci0000:00/0000:00:03.0/virtio0/net/ens3/address:fa:16:3e:2f:f3:e2
/sys/devices/pci0000:00/0000:00:05.0/virtio2/net/ens5/address:fa:16:3e:38:c7:66

Change-Id: Iaef2c7d9a5af7f45bd805a8ad6ee545ce0495cb1
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Add missing RFC2544 Ethernet frame sizes

RFC2544 tests should include the following frame sizes:

9.1 Frame sizes to be used on Ethernet

       64, 128, 256, 512, 1024, 1280, 1518

Change-Id: I6ab9cccf2bb3b2678f50da157a9a49c0d9564cbd
Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com>
Reviewed-by: Alain Jebara <alain.jebara@intel.com>
Reviewed-by: Deepak S <deepak.s@linux.intel.com>
Reviewed-by: Ross Brattain <ross.b.brattain@intel.com>
Reviewed-by: Edward MacGillivray <edward.s.macgillivray@intel.com>

Merge "add VNF scale-up template"

Merge "Resolve full path to adminrc file if full path is not provided"

Merge "Disable spoof check on vfs in sriov setup"

Merge "Adding cpu set to enable affinity for given vcpu"

Merge "Added required ubuntu packages to run IxLoad client"

Merge "pmu mirror creation ansible error fix"

Adding sample trex example

Change-Id: I3b79d33dba5cc559634122a135ff469f054214fc
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Merge "Docs: test case description for TC080 and TC081"

Merge "tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale_2: fix indent"

tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale_2: fix indent

bad YAML indent

Change-Id: I6aaf53b9bbf6005b6be4c2d6c69cefd41949f8c6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Hamonization of the titles of the documents (Testing group)

Change-Id: I6065fb416a90544539316666aac92735ee84177a
Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>

Docs: test case description for TC080 and TC081

JIRA: YARDSTICK-825

Change-Id: Ib2f8d0b88378c0dd0bd20ba38141a205fa7ae486
Signed-off-by: JingLu5 <lvjing5@huawei.com>

add VNF scale-up template

adjust worker_threads based on number of vcpus

Change-Id: Ie5444a64c15711a0e813905c69581f189432fe0b
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Create the ovs-vswitchd logging directory

Change-Id: Id7f55f7fcb2fec2866b76821595776908ac30c04
Signed-off-by: Martin Banszel <martinx.banszel@intel.com>

Merge "Fio: support input job file configuration"

tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml updated

Change-Id: Ie9cee3e0567e771cbf11a9802bd33a420cc1ee7a
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "scale-out testcase template"

scale-out testcase template

use anonymous flavors
force num_vnfs to int

added provider network examples

we have to disable port security for provider networks

removed vld_id, not needed

adjust num_vfs to allocate two physnets per VNF.

fix for multiports

Change-Id: Ic4c8314dc07f6295b9559188989d81058bb48eb0
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

add opnfv_os-odl-fdio_dvr-noha testsutie for Apex

Change-Id: Idd60153e6097cbef6513942653429d2ec96a8c17
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Resolve full path to adminrc file if full path is not provided

Change-Id: I7e9d8074b5d956c864a0b3264926eee89e66dc35
Signed-off-by: Charlie Kang <charlie.kang@intel.com>

NSB: MultiPortConfig adjust start_core

Change-Id: I8f4b6af898c9d3783f7b45471088a69e51c8baba
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Added required ubuntu packages to run IxLoad client

To fully enable IxLoad, after installing the IXIA client,
check /opt/ixia/ixload/<ver>/bin/ixloadpython and make sure
you can run this cmd inside the yardstick container. Usually
user is required to copy or link /opt/ixia/python/<ver>/bin/ixiapython
to /usr/bin/ixiapython<ver>.

Change-Id: I6bc633802ad6c70329581014ec89c75bb7f8d688
Signed-off-by: Maciej Skrocki <maciej.skrocki@intel.com>

Merge "validate_non_string_sequence: use six.string_types"

pmu mirror creation ansible error fix

  While creating the mirror for pmu event list generation, some of
  the links would fail, throwing error. This fixes that.

Change-Id: I52e96b1a6eacb5a9611845e815ecff9052df168a
Signed-off-by: Dhaval Patel <dhaval.r.patel@intel.com>

Merge "BugFix: Fix to Load Balancer - Previously Dropped packets was acceptable"

Merge "Enable static cgnapt functionality."

Enable static cgnapt functionality.

Change-Id: Ie770ca69ebdc66589ed6ca5c25bfc9a75afb8938
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

BugFix: Fix to Load Balancer - Previously Dropped packets was acceptable

Dropped packets were tolerated. Now no longer tolerated (except 0.01%)

Change-Id: I69d4f1a84d82e03a091a61419fa05fea13176c1c
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>

Merge "BugFix: Fix to negative dropped packets appearing in results."

Merge "Update the traffic profile to support sample multi-port"

Merge "BugFix: limit the count to supported range in min/max"

Merge "fix invalid chracters in ansible scripts"

Merge "Adding sample multi-port ansible script"

NSB: rename private->uplink, public->downlink in example

Change-Id: Ieb8943a432a59375b7e68a708666ad4caa4affd2
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

validate_non_string_sequence: use six.string_types

Change-Id: I7248d97a55974091daf20f0fba4b612101a0e4c5
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Disable spoof check on vfs in sriov setup

Change-Id: I8577231c2e78776cb30f7f7ded353a4fbc585aaa
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Adding cpu set to enable affinity for given vcpu

Change-Id: If2e079966939b7faa33d2833d81caad0a3669036
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Fio: support input job file configuration

Change-Id: I4bb15a398819e54f7fb3e9eac17daa4059c03376
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Merge "bugfix: remove pod_name in host and unify host parameter"

Adding sample multi-port ansible script

- trex
- ixia
- ixia correlated

Change-Id: I7d1414227c27c2bb03454eac94592e07c434f070
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

fix invalid chracters in ansible scripts

Change-Id: I3d1a491e4c0f0554a861d9bb24fbd0ac94835759
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Create test suite opnfv_os-nosdn-openbaton-ha_daily

JIRA: YARDSTICK-823

We need to create test sutie opnfv_os-nosdn-openbaton-ha_daily.yaml
to enable this scenario.

Change-Id: Ia800a9c74aaecf0d73e6f0fd559630b741ad2432
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Update the traffic profile to support sample multi-port

Change-Id: Iaf8b3e0aa3855443b0628573a1df98d177f27c76
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Merge "Relocate iniparser to yardstick.network_services.helpers"

resource: move make_resource_profile to ResourceProfile

move factory method from Collector to ResourceProfile

Change-Id: I1ddf3cffed7a3b2d45aa9cbffd3c0a5925c325d6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Relocate iniparser to yardstick.network_services.helpers

Change-Id: I1f457c9c24f2ca84dde61b64f58edaff8952670a
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>

BugFix: Fix to negative dropped packets appearing in results.

As per defect:
   the "VNF packets dropped" time series is negative - negative number of packets dropped
   is not meaningful.

Solution:
   Return a positive value.

Change-Id: I358006a691f2d6b5954333b9f7a745aba97ae44f
Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>

Merge "task: use CONF_FILE from constants"

Merge "Move arp route tbl to script and update defailt vnf config files"

Merge "ping: always save rtt data, influxdb ignore empty data"

Merge "Updating NSBperf to print right testcase"

task: use CONF_FILE from constants

I'm not sure why we hardcoded config_file when
the CONF_FILE constant will work.

Change-Id: I45a1dc17ac2b185c659725f55462214acb74838d
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

ping: always save rtt data, influxdb ignore empty data

If the SLA was failing we were raising AssertionError and not storing
the rtt in the data dict.  This caused influxdb parse errors because the
data was empty.

Fixup influxdb to ignore records with no data, so we don't try to parse
no data.

Change the ping logic to always record the rtt result even if the SLA
was not met.

Also fixup ping logic in cases where ping does not return results.  If
SLA is defined use SLA * 10 otherwise use large float that doesn't break
the grafana scale too much, maybe 999999

JIRA: YARDSTICK-809

Change-Id: Id2d51216581644a80e8c7b9aa98919a766008adf
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "task: drain background runner queues"

Merge "KVMFORNFV: To verify the execution of the bash script"

bugfix: remove pod_name in host and unify host parameter

for none-ha test case, use host to be parameter
for ha test case, use attack_host as parameter

Change-Id: I053df00adab17814dd54584a1ae081548163dc68
Signed-off-by: rexlee8776 <limingjiang@huawei.com>

Merge "Bugfix: Add opnfv_os-nosdn-ovs_dpdk testsuite for Compass4nfv"

Bugfix: Add opnfv_os-nosdn-ovs_dpdk testsuite for Compass4nfv

Change-Id: I35e6471bb561eb773bd188b36b912969188955d2
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Bugfix: Add opnfv_os-odl_l2-moon-ha_daily testsuite for Compass4nfv

Change-Id: I46c6b56e1293c3d594804d77a73c391922f03653
Signed-off-by: JingLu5 <lvjing5@huawei.com>

BugFix: limit the count to supported range in min/max

In trex, giving count beyond the allowed range causes trex client to
fail. Check the range and reset the count before starting the traffic

Change-Id: I58e54f47edb5f249087b8f7178b807ddfb13b03c
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Support to add multiple src/dst ports in flow

Change-Id: I888a6029efaad8308de8df6b6787dd46af5f1c55
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

Merge "increase number of open files for collectd"

collectd: set intel_pmu events path and OVS socket path

switch from hardcoded path to dynamic path
based on bin_path

also enable proxy for install_collectd

add barometer settings for virt and ovs_stats

Change-Id: Id138aef548332a3e3fcb3963b746e7c9f10c0948
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "NSB Prox Grafana dashboard updates"

Version changes for collectd, pmu-tools, intel_cmt_cat

  Bumping collectd version to include pmu plugin enablement,
  bounding pmu-tools and intel_cmt_cat to a commit-id

Change-Id: I666d52c5574cec0caebb07535332984fa77e5b65
Signed-off-by: Dhaval Patel <dhaval.r.patel@intel.com>

NSB Prox Grafana dashboard updates

JIRA: YARDSTICK-802

Updating the Grafana dashboards with common data
format for all BM and Opensatck Prox tests and
adding minor improvements

- Correct the test description
- Re-run the tests and verify that all the
  fields are graph correct
- Label the axes correctly.
  eg: For Throughput added label "MPPS"

Change-Id: I62b471926fe3c4c8876b70053a92d06f4373e296
Signed-off-by: Abhijit Sinha <abhijit.sinha@intel.com>

KVMFORNFV: To verify the execution of the bash script

This patch is used to verify the parameters passed and
the flow of qemu_migrate_benchmark.bash script to handle
the bugs

Change-Id: Ifc1bfa7a91a2eed178b9dc984c222074f06e0599
Signed-off-by: Navya Bathula <navyax.bathula@intel.com>

Change 'SUCCESS' to 'PASS' when report to MongoDB

JIRA: YARDSTICK-822

To keep consistent with other testing project, we need to
change 'SUCCESS' to 'PASS', 'FAILED' to 'FAIL'.

Change-Id: I392e4dc6dd2e028cd43cba0d1bf8ee04618326d7
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>