Merge "Refactor remote command execution in pktgen_dpdk"

Extended Context class with get_physical_nodes functionality

JIRA: YARDSTICK-1255

Change-Id: I446d715dc6cf716a4fcbc1b809c1b1d6303071e0
Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>

Merge "Add unit test file for DurationRunner"

Merge "Refactor remote command execution in pktgen"

NSB Topology fix for Prox 4 port test case

The uplink and downlink interfaces for the 4 port
test cases were inconsistent and fixed in this commit.

This affects all the 4 port PROX test cases.

Tested on: Heat L2fwd-4 and L3fwd-4 test case.

JIRA: YARDSTICK-1185

Change-Id: Ia2ce7ed0dc395b812f84ede94e259b42dc833579
Signed-off-by:Abhijit Sinha <abhijit.sinha@intel.com>

Bugfix: HA kill process recovery has a conflict

It happens in Nokia SUT when running in the Plugfest. The problem happens when
the start_process start to recover the killed process (like nova-api), but the
self-cured mechanism already recovered. And somehow it lead to a conflict and
has problems.
So the recover of the HA attack-recover should be improved to only recover it
when it needs to.

JIRA: YARDSTICK-1222

Change-Id: I1acb5a7d59d6fe4e0de0b0c5942fa89e051dd1ff
Signed-off-by: rexlee8776 <limingjiang@huawei.com>

Add unit test file for DurationRunner

JIRA: YARDSTICK-1199

Change-Id: I5526935207ae3661a4cb8b8d25e93bc26adc3ef2
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>

Merge "Cleanup CGNAPT unit tests"

Merge "Add "volumeMounts" parameter in Kubernetes context"

Merge "Add "volumes" parameter in Kubernetes context"

Change IxNextgen library load

Changed IxNextgen library load in IXIA RFC2544 traffic generator call.

JIRA: YARDSTICK-1116

Change-Id: Icbdc5f1255dd2676584f3a1be767287e55e71f4a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Merge "Configure ACL via static file"

Configure ACL via static file

This patch allows user to configure ACL/vFW SampleVNF ACL
via configuration file provided in TC definition. The
Yardstick applies the rules to SampleVNF from specified config
file + rules generated by Yardstick (default rules).

The example of SampleVNF ACL CLI commands generated/applied
by Yardstick can be found at (using default ACL config file):

ACL VNF ACL CLI commands:
  acl/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
  http://paste.openstack.org/show/723303/

vFW VNF ACL CLI commands:
  vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
  http://paste.openstack.org/show/723304/

Change-Id: I76a630261a982083b628e3985fc3bec14ca495db
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>

Merge "Cleanup EnvCommand test cases"

Merge "Add scale out TCs with availability zone support"

Fix pip install issues in install.yaml script

For long executing commands, ansible should use async jobs. For example,
when installing the project requirements.

There is a problem in ansible pip module when using a "editable installs".
This command should be called with "editable" flag.

JIRA: YARDSTICK-1250

Change-Id: I6d29bab2f388373e252d065ff5aa0c459a8f265a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

DPDK site change

DPDK site changed and the download URL must be changed.

New site: http://core.dpdk.org/download/

JIRA: YARDSTICK-1248

Change-Id: I87359a4417f09c86717f598c76dc67c422bd39f7
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Bump ansible version to 2.5.5

Increase ansible version to 2.5.5.

There is a closed bug present in version 2.4.2 which affects to some ansible
scripts. When calling recursively different roles, the maximum recursion
depth can be exceeded [1]. This bug is fixed in proposed version.

[1] https://github.com/ansible/ansible/issues/23609

JIRA: YARDSTICK-1249

Change-Id: Ia45b6a2ab44203e39c52357eb3cfee1e0c77eed3
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Merge "Document for Fraser test case results"

NSB setup inventory name changed

NSB setup script [1] fails when retrieving the inventory file. This script
is going to be deprecated but we still support it.

[1] https://github.com/opnfv/yardstick/blob/master/nsb_setup.sh

JIRA: YARDSTICK-1247

Change-Id: I3798531652b0e3a1a007d1d9434998fdd54645d7
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Merge "Refactor RFC2455 TRex traffic profile injection"

Merge "Add "duration" parameter to test case definition"

Merge "Update pmd/lcore mask for OVS-DPDK context"

Refactor remote command execution in pktgen_dpdk

PktgenDPDKLatency.run will raise a RuntimeError if a remote command
fails. This commit makes it use the already existing exception raising
mechanism in yardstick's ssh client.

JIRA: YARDSTICK-1166

Change-Id: I3a3c7691399044b174f5d040c015c9b907b2fe5d
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>

Refactor remote command execution in pktgen

Some remote commands in Pktgen are executed without checking the exit
status. This patch makes all remote commands check exit value, and
removes unused variables that are captured from remote command
executions.

JIRA: YARDSTICK-1166

Change-Id: I42a667ebd22d086887d61e1671bc569b03c59d33
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>

Update pmd/lcore mask for OVS-DPDK context

Added default values for pmd/lcore mask to the SampleVNF test case for
Trex TG.
lcore mask - specifies the CPU cores used for non-datapath OVS-DPDK
threads.
pmd mask - specifies the CPU cores used for data path packet processing.
Updated documentation for Standalone OVS-DPDK context.

JIRA: YARDSTICK-1246

Change-Id: I8d008a185c94d2047ffa4769d7974bc4d283fd7f
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>

Cleanup EnvCommand test cases

Refactor exception testing and remove console output.

JIRA: YARDSTICK-1235

Change-Id: I6376c3637ffb6476a2578ef963cd2420b63bab52
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Refactor RFC2455 TRex traffic profile injection

The TRex RFC2455 traffic profile configures the duration of the traffic
injection. Once the traffic is started, the traffic profile should poll
the traffic injector client for the injection status. When the injection
is finished, the traffic execution should end.

Instead of this, the traffic profile waits a fixed time (using time.sleep
method) and then stops the traffic. This approach is not accurate and may
cause shorter injection periods.

JIRA: YARDSTICK-1197

Change-Id: I5090df81ef4ec7945ff6c1aff070656b48e0fe77
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Add "duration" parameter to test case definition

Add "duration" parameter to test case definition, in scenario:options
section. This parameter will be rendered in the traffic profile.

If the parameter is not present in the test case scenario options, the
default time written in the traffic profile options will be 30 seconds
(TrafficProfile.DEFAULT_DURATION = 30). If the traffic profile
injection time is not defined, the default injection time will be 30
seconds.

  testcase:scenario:options:duration (default = 30)
      render --> traffic_profile:duration
          parse --> TrafficProfile.duration (default = 30)

Target traffic profiles (RFC2544):
  - RFC2544Profile
  - IXIARFC2544Profile

JIRA: YARDSTICK-1194

Change-Id: I968922e6bb882d7ee15aa1c4db4037face7a3492
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Merge changes from topic 'convert-sla-asserts-to-raises'

* changes:
  Replace MagicMocks with Mocks from SearchRunner UTs
  Add vsperf_dpdk SLA validation unit test cases
  Cleanup vsperf unit tests

Merge "Convert SLA asserts to raises"

Stop running tests from tests/unit

Since all tests have been moved into yardstick/tests,
temperary code to run tests in both tests and yardstick/tests
should be removed.

JIRA: YARDSTICK-837
Change-Id: I3bf3c9ece9bc8457d992e53ee8cc5cadb1dec8f0
Signed-off-by: Emma Foley <emma.l.foley@intel.com>

Merge "Cleanup BaseMonitor unit tests"

Cleanup BaseMonitor unit tests

Stop multiprocessing queues to avoid BrokenPipe exception.
Assert correctly the raise of exceptions.

JIRA: YARDSTICK-1229

Change-Id: I1a1c92c49a439283a35c9a69b59d6866f35ec09a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Replace MagicMocks with Mocks from SearchRunner UTs

This patch replaces unneeded MagicMocks with Mocks from TestSearchRunner
and TestSearchRunnerHelper.

JIRA: YARDSTICK-1169

Change-Id: I30a7bbc71c0b1062e6c0dfe634a5a8f1ecb7c684
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>

Convert SLA asserts to raises

This commit converts Python assertions to a custom exception in all
places where SLA validation is checked with an assertion.

This commit also fixes all emerged pylint errors.

JIRA: YARDSTICK-966

Change-Id: If771ed03b2cbc0a43a57fcfb9293f18740b3ff80
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>

Add vsperf_dpdk SLA validation unit test cases

JIRA: YARDSTICK-1228

Change-Id: Ic32f2bcd7f8bdf718c9c266666409d32ecab4924
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>

Cleanup vsperf unit tests

Remove unneeded mock return values and improve the test execution.

Add unit test cases about SLA validation.

JIRA: YARDSTICK-1227

Change-Id: I90b8b9c4cb6a57ae4b0869e84af0da43ee22a670
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>

Merge "assert[Greater,Equal] -> assert_{,not_}called"

Bugfix: heat conext of test case yamls jinja2 bypass sriov type

the "if provider" condition bypass when provider = "sriov",
related to https://gerrit.opnfv.org/gerrit/#/c/32789/

  networks:
    test:
      cidr: '10.0.1.0/24'
      {% if provider == "vlan" %}
      provider: {{provider}}
      physical_network: {{physical_network}}
        {% if segmentation_id %}
      segmentation_id: {{segmentation_id}}
        {% endif %}
      {% endif %}

JIRA: YARDSTICK-1242

Change-Id: I6a611e839654f401648d9bd6188589816f7b4946
Signed-off-by: rexlee8776 <limingjiang@huawei.com>

Merge "Add "host_name_separator" variable to Context class"

Merge "Install dependencies: bare-metal, standalone"

Install dependencies: bare-metal, standalone

This will install all dependencies needed on the bare-metal and standalone
physical nodes.

JIRA: YARDSTICK-1177

Change-Id: I719ca64d4da098d6dd60597e5fc4ad6f4715f79e
Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>

Stop time.sleep mocks in unit tests

tVpeApproxVnf and TestProxSocketHelper classes mock "time.sleep" in
the setUp method without deleting it at the end of the test execution.

JIRA: YARDSTICK-1243

Change-Id: Iff31d9c7b400ad8a47f37792aeb0d20328b9d9e1
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Add send socket commands function

Pktgen provides a TCP socket connection to allow the user to control
it from a remote console or program [1].

This new method will provide Yardstick the ability to send string
commands to a port in a remote host.

[1] http://pktgen-dpdk.readthedocs.io/en/latest/socket.html

JIRA: YARDSTICK-1186

Change-Id: I9d64ccad662fa3599de65654c5dab02833fcc91d
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Create user defined "NodePort" services per pod

This feature will give the user the ability to create user defined "NodePort"
services per pod, a part from the default SSH port created, by default, for
each pod created.

Example of Kubernetes yaml definition:
  apiVersion: v1
  kind: Servicemeta
  data:
    name: pod-1-service-nodeport
  spec:
    type: NodePort
    ports:
      - name: web
        port: 80
        targetPort: 8888
        nodePort: 33333
    clusterIP: 10.254.0.8
    selector:
      app: pod-1

Example of Yardstick test case definition:
  context:
    type: Kubernetes
    servers:
      host:
        containers:
          - image: ...
            securityContext:
              allowPrivilegeEscalation: false
          - image: ...
        node_ports:
          - port: <port number>         # Mandatory
            name: <port name>           # Optional
            targetPort: <port name>     # Optional, default: targetPort=port
            nodePort: <port number>     # Optional, assigned by Kubernetes

Kubernetes service, type "NodePort" [1]

[1] https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport

JIRA: YARDSTICK-1181

Change-Id: Ia7730e4569d5526d573402a98c27d55c5c82cdef
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Change "KubernetesObject" class name to "ReplicationController"

The class name "KubernetesObject" is not accurate. This class holds
the definition of a Kubernetes replication controller. The suggested name
is therefore "ReplicationControllerObject".

JIRA: YARDSTICK-1180

Change-Id: I9ee9228f9528cb779e54a9bc178760b6e24c85c2
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Specify the networks to be used per pod

If CRD "Network" is defined and network items are created, each pod (server)
can have access to one or several networks. This is defined in the metadata
section, as "annotations.networks" [1].

Example of Kubernetes pod definition with networks:
  apiVersion: v1
  kind: Pod
  metadata:
    name: test-pod
    annotations:
      networks: '[{"name": "flannel"}]'

Example of Yardstick server definition with networks:
  context:
    type: Kubernetes
    servers:
      host:
        containers:
          - name: ...
        networks:
          - flannel  # These names must be defined in
                     # context.networks
    ...
    networks:
      - name: flannel
        plugin: flannel

Kubernetes annotations [2].

[1]https://github.com/intel/multus-cni/tree/b9446232cdf4f1b6f2bea583291973cc97e963f4#configuring-multus-to-use-kubeconfig-and-a-default-network
[2]https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/

JIRA: YARDSTICK-1178

Change-Id: I6e7b4bacf10810833ec733c14d44e5db613675e3
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Add new Kubernetes resource kind: "Network"

Add a new Kubernetes resource kind: "Network" [1] [2]
Kubernetes network plugins (alpha definition): [3]

Network definition example in Kubernetes:
  apiVersion: "kubernetes.com/v1"
  kind: Network
  metadata:
    name: flannel
  plugin: flannel
  args: '[
    {delegate": {"isDefaultGateway": true}}
  ]'

Proposed Kubernetes context network definition:
  context:
    networks:
      - name: flannel
        plugin: flannel
        args: (string)
      - name: sriov_upload
        plugin: sriov
        args: (string)

[1]https://builders.intel.com/docs/networkbuilders/multiple-network-interfaces-in-kubernetes-application-note.pdf
[2]http://cdn.opensourcecloud.cn/zt/k8s/01.pdf
[3]https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/

JIRA: YARDSTICK-1160

Change-Id: I71a49ac14e3d28ded91d2ed3cd9cc527e40303f7
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Add new Kubernetes resource kind: "CustomResourceDefinition"

Custom resource definition example in Kubernetes:
  apiVersion: "apiextensions.k8s.io/v1beta"
  kind: CustomResourceDefinition
  metadata:
    name: networks.kubernetes.com
  spec:
    group: kubernetes.com
    version: v1
    scope: Namespaced
    names:
      plural: networks
      singular: network
      kind: Network

Proposed Kubernetes context network definition:
  context:
    custom_resources:
      - name: network        # name of the resource (singular)
        version: v1          # optional, "v1" by default
        scope: Namespaced    # optional, "Namespaced" by default

From this definition, we will extract the Kubernetes parameters:
  - metadata.name: custom_resources.name + "s" + context_name + ".com"
  - spec.group: context_name + ".com"
  - spec.scope: custom_resources.scope
  - spec.version: custom_resources.version
  - spec.names.plural: custom_resources.name + "s"
  - spec.names.singular: custom_resources.name
  - spec.names.kind: custom_resources.name with first capital letter

[1] https://kubernetes.io/docs/concepts/api-extension/custom-resources/

JIRA: YARDSTICK-1163

Change-Id: If8980dc3f6ddf9c6949bf15be8011aa98482ddc9
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Add "securityContext" parameter in Kubernetes context

This new parameter, "securityContext", will allow the user to define the
privilege and access control settings for a pod or a container [1].

Example of "securityContext" definition in the pod (if only one container
is defined):

  context:
    type: Kubernetes
    servers:
      host:
        image: ...
        securityContext:
          runAsUser: 1000
          fsGroup: 2000 

Example of "securityContext" definition in the pod and the container (if
several containers are defined):     

  context:
    type: Kubernetes
    servers:
      host:
        securityContext:
          runAsUser: 1000
          fsGroup: 2000 
        containers:
          - image: ...
            securityContext:
              allowPrivilegeEscalation: false
          - image: ...

[1] https://kubernetes.io/docs/tasks/configure-pod-container/security-context/

JIRA: YARDSTICK-1156

Change-Id: I597a300c68cd834522a284b1cca0faa918493342
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Define several containers per pod in a replication controller

Add the ability to define not only one but many containers per pod in a
replication controller descriptor. This feature must be backwards
compatible; all current test cases using the "single container" server
definition must be accepted.

Example of single container pod definition:

  context:
    type: Kubernetes
    servers:
      host:
        image: ...
        commands: ...
        volumes:
          - name: volume1              # mandatory
            <volume type definition>   # mandatory

Example of several container pod definition:

  context:
    type: Kubernetes
    servers:
      host:
        containers:  # if this key is present, all container specific
                       parameters (image, commands, args, volumeMounts, etc.)
                       must be defined per container
          - image: ...
            commands: ...
          - image: ...
            commands: ...
        volumes:
          - name: volume1              # mandatory
            <volume type definition>   # mandatory

NOTE: other parameters, like "volumes" or "nodeSelector", are common to all
containers in the pod.

JIRA: YARDSTICK-1155

Change-Id: Ib95668c68e9c09e6de3f1aa41c903cc52e6809ad
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Add "volumeMounts" parameter in Kubernetes context

This new parameter, "volumeMounts", will allow the user to automatically
create new volumes mounted inside the container. Example of Kubernetes
context definition:

  context:
    type: Kubernetes
    servers:
      host:
        volumeMounts:
          - name: volume1              # mandatory
            mountPath: /dev/hugepages  # mandatory
            readOnly: True             # optional, default=False

JIRA: YARDSTICK-1151

Change-Id: Ic00b45e6d603a9f85e3f0e25becdf0bce14e637c
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Add "volumes" parameter in Kubernetes context

This new parameter, "volumes", will allow the user to automatically create
new volumes. Example of Kubernetes context definition:

  context:
    type: Kubernetes
    servers:
      host:
        image: ...
        commands: ...
        volumes:
          - name: volume1              # mandatory
            <volume type definition>   # mandatory

The volume type and the definition must be one of the supported ones in
Kubernetes [1].

[1] https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes

JIRA: YARDSTICK-1152

Change-Id: I44a91c605f047de4f286407e28fb5aa2e921b00a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

assert[Greater,Equal] -> assert_{,not_}called

assertEqual(mock_xxx.call_count, 1) -> mock_xxx.assert_called_once
assertEqual(mock_xxx.call_count, 0) -> mock_xxx.assert_not_called
assertGreater(mock.call_count, 0) -> mock.assert_called()
assertGreaterEqual(mock.call_count, 1) -> mock.assert_called()

JIRA: YARDSTICK-1069
Change-Id: I890084d120c8e78304e169e2a0e5d30011a41525
Signed-off-by: Emma Foley <emma.l.foley@intel.com>

Improve IXIA IxNetwork library and traffic profile (4)

This patch implements an active wait for the traffic injection. Once the
traffic is started, the traffic generator class will poll periodically the
IXIA traffic generator chassis to retrieve the status of the traffic
("started", "stopped").

Now the latency statistics are retrieved and reported for each injection
period.

JIRA: YARDSTICK-1116

Change-Id: I4422e2c88b4fc97b7cac3de8a82b2d75467c4117
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>

Improve IXIA IxNetwork library and traffic profile (3)

This patch modifies IP packet parameters.

"IxNextgen.update_ip_packet" modifies the L3 packet according to the
test case and setup the IP addresses.

JIRA: YARDSTICK-1116

Change-Id: I46ff75ab1989d0e6f5cc876418a015386717e06f
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>

Improve IXIA IxNetwork library and traffic profile (2)

This patch modifies the way the packet frame and the flow is configured
before the injection.

"IxNextgen.update_frame" modifies the L2 frame according to the
test case and setup the frame rate, frame size, traffic injection
duration and MAC addresses.

JIRA: YARDSTICK-1116

Change-Id: Ife08f15a4eda24d7835c92c4172b450854d112ee
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>

Improve IXIA IxNetwork library and traffic profile (1)

This patch introduces a new way to configure the TCL IxNetwork server.
All the configuration is done using the TCL API, removing the need of
using the pre-saved configuration file.

"IxNextgen.assign_ports" creates and assigns the virtual ports for each
physical port defined in the test case.

"IxNextgen.create_traffic_item" creates one traffic item and two flow
groups per pair of ports, in both directions.

"IxNextgen.gt_statistics" retrieves only the required statistics to
generate the samples blob in the traffic generator.

JIRA: YARDSTICK-1116

Change-Id: I8f1c0c55e99c274b2ed8276ed9a385c502e16d93
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>

Merge changes from topics 'YARDSTICK-1218', 'YARDSTICK-1216', 'YARDSTICK-1215', 'YARDSTICK-1214'

* changes:
  Move IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptions
  Move ErrorClass definition to exceptions module
  Convert SSH custom exceptions to Yardstick exceptions
  Remove AnsibleCommon class method mock

Cleanup CGNAPT unit tests

Remove unneeded mocks, clean console output and improve the
test execution.

JIRA: YARDSTICK-1226

Change-Id: I83e91028c17208a39d3813e85b44c4df1961e01e
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Cleanup process test cases

Remove logging console output during test execution.

JIRA: YARDSTICK-1234

Change-Id: I8ecab0219ffd1919f2fa97a90d5be11018b11322
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Cleanup OpenStack utils test cases

Remove logging console output during test execution.

JIRA: YARDSTICK-1230

Change-Id: I1b61f0f18bfb6e5a63c9027537cfb736261c4926
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Move tests: unit/network_services/vnf_generic

* Fix pylint errors
* Add TODOs

Some errors are ignored locally, as they were a symptom of other problems.
These issues have been flagged with a TODO, and should be fixed later.

JIRA: YARDSTICK-837

Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Change-Id: I3d95537a64bde923703421510d27bac0d6dd92bd

Refactor "utils.parse_ini_file" testing

JIRA: YARDSTICK-1218

Change-Id: I5136b22110d215beb7af72998ad47a8dba27f63b
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Merge "Remove print out of logger exception in TestUtils"

Merge "Cleanup TestYardstickNSCli class"

Merge "Remove __init__ method overriding in HeatContextTestCase"

Merge "Standalone XML machine type is not longer valid"

Merge "Error when adding SR-IOV interfaces in SR-IOV context"

Remove print out of logger exception in TestUtils

Remove print out of logger exception while executing
"TestUtils.test_parse_ini_file_missing_section_header" test case.

JIRA: YARDSTICK-1225

Change-Id: I2a4e447c5d119fb9585b477d74f3d0a002c8278e
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Cleanup TestYardstickNSCli class

Remove the console output during the test execution. Refactor the
"parse_arguments" test case.

JIRA: YARDSTICK-1224

Change-Id: I68954b986dbea70752adb121e0445e162bdfddcb
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Merge "Fix latency in l2 and l3 PROX generator config files"

Merge "Use TRex release v2.41 to support both x86 and aarch64"

Merge "Remove instantiated contexts in "test_task""

Fix latency in l2 and l3 PROX generator config files

JIRA: YARDSTICK-1211

Change-Id: I0899d470ca5a0ec7d42d36a9ff9c39cc7369310c
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>

Merge "Add NSB PROX Baremetal dashboard for L3"

Merge "Fix NSB NFVi metrics accuracy"

Merge "Bump oslo.messaging version to 5.36.0"

Fix NSB NFVi metrics accuracy

JIRA: YARDSTICK-1212

Change-Id: Ic4d4a3c00e4e278f4de06cc176ff663892895569
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>

Move IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptions

JIRA: YARDSTICK-1218

Change-Id: I804065e9bce3e728f5bf9e756a78df8fd28f74ac
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Move ErrorClass definition to exceptions module

JIRA: YARDSTICK-1216

Change-Id: I82556e1d1b0c723221a58e188067cbce560b8338
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Convert SSH custom exceptions to Yardstick exceptions

JIRA: YARDSTICK-1215

Change-Id: I5ecfd3dccd91b07cd8de5309dfa1a372eff16ed0
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Remove AnsibleCommon class method mock

In [1], several methods in "AnsibleCommon" class are mocked, but those
mocks are not removed after the test execution. Depending on the test
execution order, this affects other results.

[1] https://github.com/opnfv/yardstick/blob/stable/fraser/yardstick/tests/unit/service/test_environment.py#L20

JIRA: YARDSTICK-1214

Change-Id: I85ef702b3b5b2fda5cf453a21b9f0bec61b155f0
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Add NSB PROX Baremetal dashboard for L3

JIRA: YARDSTICK-1213

Change-Id: Id96edc5d23d3530110466e1cf69327c3e267e4c0
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>

Remove __init__ method overriding in HeatContextTestCase

Unit test case clases should implement setUp() method instead of overriding
__init__() method.

JIRA: YARDSTICK-1210

Change-Id: I0f9a10575f9aeebf76dfa84619e59589b3ea83f6
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Remove instantiated contexts in "test_task"

During unit testing, all contexts instantiated must be deleted at the end
of the test to avoid clashes with other tests.

JIRA: YARDSTICK-1209

Change-Id: I1994b915001b4a518e096833a0c5d33febbd8de6
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Replace glance delete image with shade client.

Remove get_image_id function.
Function delete_image now uses shade client.

JIRA: YARDSTICK-892

Change-Id: I6e8510dfa49aa14786ed7ac3382b85c4e699fb9e
Signed-off-by: Shobhi Jain <shobhi.jain@intel.com>

Merge "Replace glance create image with shade client."

Merge "Set "cmd2" library to version 0.8.6"

Replace glance create image with shade client.

Function create_image now uses shade client.

JIRA: YARDSTICK-892

Change-Id: Ia41d9ce702a1f24031080f8a365c1b2bd9ac9faa
Signed-off-by: Shobhi Jain <shobhi.jain@intel.com>

PTL change from Ross Brattain to Mingjiang Li (Rex)

results of Yardstick PTL election

https://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_b99c79421ce70058&rkey=c39fb1d5405efc15

Change-Id: I1a71f4415a737baec4bd44085ac3115fb86bd560
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Bump oslo.messaging version to 5.36.0

A decoding error was introduced in the AMQP libraries installed by
oslo.messaging version 5.30.2.

Error example:

File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/
        .tox/functional/local/lib/python2.7/site-packages/amqp/utils.py",
         line 89, in str_to_bytes
  return s.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd2 in position 3: ordinal not in range(128)

This problem is fixed in the library versions installed in version 5.36.0.

JIRA: YARDSTICK-1204

Change-Id: I4719d8e0a0891ebbb3b87c31d593d15f897ef53a
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Set "cmd2" library to version 0.8.6

Last "cmd2" version, 0.9.0, removed Python 2+ support [1]. In order to
support Python 2.7, version 0.8.6 (or earlier) must be installed.

[1] https://github.com/python-cmd2/cmd2/commit/e54e5a5b891548df4df0acb04b6e607ad8f871b1

JIRA: YARDSTICK-1205

Change-Id: Ic2fe57438720de5b21c75f9456e653c6facf3ae9
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>

Merge "PROX: [WIP] Added scale up TCs."

Merge "[release] Yardstick release note 6.1.0"

Add scale out TCs with availability zone support

Add PROX and vFW scale out TCs with availability zone
configuration. To be able to specify the availability
zone for specific VNF or TG the following example of
command line option shoud be used:

  yardstick -d task start --task-args='{
    "num_vnfs": 2, "availability_zone":
     {
      "vnf_0": "az_0", "tg_0": "az_0",
      "vnf_1": "az_1", "tg_1": "az_1",
      ... }}' <SCALE-OUT-TC-FILE>

The steps to run new scale-out TCs is documented in the
userguide.

JIRA: YARDSTICK-1203

Change-Id: Ib3878485c5483981fe9b48ab02995712bcc31ae5
Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>

Bugfix: Yardstick GUI build fails on Fraser 6.1.0

Yardstick web GUI build fails due to angular version

JIRA: YARDSTICK-1202

Change-Id: Ie075d18cb2eac2070267759978cbdb7c784599ef
Signed-off-by: rexlee8776 <limingjiang@huawei.com>

Dockerfile: add ping for prepare_env.sh

INFO: Creating openstack credentials ..
Verifying connectivity to 10.20.0.2...
/home/opnfv/repos/yardstick//tests/ci/prepare_env.sh: line 56: ping: command not found
/home/opnfv/repos/yardstick//tests/ci/prepare_env.sh: line 56: ping: command not found

https://build.opnfv.org/ci/view/yardstick/job/yardstick-fuel-baremetal-daily-master/2400/console

JIRA: YARDSTICK-1201

Change-Id: Ib9e139046724ece4be7a85cc1ae74cbb0ec8e203
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Use TRex release v2.41 to support both x86 and aarch64

Currently Yardstick uses Cisco's TRex release v2.28 which
can only support x86, for the consideration of
multi-arch support of Yardstick, the trex release v2.41
is now available and it can support multi-arch(both x86
and aarch64) now.

For the external lib of v2.41, for x86_64, it looks like:
external_libs/pyzmq-14.5.0/python2/intel/ucs4/64bit/zmq
for arm64, it looks like:
external_libs/pyzmq-14.5.0/python2/arm/ucs4/64bit/zmq
So replace the trex release v2.28 with v2.41.

JIRA: YARDSTICK-1191

Change-Id: I22dc3499283d48a89da1145cc480fd7d2a93a1cc
Signed-off-by: trevortao <trevor.tao@arm.com>