vnfdgen: replace yaml.load with yaml.safe_load

JIRA: YARDSTICK-760

Change-Id: I145c948c1f08562e12877f5d0fe7e797e9d989f9
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Bugfix: missing license in api directory

JIRA: YARDSTICK-759

In api directory, there are many files missing license header.
We need to add it.

Change-Id: Ia46cf207295007d4eb7fbf0d3319611f256c36bc
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Open storperf testcase to huawei-pod2"

Merge "base: remove redudant ip_to_convert"

Merge "Improvement: support more parameters in fio testcase"

Merge "HA testcase containerized Compass support"

Merge "heat: port_security_enabled, allowed_address_pairs, disable gateway_ip"

Add API(v2) to get all task info

JIRA: YARDSTICK-754

API: /api/v2/yardstick/tasks
METHOD: GET

Change-Id: I556b0880674d78d67e0a556c4d95f902131cd027
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Add API(v2) to run task"

Merge "Add API(v2) to get all images"

Merge "Add API(v2) to load images"

Add API(v2) to run task

JIRA: YARDSTICK-741

API: /api/v2/yardstick/tasks/<task_id>
METHOD: PUT
PARAMS:
{
    'action': 'run'
}

Change-Id: Ia5340c6ff45e3e7e70ab4597c39476138e7016f2
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to get all images

JIRA: YARDSTICK-752

API: /api/v2/yardstick/images
METHOD: GET

Change-Id: I7b0bba85d875575ee684cc42cad977af91d6f84e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Add API(v2) to add suite to task"

Add API(v2) to load images

JIRA: YARDSTICK-751

API: /api/v2/yardstick/images/action
METHOD: POST
PARAMS:
{
    'action': 'load_image'
}

Change-Id: I9554f35c679ae2a861c7922080aa54a1ffc5bac7
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to add suite to task

JIRA: YARDSTICK-740

API: /api/v2/yardstick/tasks/<task_id>
METHOD: PUT
PARAMS:
{
    'action': 'add_suite',
    'args': {
        'suite_name': 'opnfv_smoke',
        'suite_content': suite_content
    }
}

Change-Id: I1f81c45f143b2fea636cb87f60f2f394bf1380b1
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Add API(v2) to delete certain test suite"

Merge "Add API(v2) to get certain test suite info"

Merge "Add API(v2) to get all test suite"

Merge "Add API(v2) to create test suite"

Merge "Add API to delete certain test case"

Merge "Add API(v2) to get certain test case info"

Merge "Add API(v2) to get all test case info"

Merge "Add API(v2) to upload test case"

Merge "Add API(v2) to add case to task"

Merge "Add API(v2) to delete certain task"

Merge "Add API(v2) to get certain task info"

Merge "Add API(v2) to add environment to task"

Merge "Add API(v2) to create task"

Merge "Add API(v2) to delete certain project"

Merge changes from topics 'get_project', 'get_projects'

* changes:
  Add API(V2) to get certain project info
  Add API(v2) to get all projects info

Merge "Add API(v2) to create project"

Improvement: support more parameters in fio testcase

JIRA: YARDSTICK-750

The parameters that fio testcase support now maybe insufficient in some usecase.
The test case should support parameters such as: rwmixwrite, size, direct and numjobs.
This work is about add more parameters that can be specified in options section
in the YAML file.

Change-Id: Icccf024dc78d82bc453480f3baafb8aaa5c19538
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Add API(v2) to delete certain test suite

JIRA: YARDSTICK-749

API: /api/v2/yardstick/testsuites/<suite_name>
METHOD: DELETE

Change-Id: Idfb5086ab3ac9e7ea6f9512ee5c7f02090d554c0
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

HA testcase containerized Compass support

Compass will be containerized in Euphrates release.
This work is about support HA testcase in a containerized environment.

Change-Id: Ibb668c81f046d6400293f8cd58181b9f20aeebe3
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Add API(v2) to get certain test suite info

JIRA: YARDSTICK-748

API: /api/v2/yardstick/testsuites/<suite_name>
METHOD: GET

Change-Id: I0d60ddedff4f76504ebd2e61257bf1b9cad3b923
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to get all test suite

JIRA: YARDSTICK-747

API: /api/v2/yardstick/testsuites
METHOD: GET

Change-Id: I2375b6d2ea54704f5a2328cc8110058af038e777
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to create test suite

JIRA: YARDSTICK-746

API: /api/v2/yardstick/testsuites/action
METHOD: POST
PARAMS:
{
    'action': 'create_suite',
    'args': {
        'name': 'case_name',
        'testcases': [
            'opnfv_yardstick_tc002'
        ]
    }
}

Change-Id: I7e07c599e64e7eedda274c72344e394f5535f055
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API to delete certain test case

JIRA: YARDSTICK-745

API: /api/v2/yardstick/testcases/<case_name>
METHOD: DELETE

Change-Id: Ic75ee60abfc00d20328ee45b9830dea1fe464482
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to get certain test case info

JIRA: YARDSTICK-744

API: /api/v2/yardstick/testcases
METHOD: GET

Change-Id: I4b1867f713bce2f9f2342ecec8540b08b0bb3456
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to get all test case info

JIRA: YARDSTICK-743

API: /api/v2/yardstick/testcases
METHOD: GET

Change-Id: Iced97206cb4c5f68ee8a5729af937e8d45ca6e87
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to upload test case

JIRA: YARDSTICK-742

API: /api/v2/yardstick/testcases/action
METHOD: POST
PARAMS:
{
   'action': 'upload_case',
    'args': {
        'file': file
    }
}

Change-Id: Id305e72a24ec28295ac4bae5e5b9ad83eb2c1e34
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to add case to task

JIRA: YARDSTICK-739

API: /api/v2/yardstick/tasks/<task_id>
METHOD: PUT
PARAMS:
{
    'action': 'add_case',
    'args': {
        'case_name': 'opnfv_yardstick_tc002',
        'case_content': case_content
    }
}

Change-Id: I53e71e3d662399b08029ed417b5873a4775ca32d
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

heat: port_security_enabled, allowed_address_pairs, disable gateway_ip

    xe0:
      cidr: '10.0.2.0/24'
      vld_id: public
      allowed_address_pairs:
        - ip_address:
            '0.0.0.0/0'
    xe1:
      cidr: '10.0.3.0/24'
      vld_id: private
      allowed_address_pairs:
        - ip_address:
            '0.0.0.0/0'

We need to disable gateways on data plane interfaces
With TGs and VFNs we probably don't need gateway on data interfaces

Change-Id: I276a7d591a0a3ff03877c4f525303fc2874de683
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Add API(v2) to delete certain task

JIRA: YARDSTICK-738

API: /api/v2/yardstick/tasks/<task_id>
METHOD: DELETE

Change-Id: I28215d2cae63551455abacc4142cff3c4aaed281
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to get certain task info

JIRA: YARDSTICK-737

API: /api/v2/yardstick/tasks/<task_id>
METHOD: GET

Change-Id: Ia360f4bba05e196e07c9eb339061ccfbf5df8137
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to add environment to task

JIRA: YARDSTICK-736

API: /api/v2/yardstick/tasks/<task_id>
METHOD: PUT
PARAMS:
{
    'action': 'add_environment',
    'args': {
        'environment_id': environment_id
    }
}

Change-Id: Idc31cb21de882d008202b6395158e9f9aa9244db
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to create task

JIRA: YARDSTICK-735

API: /api/v2/yardstick/tasks/action
METHOD: POST
PARAMS:
{
    'action': 'create_task',
    'args': {
        'name': 'task1',
        'project_id': project_id
    }
}

Change-Id: I1f9c743f32bbcff999e37cf53ebfa96b41c61e5e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Add API(v2) to delete container"

Merge "Add API(v2) to get container info"

Merge "Add API(v2) to create grafana"

Merge "Add API(v2) to create influxdb"

Merge "Add API(v2) to delete pod"

Add API(v2) to delete certain project

JIRA: YARDSTICK-734

API: /api/v2/yardstick/projects/<project_id>
METHOD: DELETE

Change-Id: I6628ec42aa7c4c6ab073ddc33c119ea1c03b81fe
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(V2) to get certain project info

JIRA: YARDSTICK-733

API: /api/v2/yardstick/projects/<project_id>
METHOD: GET

Change-Id: I20a615af96229aefac6ef86566a7b9c5d42f9633
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to get all projects info

JIRA: YARDSTICK-732

API: /api/v2/yardstick/projects
METHOD: GET

Change-Id: Ifd67f11f516270d96c0e80b5b8c76a939583d9e5
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to create project

JIRA: YARDSTICK-731

API: /api/v2/yardstick/projects/action
METHOD: POST
PARAMS:
{
    'action': 'create_project',
    'args': {
        'name': 'project1'
    }
}

Change-Id: I92332dec8dcf3253f0cbf1668332ce5110111d73
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

base: remove redudant ip_to_convert

remove extra newline as well

Change-Id: I9038621720c40802aa041bad66007b38070cb26e
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "test_base: speedup unittest, wait for queue not empty"

Add API(v2) to delete container

JIRA: YARDSTICK-729

API: /api/v2/yardstick/containers/<container_id>
METHOD: DELETE

Change-Id: I4cda76329edd7b988a357f1fcb2b5041bf48be64
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to get container info

JIRA: YARDSTICK-728

API: /api/v2/yardstick/containers/<container_id>
METHOD: GET

Change-Id: Iba34e6c6dd84573dc9d5914a584d920678c0de36
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to create grafana

JIRA: YARDSTICK-727

API: /api/v2/yardstick/containers/action
METHOD: POST
PARAMS:
{
    'action': 'create_influxdb',
    'args': {
        'environment_id': environment_id
    }
}

Change-Id: I12fc23261ba62df6f923a4d34455ad8b46c4aa8f
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Open storperf testcase to huawei-pod2

JIRA: YARDSTICK-712

Change-Id: If91c936a3a59580a987cb6762db5d063edf6c9d7
Signed-off-by: JingLu5 <lvjing5@huawei.com>

Merge "attacker_process: fix typo s/envrioment/environment/"

Merge "test_nstat: speedup unittests, sleep for 0 seconds"

Add API(v2) to create influxdb

JIRA: YARDSTICK-726

API: /api/v2/yardstick/containers/action
METHOD: POST
PARAMS:
{
    'action': 'create_influxdb',
    'args': {
        'environment_id': environment_id
    }
}

Change-Id: I20961fc6ae9918c894de7fabd3f81ca966bd24cd
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "pktgen: speedup unittest, mock time.sleep"

Merge "tg: speedup unittests, mock time.sleep"

Merge "test_pktgen_dpdk_throughput: speedup unittest, mock time.sleep()"

test_nstat: speedup unittests, sleep for 0 seconds

Change-Id: I3f961e26fb045ea7501dc3a5b630b3ef94fec3ee
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

test_heat: ipaddress expects unicode inputs

weird error

E           AddressValueError: '10.20.0.0/15' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

I guess we need to convert the stack.outputs mock to unicode

FAILED
tests/unit/benchmark/contexts/test_heat.py:137 (HeatContextTestCase.test_add_server_port)
self = <tests.unit.benchmark.contexts.test_heat.HeatContextTestCase testMethod=test_add_server_port>

    def test_add_server_port(self):
        network1 = mock.MagicMock()
        network1.vld_id = 'vld111'
        network2 = mock.MagicMock()
        network2.vld_id = 'vld777'
        self.test_context.name = 'foo'
        self.test_context.stack = mock.MagicMock()
        self.test_context.networks = {
            'a': network1,
            'c': network2,
        }
        self.test_context.stack.outputs = {
            'b': '10.20.30.45',
            'b-subnet_id': 1,
            'foo-a-subnet-cidr': '10.20.0.0/15',
            'foo-a-subnet-gateway_ip': '10.20.30.1',
            'b-mac_address': '00:01',
            'b-device_id': 'dev21',
            'b-network_id': 'net789',
            'd': '40.30.20.15',
            'd-subnet_id': 2,
            'foo-c-subnet-cidr': '40.30.0.0/18',
            'foo-c-subnet-gateway_ip': '40.30.20.254',
            'd-mac_address': '00:10',
            'd-device_id': 'dev43',
            'd-network_id': 'net987',
        }
        server = mock.MagicMock()
        server.ports = OrderedDict([
            ('a', {'stack_name': 'b'}),
            ('c', {'stack_name': 'd'}),
        ])

        expected = {
            "private_ip": '10.20.30.45',
            "subnet_id": 1,
            "subnet_cidr": '10.20.0.0/15',
            "network": '10.20.0.0',
            "netmask": '255.254.0.0',
            "gateway_ip": '10.20.30.1',
            "mac_address": '00:01',
            "device_id": 'dev21',
            "network_id": 'net789',
            "network_name": 'a',
            "local_mac": '00:01',
            "local_ip": '10.20.30.45',
            "vld_id": 'vld111',
        }
>       self.test_context.add_server_port(server)

tests/unit/benchmark/contexts/test_heat.py:186:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
yardstick/benchmark/contexts/heat.py:307: in add_server_port
    network_name, port['stack_name'], self.stack.outputs)
yardstick/benchmark/contexts/heat.py:315: in make_interface_dict
    subnet_ip = ipaddress.ip_network(subnet_cidr)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = '10.20.0.0/15', strict = True

    def ip_network(address, strict=True):
        """Take an IP string/int and return an object of the correct type.

        Args:
            address: A string or integer, the IP network.  Either IPv4 or
              IPv6 networks may be supplied; integers less than 2**32 will
              be considered to be IPv4 by default.

        Returns:
            An IPv4Network or IPv6Network object.

        Raises:
            ValueError: if the string passed isn't either a v4 or a v6
              address. Or if the network has host bits set.

        """
        try:
            return IPv4Network(address, strict)
        except (AddressValueError, NetmaskValueError):
            pass

        try:
            return IPv6Network(address, strict)
        except (AddressValueError, NetmaskValueError):
            pass

        if isinstance(address, bytes):
            raise AddressValueError(
                '%r does not appear to be an IPv4 or IPv6 network. '
                'Did you pass in a bytes (str in Python 2) instead of'
>               ' a unicode object?' % address)
E           AddressValueError: '10.20.0.0/15' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

../../yardstick/yardstick_venv/lib/python2.7/site-packages/ipaddress.py:199: AddressValueError

Change-Id: Ie3b087a26a054203573eaa9b13c3e90152bba6a9
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "test_tg_rfc2544_trex: speedup unittest, set time.sleep(0)"

test_tg_rfc2544_trex: speedup unittest, set time.sleep(0)

Change-Id: I6fdae629bdbd45cba715a838c41613b914a3abcb
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

test_delete_all_calls_delete: fix delete mock

we need to mock HeatStack.delete() before
we instantiate otherwise we can't
reach the instantiated objects delete() method

we need to patch the class so we patch all instances

Change-Id: I36f9476dcfb83e2d583c5a9f72dc27fce57258eb
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

test_heat: don't waste 2 seconds in unittest

remove duration checks and test with sleep 0

Change-Id: I22516cde56ac3a02358ca9c3e652cead10580d76
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "Bugfix: kubernetes context do not implement _get_network"

Bugfix: kubernetes context do not implement _get_network

Since kubernetes context do not implement _get_network,
so when run unit test case, we will get a error:

    TypeError: Can't instantiate abstract class KubernetesContext with
    abstract methods _get_network

Change-Id: Ib56abe7c580ef8a6fc9f52f3fcd566d0fa70e1cc
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "apexlake: speed unittest, mock time.sleep"

Merge "test_attacker_baremetal: don't run local commands"

Merge "Kubernetes (k8s) support"

Merge "add network info to topology"

Kubernetes (k8s) support

JIRA: YARDSTICK-682

We decide to support k8s in E release.
We need to discuss with openretriver team and then rewrite the ping
test case under k8s as the first step.

Change-Id: I3f81ebca8de5c1f3a8b7d42581cd7342dc320239
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

add release date

Change-Id: Icf12b7a59144754f65f67ed54a5c455a64c5042f
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Danube 3.1 release notes

Change-Id: I7032dd2348dd7212484a4a3cbd40e1d7698c8668
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Add API(v2) to delete pod

JIRA: YARDSTICK-725

API: /api/v2/yardstick/pods/<pod_id>
METHOD: DELETE

Change-Id: Ib9e9f9ced455c4ee3b6e1f63d773ea2e3ee906a6
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Initial all url of api v2"

attacker_process: fix typo s/envrioment/environment/

Change-Id: I1e5db657eb88c73b5d9223c47e2d97dc79eec1f1
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "test_monitor_command: mock LOG so we don't output error"

Initial all url of api v2

JIRA: YARDSTICK-648

Change-Id: I915889a974da9c49188c25fe2443ece481987ecb
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to get pod info

JIRA: YARDSTICK-724

API: /api/v2/yardstick/pods/<pod_id>
METHOD: GET

Change-Id: I49608eab1f1625cd4358732da052ce3745e28c24
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

test_attacker_baremetal: don't run local commands

these tests were failing to mock subprocess.check_output
and thus were trying to run sudo commands on the local
system.

This is dangerous.  Add the subprocess mock.

Also mock the LOG object so we don't print
bogus Runtime error tracebacks in the unittest logs
when we test assertRaises()

Change-Id: I01535f9952fbd95ce2f5972b641c51ff836e7e8c
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

test_monitor_command: mock LOG so we don't output error

Don't print fake tracebacks to the unittest logs

Change-Id: I8a468b8c6566f02be88a9dd222567c14c66b0956
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>

Merge "Add API(v2) to delete openrc"

Add API(v2) to upload pod file

JIRA: YARDSTICK-723

API: /api/v2/yardstick/pods/action
METHOD: POST
PARAMS:
{
    'action': 'upload_pod_file',
    'args': {
        'file': file,
        'environment_id': environment_id
    }
}

Change-Id: I5eb065d8b46080a94c989ec9b8217dc54900bd06
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to delete openrc

JIRA: YARDSTICK-722

API: /api/v2/yardstick/environments/openrcs/<openrc_id>
METHOD: DELETE

Change-Id: Ia0989fe74681654750af80ddb935bb3e63485f43
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to get openrc info

JIRA: YARDSTICK-721

API: /api/v2/yardstick/environments/openrcs/<openrc_id>
METHOD: GET

Change-Id: I57241381cf50fef0dc4eec3281c01ba106262a5a
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2)  to update openrc

JIRA: YARDSTICK-720

API: /api/v2/yardstick/environments/openrcs/action
METHOD: POST
PARAMS:
{
    'action': 'update_openrc',
        'args': {
            'openrc': {
                "EXTERNAL_NETWORK": "ext-net",
                "OS_AUTH_URL": "http://192.168.23.51:5000/v3",
                "OS_IDENTITY_API_VERSION": "3",
                "OS_IMAGE_API_VERSION": "2",
                "OS_PASSWORD": "console",
                "OS_PROJECT_DOMAIN_NAME": "default",
                "OS_PROJECT_NAME": "admin",
                "OS_TENANT_NAME": "admin",
                "OS_USERNAME": "admin",
                "OS_USER_DOMAIN_NAME": "default"
            },
            'environment_id': environment_id
        }
}

Change-Id: Ie9a1614190a01456fd0896f0bdfd05f9d0724fc6
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Add API(v2) to upload openrc

JIRA: YARDSTICK-719

API: /api/v2/yardstick/environments/openrcs/action
METHOD: POST
PARAMS:
{
    'action': 'upload_openrc',
    'args': {
        'file': file,
        'environment_id': environment_id
    }
}

Change-Id: If367904a2d0c2d4a192fdc87a8da21ac6549269e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>

Merge "Add API to delete single environment"

Merge "Add API to get single environment info"