X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=blobdiff_plain;f=docs%2Ftesting%2Fuser%2Fuserguide%2F13-nsb-installation.rst;h=3a06be648587b73670a1ff15753f0d03651771af;hb=94e4d4b44d9f78f87e53267622da39fa4091624d;hp=92c7349681a472ea6d037ec7c476a781742daacd;hpb=e57fc0187a482eae59ca3a6c35c6bfc486de5521;p=yardstick.git diff --git a/docs/testing/user/userguide/13-nsb-installation.rst b/docs/testing/user/userguide/13-nsb-installation.rst index 92c734968..3a06be648 100644 --- a/docs/testing/user/userguide/13-nsb-installation.rst +++ b/docs/testing/user/userguide/13-nsb-installation.rst @@ -1,7 +1,7 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International .. License. .. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, 2016-2018 Intel Corporation. +.. (c) OPNFV, 2016-2019 Intel Corporation. .. Convention for heading levels in Yardstick documentation: @@ -21,6 +21,7 @@ NSB Installation .. _OVS-DPDK: http://docs.openvswitch.org/en/latest/intro/install/dpdk/ .. _devstack: https://docs.openstack.org/devstack/pike/> +.. _OVS-DPDK-versions: http://docs.openvswitch.org/en/latest/faq/releases/ Abstract -------- @@ -95,9 +96,10 @@ The ``nsb_setup.sh`` allows to: 1. Install Yardstick in specified mode: bare metal or container. Refer :doc:`04-installation`. 2. Install package dependencies on remote servers used as traffic generator or - sample VNF. Add such servers to ``install-inventory.ini`` file to either + sample VNF. Install DPDK, sample VNFs, TREX, collectd. + Add such servers to ``install-inventory.ini`` file to either ``yardstick-standalone`` or ``yardstick-baremetal`` server groups. - Configures IOMMU, hugepages, open file limits, CPU isolation, etc. + It configures IOMMU, hugepages, open file limits, CPU isolation, etc. 3. Build VM image either nsb or normal. The nsb VM image is used to run Yardstick sample VNF tests, like vFW, vACL, vCGNAPT, etc. The normal VM image is used to run Yardstick ping tests in OpenStack context. @@ -136,21 +138,25 @@ Modify the Yardstick installation inventory used by Ansible:: [yardstick:children] jumphost - [yardstick-standalone] - standalone ansible_host=192.168.2.51 ansible_connection=ssh - [yardstick-baremetal] - baremetal ansible_host=192.168.2.52 ansible_connection=ssh + baremetal ansible_host=192.168.2.51 ansible_connection=ssh + + [yardstick-standalone] + standalone ansible_host=192.168.2.52 ansible_connection=ssh [all:vars] - arch_amd64=amd64 - arch_arm64=arm64 - inst_mode_baremetal=baremetal - inst_mode_container=container - inst_mode_container_pull=container_pull - ubuntu_archive={"amd64": "http://archive.ubuntu.com/ubuntu/", "arm64": "http://ports.ubuntu.com/ubuntu-ports/"} - ansible_user=root - ansible_ssh_pass=root # OR ansible_ssh_private_key_file=/root/.ssh/id_rsa + # Uncomment credentials below if needed + ansible_user=root + ansible_ssh_pass=root + # ansible_ssh_private_key_file=/root/.ssh/id_rsa + # When IMG_PROPERTY is passed neither normal nor nsb set + # "path_to_vm=/path/to/image" to add it to OpenStack + # path_to_img=/tmp/workspace/yardstick-image.img + + # List of CPUs to be isolated (not used by default) + # Grub line will be extended with: + # "isolcpus= nohz=on nohz_full= rcu_nocbs=1" + # ISOL_CPUS=2-27,30-55 # physical cpu's for all NUMA nodes, four cpu's reserved .. warning:: @@ -178,14 +184,18 @@ Modify the Yardstick installation inventory used by Ansible:: .. note:: CPU isolation can be applied to the remote servers, like: - ISOL_CPUS=2-27,30-55 - Uncomment and modify accordingly in ``install-inventory.ini`` file. + ISOL_CPUS=2-27,30-55. Uncomment and modify accordingly in + ``install-inventory.ini`` file. By default ``nsb_setup.sh`` pulls Yardstick image based on Ubuntu 16.04 from docker hub and starts container, builds NSB VM image based on Ubuntu 16.04, installs packages to the servers given in ``yardstick-standalone`` and ``yardstick-baremetal`` host groups. +To pull Yardstick built based on Ubuntu 18 run:: + + ./nsb_setup.sh -i opnfv/yardstick-ubuntu-18.04:latest + To change default behavior modify parameters for ``install.yaml`` in ``nsb_setup.sh`` file. @@ -196,11 +206,15 @@ To execute an installation for a **BareMetal** or a **Standalone context**:: ./nsb_setup.sh - To execute an installation for an **OpenStack** context:: ./nsb_setup.sh +.. note:: + + Yardstick may not be operational after distributive linux kernel update if + it has been installed before. Run ``nsb_setup.sh`` again to resolve this. + .. warning:: The Yardstick VM image (NSB or normal) cannot be built inside a VM. @@ -217,11 +231,75 @@ execute:: docker exec -it yardstick bash +.. note:: + + It may be needed to configure tty in docker container to extend commandline + character length, for example: + + stty size rows 58 cols 234 + It will also automatically download all the packages needed for NSB Testing -setup. Refer chapter :doc:`04-installation` for more on Docker +setup. Refer chapter :doc:`04-installation` for more on Docker. **Install Yardstick using Docker (recommended)** +Bare Metal context example +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Let's assume there are three servers acting as TG, sample VNF DUT and jump host. + +Perform following steps to install NSB: + +1. Clone Yardstick repo to jump host. +2. Add TG and DUT servers to ``yardstick-baremetal`` group in + ``install-inventory.ini`` file to install NSB and dependencies. Install + python on servers. +3. Start deployment using docker image based on Ubuntu 16: + +.. code-block:: console + + ./nsb_setup.sh + +4. Reboot bare metal servers. +5. Enter to yardstick container and modify pod yaml file and run tests. + +Standalone context example for Ubuntu 18 +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Let's assume there are three servers acting as TG, sample VNF DUT and jump host. +Ubuntu 18 is installed on all servers. + +Perform following steps to install NSB: + +1. Clone Yardstick repo to jump host. +2. Add TG server to ``yardstick-baremetal`` group in + ``install-inventory.ini`` file to install NSB and dependencies. + Add server where VM with sample VNF will be deployed to + ``yardstick-standalone`` group in ``install-inventory.ini`` file. + Target VM image named ``yardstick-nsb-image.img`` will be placed to + ``/var/lib/libvirt/images/``. + Install python on servers. +3. Modify ``nsb_setup.sh`` on jump host: + +.. code-block:: console + + ansible-playbook \ + -e IMAGE_PROPERTY='nsb' \ + -e OS_RELEASE='bionic' \ + -e INSTALLATION_MODE='container_pull' \ + -e YARD_IMAGE_ARCH='amd64' ${extra_args} \ + -i install-inventory.ini install.yaml + +4. Start deployment with Yardstick docker images based on Ubuntu 18: + +.. code-block:: console + + ./nsb_setup.sh -i opnfv/yardstick-ubuntu-18.04:latest -o + +5. Reboot servers. +6. Enter to yardstick container and modify pod yaml file and run tests. + + System Topology --------------- @@ -234,7 +312,7 @@ System Topology | | | | | | (1)<-----(1) | | +----------+ +----------+ - trafficgen_1 vnf + trafficgen_0 vnf Environment parameters and credentials @@ -291,7 +369,7 @@ OpenStack:: Finally, you should be able to run the testcase:: - yardstick --debug task start yardstick/samples/vnf_samples/nsut// + yardstick --debug task start ./yardstick/samples/vnf_samples/nsut// Network Service Benchmarking - Bare-Metal ----------------------------------------- @@ -310,7 +388,7 @@ Bare-Metal 2-Node setup | | | | | | (n)<-----(n) | | +----------+ +----------+ - trafficgen_1 vnf + trafficgen_0 vnf Bare-Metal 3-Node setup - Correlated Traffic ++++++++++++++++++++++++++++++++++++++++++++ @@ -324,7 +402,7 @@ Bare-Metal 3-Node setup - Correlated Traffic | | | | | | | | | |(1)<---->(0)| | +----------+ +----------+ +------------+ - trafficgen_1 vnf trafficgen_2 + trafficgen_0 vnf trafficgen_1 Bare-Metal Config pod.yaml @@ -332,13 +410,13 @@ Bare-Metal Config pod.yaml Before executing Yardstick test cases, make sure that ``pod.yaml`` reflects the topology and update all the required fields.:: - cp /etc/yardstick/nodes/pod.yaml.nsb.sample /etc/yardstick/nodes/pod.yaml + cp ./etc/yardstick/nodes/pod.yaml.nsb.sample /etc/yardstick/nodes/pod.yaml .. code-block:: YAML nodes: - - name: trafficgen_1 + name: trafficgen_0 role: TrafficGen ip: 1.1.1.1 user: root @@ -447,7 +525,7 @@ On Host, where VM is created: .. code-block:: YAML servers: - vnf: + vnf_0: network_ports: mgmt: cidr: '1.1.1.7/24' @@ -505,7 +583,7 @@ SR-IOV 2-Node setup | | (n)<----->(n) | ----------------- | | | | | +----------+ +-------------------------+ - trafficgen_1 host + trafficgen_0 host @@ -533,15 +611,15 @@ SR-IOV 3-Node setup - Correlated Traffic | | | | | | | | | (n)<----->(n) | -----| (n)<-->(n) | | +----------+ +---------------------+ +--------------+ - trafficgen_1 host trafficgen_2 + trafficgen_0 host trafficgen_1 Before executing Yardstick test cases, make sure that ``pod.yaml`` reflects the topology and update all the required fields. .. code-block:: console - cp /etc/yardstick/nodes/standalone/trex_bm.yaml.sample /etc/yardstick/nodes/standalone/pod_trex.yaml - cp /etc/yardstick/nodes/standalone/host_sriov.yaml /etc/yardstick/nodes/standalone/host_sriov.yaml + cp ./etc/yardstick/nodes/standalone/trex_bm.yaml.sample /etc/yardstick/nodes/standalone/pod_trex.yaml + cp ./etc/yardstick/nodes/standalone/host_sriov.yaml /etc/yardstick/nodes/standalone/host_sriov.yaml .. note:: Update all the required fields like ip, user, password, pcis, etc... @@ -552,7 +630,7 @@ SR-IOV Config pod_trex.yaml nodes: - - name: trafficgen_1 + name: trafficgen_0 role: TrafficGen ip: 1.1.1.1 user: root @@ -588,7 +666,7 @@ SR-IOV Config host_sriov.yaml password: "" SR-IOV testcase update: -``/samples/vnf_samples/nsut/vfw/tc_sriov_rfc2544_ipv4_1rule_1flow_64B_trex.yaml`` +``./samples/vnf_samples/nsut/vfw/tc_sriov_rfc2544_ipv4_1rule_1flow_64B_trex.yaml`` Update contexts section ''''''''''''''''''''''' @@ -613,7 +691,7 @@ Update contexts section user: "" # update VM username password: "" # update password servers: - vnf: + vnf_0: network_ports: mgmt: cidr: '1.1.1.61/24' # Update VM IP address, if static, / or if dynamic, / @@ -678,7 +756,7 @@ On Host, where VM is created: .. code-block:: YAML servers: - vnf: + vnf_0: network_ports: mgmt: cidr: '1.1.1.7/24' @@ -742,7 +820,7 @@ OVS-DPDK 2-Node setup | | | (ovs-dpdk) | | | | (n)<----->(n) |------------------ | +----------+ +-------------------------+ - trafficgen_1 host + trafficgen_0 host OVS-DPDK 3-Node setup - Correlated Traffic @@ -772,14 +850,14 @@ OVS-DPDK 3-Node setup - Correlated Traffic | | | (ovs-dpdk) | | | | | | (n)<----->(n) | ------ |(n)<-->(n)| | +----------+ +-------------------------+ +------------+ - trafficgen_1 host trafficgen_2 + trafficgen_0 host trafficgen_1 Before executing Yardstick test cases, make sure that the ``pod.yaml`` reflects the topology and update all the required fields:: - cp /etc/yardstick/nodes/standalone/trex_bm.yaml.sample /etc/yardstick/nodes/standalone/pod_trex.yaml - cp /etc/yardstick/nodes/standalone/host_ovs.yaml /etc/yardstick/nodes/standalone/host_ovs.yaml + cp ./etc/yardstick/nodes/standalone/trex_bm.yaml.sample /etc/yardstick/nodes/standalone/pod_trex.yaml + cp ./etc/yardstick/nodes/standalone/host_ovs.yaml /etc/yardstick/nodes/standalone/host_ovs.yaml .. note:: Update all the required fields like ip, user, password, pcis, etc... @@ -790,7 +868,7 @@ OVS-DPDK Config pod_trex.yaml nodes: - - name: trafficgen_1 + name: trafficgen_0 role: TrafficGen ip: 1.1.1.1 user: root @@ -825,7 +903,7 @@ OVS-DPDK Config host_ovs.yaml password: "" ovs_dpdk testcase update: -``/samples/vnf_samples/nsut/vfw/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml`` +``./samples/vnf_samples/nsut/vfw/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml`` Update contexts section ''''''''''''''''''''''' @@ -861,7 +939,7 @@ Update contexts section user: "" # update VM username password: "" # update password servers: - vnf: + vnf_0: network_ports: mgmt: cidr: '1.1.1.61/24' # Update VM IP address, if static, / or if dynamic, / @@ -881,6 +959,144 @@ Update contexts section cidr: '152.16.40.10/24' gateway_ip: '152.16.100.20' +OVS-DPDK configuration options +++++++++++++++++++++++++++++++ + +There are number of configuration options available for OVS-DPDK context in +test case. Mostly they are used for performance tuning. + +OVS-DPDK properties: +'''''''''''''''''''' + +OVS-DPDK properties example under *ovs_properties* section: + + .. code-block:: console + + ovs_properties: + version: + ovs: 2.8.1 + dpdk: 17.05.2 + pmd_threads: 4 + pmd_cpu_mask: "0x3c" + ram: + socket_0: 2048 + socket_1: 2048 + queues: 2 + vpath: "/usr/local" + max_idle: 30000 + lcore_mask: 0x02 + dpdk_pmd-rxq-affinity: + 0: "0:2,1:2" + 1: "0:2,1:2" + 2: "0:3,1:3" + 3: "0:3,1:3" + vhost_pmd-rxq-affinity: + 0: "0:3,1:3" + 1: "0:3,1:3" + 2: "0:4,1:4" + 3: "0:4,1:4" + +OVS-DPDK properties description: + + +-------------------------+-------------------------------------------------+ + | Parameters | Detail | + +=========================+=================================================+ + | version || Version of OVS and DPDK to be installed | + | || There is a relation between OVS and DPDK | + | | version which can be found at | + | | `OVS-DPDK-versions`_ | + | || By default OVS: 2.6.0, DPDK: 16.07.2 | + +-------------------------+-------------------------------------------------+ + | lcore_mask || Core bitmask used during DPDK initialization | + | | where the non-datapath OVS-DPDK threads such | + | | as handler and revalidator threads run | + +-------------------------+-------------------------------------------------+ + | pmd_cpu_mask || Core bitmask that sets which cores are used by | + | || OVS-DPDK for datapath packet processing | + +-------------------------+-------------------------------------------------+ + | pmd_threads || Number of PMD threads used by OVS-DPDK for | + | | datapath | + | || This core mask is evaluated in Yardstick | + | || It will be used if pmd_cpu_mask is not given | + | || Default is 2 | + +-------------------------+-------------------------------------------------+ + | ram || Amount of RAM to be used for each socket, MB | + | || Default is 2048 MB | + +-------------------------+-------------------------------------------------+ + | queues || Number of RX queues used for DPDK physical | + | | interface | + +-------------------------+-------------------------------------------------+ + | dpdk_pmd-rxq-affinity || RX queue assignment to PMD threads for DPDK | + | || e.g.: : : | + +-------------------------+-------------------------------------------------+ + | vhost_pmd-rxq-affinity || RX queue assignment to PMD threads for vhost | + | || e.g.: : : | + +-------------------------+-------------------------------------------------+ + | vpath || User path for openvswitch files | + | || Default is ``/usr/local`` | + +-------------------------+-------------------------------------------------+ + | max_idle || The maximum time that idle flows will remain | + | | cached in the datapath, ms | + +-------------------------+-------------------------------------------------+ + + +VM image properties +''''''''''''''''''' + +VM image properties example under *flavor* section: + + .. code-block:: console + + flavor: + images: + ram: 8192 + extra_specs: + machine_type: 'pc-i440fx-xenial' + hw:cpu_sockets: 1 + hw:cpu_cores: 6 + hw:cpu_threads: 2 + hw_socket: 0 + cputune: | + + + + ... + + + + +VM image properties description: + + +-------------------------+-------------------------------------------------+ + | Parameters | Detail | + +=========================+=================================================+ + | images || Path to the VM image generated by | + | | ``nsb_setup.sh`` | + | || Default path is ``/var/lib/libvirt/images/`` | + | || Default file name ``yardstick-nsb-image.img`` | + | | or ``yardstick-image.img`` | + +-------------------------+-------------------------------------------------+ + | ram || Amount of RAM to be used for VM | + | || Default is 4096 MB | + +-------------------------+-------------------------------------------------+ + | hw:cpu_sockets || Number of sockets provided to the guest VM | + | || Default is 1 | + +-------------------------+-------------------------------------------------+ + | hw:cpu_cores || Number of cores provided to the guest VM | + | || Default is 2 | + +-------------------------+-------------------------------------------------+ + | hw:cpu_threads || Number of threads provided to the guest VM | + | || Default is 2 | + +-------------------------+-------------------------------------------------+ + | hw_socket || Generate vcpu cpuset from given HW socket | + | || Default is 0 | + +-------------------------+-------------------------------------------------+ + | cputune || Maps virtual cpu with logical cpu | + +-------------------------+-------------------------------------------------+ + | machine_type || Machine type to be emulated in VM | + | || Default is 'pc-i440fx-xenial' | + +-------------------------+-------------------------------------------------+ + OpenStack with SR-IOV support ----------------------------- @@ -918,7 +1134,7 @@ Single node OpenStack with external TG | | (PF1)<----->(PF1) +--------------------+ | | | | | +----------+ +----------------------------+ - trafficgen_1 host + trafficgen_0 host Host pre-configuration @@ -1081,7 +1297,7 @@ Multi node OpenStack TG and VNF setup (two nodes) | |sample-VNF VM | | | |sample-VNF VM | | | | | | | | | | | | TG | | | | DUT | | - | | trafficgen_1 | | | | (VNF) | | + | | trafficgen_0 | | | | (VNF) | | | | | | | | | | | +--------+ +--------+ | | +--------+ +--------+ | | | VF NIC | | VF NIC | | | | VF NIC | | VF NIC | | @@ -1169,8 +1385,8 @@ IxLoad .. code-block:: console - cp /etc/yardstick/nodes/pod.yaml.nsb.sample.ixia \ - etc/yardstick/nodes/pod_ixia.yaml + cp ./etc/yardstick/nodes/pod.yaml.nsb.sample.ixia \ + /etc/yardstick/nodes/pod_ixia.yaml Config ``pod_ixia.yaml`` @@ -1193,7 +1409,7 @@ IxLoad 4. Create a folder ``Results`` in c:\ and share the folder on the network. 5. Execute testcase in samplevnf folder e.g. - ``/samples/vnf_samples/nsut/vfw/tc_baremetal_http_ixload_1b_Requests-65000_Concurrency.yaml`` + ``./samples/vnf_samples/nsut/vfw/tc_baremetal_http_ixload_1b_Requests-65000_Concurrency.yaml`` IxNetwork ^^^^^^^^^ @@ -1205,8 +1421,8 @@ installed as part of the requirements of the project. .. code-block:: console - cp /etc/yardstick/nodes/pod.yaml.nsb.sample.ixia \ - etc/yardstick/nodes/pod_ixia.yaml + cp ./etc/yardstick/nodes/pod.yaml.nsb.sample.ixia \ + /etc/yardstick/nodes/pod_ixia.yaml Configure ``pod_ixia.yaml`` @@ -1226,7 +1442,7 @@ installed as part of the requirements of the project. (or ``IxNetworkApiServer``) 3. Execute testcase in samplevnf folder e.g. - ``/samples/vnf_samples/nsut/vfw/tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml`` + ``./samples/vnf_samples/nsut/vfw/tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml`` Spirent Landslide -----------------