.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, 2016-2017 Intel Corporation.
+.. (c) OPNFV, 2016-2018 Intel Corporation.
+..
+ Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
Yardstick - NSB Testing - Operation
===================================
downlink_0:
- xe0
+
+Availability zone
+^^^^^^^^^^^^^^^^^
+
+The configuration of the availability zone is requred in cases where location
+of exact compute host/group of compute hosts needs to be specified for
+:term:`SampleVNF` or traffic generator in the heat test case. If this is the
+case, please follow the instructions below.
+
+.. _`Create a host aggregate`:
+
+1. Create a host aggregate in the OpenStack and add the available compute hosts
+ into the aggregate group.
+
+ .. note:: Change the ``<AZ_NAME>`` (availability zone name), ``<AGG_NAME>``
+ (host aggregate name) and ``<HOST>`` (host name of one of the compute) in the
+ commands below.
+
+ .. code-block:: bash
+
+ # create host aggregate
+ openstack aggregate create --zone <AZ_NAME> \
+ --property availability_zone=<AZ_NAME> <AGG_NAME>
+ # show available hosts
+ openstack compute service list --service nova-compute
+ # add selected host into the host aggregate
+ openstack aggregate add host <AGG_NAME> <HOST>
+
+2. To specify the OpenStack location (the exact compute host or group of the hosts)
+ of SampleVNF or traffic generator in the heat test case, the ``availability_zone`` server
+ configuration option should be used. For example:
+
+ .. note:: The ``<AZ_NAME>`` (availability zone name) should be changed according
+ to the name used during the host aggregate creation steps above.
+
+ .. code-block:: yaml
+
+ context:
+ name: yardstick
+ image: yardstick-samplevnfs
+ ...
+ servers:
+ vnf__0:
+ ...
+ availability_zone: <AZ_NAME>
+ ...
+ tg__0:
+ ...
+ availability_zone: <AZ_NAME>
+ ...
+ networks:
+ ...
+
+There are two example of SampleVNF scale out test case which use the
+``availability zone`` feature to specify the exact location of scaled VNFs and
+traffic generators.
+
+Those are:
+
+.. code-block:: console
+
+ <repo>/samples/vnf_samples/nsut/prox/tc_prox_heat_context_l2fwd_multiflow-2-scale-out.yaml
+ <repo>/samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale_out.yaml
+
+.. note:: This section describes the PROX scale-out testcase, but the same
+ procedure is used for the vFW test case.
+
+1. Before running the scale-out test case, make sure the host aggregates are
+ configured in the OpenStack environment. To check this, run the following
+ command:
+
+ .. code-block:: console
+
+ # show configured host aggregates (example)
+ openstack aggregate list
+ +----+------+-------------------+
+ | ID | Name | Availability Zone |
+ +----+------+-------------------+
+ | 4 | agg0 | AZ_NAME_0 |
+ | 5 | agg1 | AZ_NAME_1 |
+ +----+------+-------------------+
+
+2. If no host aggregates are configured, please follow the instructions to
+ `Create a host aggregate`_
+
+
+3. Run the SampleVNF PROX scale-out test case, specifying the
+ ``availability zone`` of each VNF and traffic generator as task arguments.
+
+ .. note:: The ``az_0`` and ``az_1`` should be changed according to the host
+ aggregates created in the OpenStack.
+
+ .. code-block:: console
+
+ yardstick -d task start \
+ <repo>/samples/vnf_samples/nsut/prox/tc_prox_heat_context_l2fwd_multiflow-2-scale-out.yaml\
+ --task-args='{
+ "num_vnfs": 4, "availability_zone": {
+ "vnf_0": "az_0", "tg_0": "az_1",
+ "vnf_1": "az_0", "tg_1": "az_1",
+ "vnf_2": "az_0", "tg_2": "az_1",
+ "vnf_3": "az_0", "tg_3": "az_1"
+ }
+ }'
+
+ ``num_vnfs`` specifies how many VNFs are going to be deployed in the
+ ``heat`` contexts. ``vnf_X`` and ``tg_X`` arguments configure the
+ availability zone where the VNF and traffic generator is going to be deployed.
+
+
Collectd KPIs
-------------
NSB can collect KPIs from collected. We have support for various plugins
-enabled by the Barometer project.
+enabled by the :term:`Barometer` project.
The default yardstick-samplevnf has collectd installed. This allows for
collecting KPIs from the VNF.
To collectd KPIs from the NFVi compute nodes:
-
* install_collectd on the compute nodes
* create pod.yaml for the compute nodes
* enable specific plugins depending on the vswitch and DPDK
- example pod.yaml section for Compute node running collectd.
+ example ``pod.yaml`` section for Compute node running collectd.
.. code-block:: yaml
An example scale-up Heat testcase is:
-.. literalinclude:: /submodules/yardstick/samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml
+.. literalinclude:: /../samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml
:language: yaml
This testcase template requires specifying the number of VCPUs, Memory and Ports.
.. code-block:: console
- yardstick task start --task-args='{"mem": 10480, "vcpus": 4, "ports": 2}' \
+ yardstick task start --task-args='{"mem": 10480, "vcpus": 4, "vports": 2}' \
samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml
In order to support ports scale-up, traffic and topology templates need to be used in testcase.
A example topology template is:
-.. literalinclude:: /submodules/yardstick/samples/vnf_samples/nsut/vfw/vfw-tg-topology-scale-up.yaml
+.. literalinclude:: /../samples/vnf_samples/nsut/vfw/vfw-tg-topology-scale-up.yaml
:language: yaml
This template has ``vports`` as an argument. To pass this argument it needs to
A example traffic profile template is:
-.. literalinclude:: /submodules/yardstick/samples/vnf_samples/traffic_profiles/ipv4_throughput-scale-up.yaml
+.. literalinclude:: /../samples/vnf_samples/traffic_profiles/ipv4_throughput-scale-up.yaml
:language: yaml
There is an option to provide predefined config for SampleVNFs. Path to config
file: /etc/yardstick/nodes/pod.yaml
Scale-Out
---------------------
+---------
VNFs performance data with scale-out helps
- * in capacity planning to meet the given network node requirements
- * in comparison between different VNF vendor offerings
+ * capacity planning to meet the given network node requirements
+ * comparison between different VNF vendor offerings
* better the scale-out index, provides the flexibility in meeting future
capacity requirements
OVS-DPDK uses OVS network with DPDK drivers.
SRIOV enables network traffic to bypass the software switch layer of the Hyper-V stack.
+Emulated machine type
+^^^^^^^^^^^^^^^^^^^^^
+
+For better performance test results of emulated VM spawned by Yardstick SA
+context (OvS-DPDK/SRIOV), it may be important to control the emulated machine
+type used by QEMU emulator. This attribute can be configured via TC definition
+in ``contexts`` section under ``extra_specs`` configuration.
+
+For example:
+
+.. code-block:: yaml
+
+ contexts:
+ ...
+ - type: StandaloneSriov
+ ...
+ flavor:
+ ...
+ extra_specs:
+ ...
+ machine_type: pc-i440fx-bionic
+
+Where, ``machine_type`` can be set to one of the emulated machine type
+supported by QEMU running on SUT platform. To get full list of supported
+emulated machine types, the following command can be used on the target SUT
+host.
+
+.. code-block:: yaml
+
+ # qemu-system-x86_64 -machine ?
+
+By default, the ``machine_type`` option is set to ``pc-i440fx-xenial`` which is
+suitable for running Ubuntu 16.04 VM image. So, if this type is not supported
+by the target platform or another VM image is used for stand alone (SA) context
+VM (e.g.: ``bionic`` image for Ubuntu 18.04), this configuration should be
+changed accordingly.
+
Standalone with OVS-DPDK
^^^^^^^^^^^^^^^^^^^^^^^^
Sample test case file
^^^^^^^^^^^^^^^^^^^^^
- 1. Prepare SampleVNF image and copy it to ``flavor/images``.
- 2. Prepare context files for TREX and SampleVNF under ``contexts/file``.
- 3. Add bridge named ``br-int`` to the baremetal where SampleVNF image is deployed.
- 4. Modify ``networks/phy_port`` accordingly to the baremetal setup.
- 5. Run test from:
+1. Prepare SampleVNF image and copy it to ``flavor/images``.
+2. Prepare context files for TREX and SampleVNF under ``contexts/file``.
+3. Add bridge named ``br-int`` to the baremetal where SampleVNF image is deployed.
+4. Modify ``networks/phy_port`` accordingly to the baremetal setup.
+5. Run test from:
-.. literalinclude:: /submodules/yardstick/samples/vnf_samples/nsut/acl/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
+.. literalinclude:: /../samples/vnf_samples/nsut/acl/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
:language: yaml
+
+Preparing test run of vEPC test case
+------------------------------------
+
+Provided vEPC test cases are examples of emulation of vEPC infrastructure
+components, such as UE, eNodeB, MME, SGW, PGW.
+
+Location of vEPC test cases: ``samples/vnf_samples/nsut/vepc/``.
+
+Before running a specific vEPC test case using NSB, some preconfiguration
+needs to be done.
+
+Update Spirent Landslide TG configuration in pod file
+=====================================================
+
+Examples of ``pod.yaml`` files could be found in
+:file:`etc/yardstick/nodes/standalone`.
+The name of related pod file could be checked in the context section of NSB
+test case.
+
+The ``pod.yaml`` related to vEPC test case uses some sub-structures that hold the
+details of accessing the Spirent Landslide traffic generator.
+These subsections and the changes to be done in provided example pod file are
+described below.
+
+1. ``tas_manager``: data under this key holds the information required to
+access Landslide TAS (Test Administration Server) and perform needed
+configurations on it.
+
+ * ``ip``: IP address of TAS Manager node; should be updated according to test
+ setup used
+ * ``super_user``: superuser name; could be retrieved from Landslide documentation
+ * ``super_user_password``: superuser password; could be retrieved from
+ Landslide documentation
+ * ``cfguser_password``: password of predefined user named 'cfguser'; default
+ password could be retrieved from Landslide documentation
+ * ``test_user``: username to be used during test run as a Landslide library
+ name; to be defined by test run operator
+ * ``test_user_password``: password of test user; to be defined by test run
+ operator
+ * ``proto``: *http* or *https*; to be defined by test run operator
+ * ``license``: Landslide license number installed on TAS
+
+2. The ``config`` section holds information about test servers (TSs) and
+systems under test (SUTs). Data is represented as a list of entries.
+Each such entry contains:
+
+ * ``test_server``: this subsection represents data related to test server
+ configuration, such as:
+
+ * ``name``: test server name; unique custom name to be defined by test
+ operator
+ * ``role``: this value is used as a key to bind specific Test Server and
+ TestCase; should be set to one of test types supported by TAS license
+ * ``ip``: Test Server IP address
+ * ``thread_model``: parameter related to Test Server performance mode.
+ The value should be one of the following: "Legacy" | "Max" | "Fireball".
+ Refer to Landslide documentation for details.
+ * ``phySubnets``: a structure used to specify IP ranges reservations on
+ specific network interfaces of related Test Server. Structure fields are:
+
+ * ``base``: start of IP address range
+ * ``mask``: IP range mask in CIDR format
+ * ``name``: network interface name, e.g. *eth1*
+ * ``numIps``: size of IP address range
+
+ * ``preResolvedArpAddress``: a structure used to specify the range of IP
+ addresses for which the ARP responses will be emulated
+
+ * ``StartingAddress``: IP address specifying the start of IP address range
+ * ``NumNodes``: size of the IP address range
+
+ * ``suts``: a structure that contains definitions of each specific SUT
+ (represents a vEPC component). SUT structure contains following key/value
+ pairs:
+
+ * ``name``: unique custom string specifying SUT name
+ * ``role``: string value corresponding with an SUT role specified in the
+ session profile (test session template) file
+ * ``managementIp``: SUT management IP adress
+ * ``phy``: network interface name, e.g. *eth1*
+ * ``ip``: vEPC component IP address used in test case topology
+ * ``nextHop``: next hop IP address, to allow for vEPC inter-node communication
+
+Update NSB test case definitions
+================================
+NSB test case file designated for vEPC testing contains an example of specific
+test scenario configuration.
+Test operator may change these definitions as required for the use case that
+requires testing.
+Specifically, following subsections of the vEPC test case (section **scenarios**)
+may be changed.
+
+1. Subsection ``options``: contains custom parameters used for vEPC testing
+
+ * subsection ``dmf``: may contain one or more parameters specified in
+ ``traffic_profile`` template file
+ * subsection ``test_cases``: contains re-definitions of parameters specified
+ in ``session_profile`` template file
+
+ .. note:: All parameters in ``session_profile``, value of which is a
+ placeholder, needs to be re-defined to construct a valid test session.
+
+2. Subsection ``runner``: specifies the test duration and the interval of
+TG and VNF side KPIs polling. For more details, refer to :doc:`03-architecture`.
+
+Preparing test run of vPE test case
+-----------------------------------
+The vPE (Provider Edge Router) is a :term: `VNF` approximation
+serving as an Edge Router. The vPE is approximated using the
+``ip_pipeline`` dpdk application.
+
+ .. image:: images/vPE_Diagram.png
+ :width: 800px
+ :alt: NSB vPE Diagram
+
+The ``vpe_config`` file must be passed as it is not auto generated.
+The ``vpe_script`` defines the rules applied to each of the pipelines. This can be
+auto generated or a file can be passed using the ``script_file`` option in
+``vnf_config`` as shown below. The ``full_tm_profile_file`` option must be
+used if a traffic manager is defined in ``vpe_config``.
+
+.. code-block:: yaml
+
+ vnf_config: { file: './vpe_config/vpe_config_2_ports',
+ action_bulk_file: './vpe_config/action_bulk_512.txt',
+ full_tm_profile_file: './vpe_config/full_tm_profile_10G.cfg',
+ script_file: './vpe_config/vpe_script_sample' }
+
+Testcases for vPE can be found in the ``vnf_samples/nsut/vpe`` directory.
+A testcase can be started with the following command as an example:
+
+.. code-block:: bash
+
+ yardstick task start /yardstick/samples/vnf_samples/nsut/vpe/tc_baremetal_rfc2544_ipv4_1flow_64B_ixia.yaml
Code Review / yardstick.git / blobdiff