.. (c) OPNFV, Intel Corporation, AT&T and others.
vSwitchPerf test suites userguide
-=================================
+---------------------------------
General
--------
+^^^^^^^
VSPERF requires a traffic generators to run tests, automated traffic gen
support in VSPERF includes:
<http://artifacts.opnfv.org/vswitchperf/docs/configguide/trafficgen.html>`__
VSPERF Installation
---------------------
+^^^^^^^^^^^^^^^^^^^
+
To see the supported Operating Systems, vSwitches and system requirements,
please follow the `installation instructions
<http://artifacts.opnfv.org/vswitchperf/docs/configguide/installation.html>`__ to
install.
Traffic Generator Setup
------------------------
+^^^^^^^^^^^^^^^^^^^^^^^
+
Follow the `Traffic generator instructions
<http://artifacts.opnfv.org/vswitchperf/docs/configguide/trafficgen.html>`__ to
install and configure a suitable traffic generator.
Cloning and building src dependencies
--------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In order to run VSPERF, you will need to download DPDK and OVS. You can
do this manually and build them in a preferred location, OR you could
$ make clobber
Configure the ``./conf/10_custom.conf`` file
---------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
The ``10_custom.conf`` file is the configuration file that overrides
default configurations in all the other configuration files in ``./conf``
The supplied ``10_custom.conf`` file **MUST** be modified, as it contains
the custom configuration value.
Using a custom settings file
-----------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If your ``10_custom.conf`` doesn't reside in the ``./conf`` directory
of if you want to use an alternative configuration file, the file can
3. Configuration file(s)
vloop_vnf
----------
+^^^^^^^^^
+
vsperf uses a VM called vloop_vnf for looping traffic in the PVP and PVVP
deployment scenarios. The image can be downloaded from
`<http://artifacts.opnfv.org/>`__.
Alternatively you can use your own QEMU image.
l2fwd Kernel Module
--------------------
+^^^^^^^^^^^^^^^^^^^
+
A Kernel Module that provides OSI Layer 2 Ipv4 termination or forwarding with
support for Destination Network Address Translation (DNAT) for both the MAC and
IP addresses. l2fwd can be found in <vswitchperf_dir>/src/l2fwd
Executing tests
----------------
+^^^^^^^^^^^^^^^
Before running any tests make sure you have root permissions by adding
the following line to /etc/sudoers:
$ ./vsperf --conf-file user_settings.py
--tests RFC2544Tput
- --test-param "duration=10;pkt_sizes=128"
+ --test-params "duration=10;pkt_sizes=128"
For all available options, check out the help dialog:
$ ./vsperf --help
Executing Vanilla OVS tests
-----------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
1. If needed, recompile src for all OVS variants
.. code-block:: console
VSWITCH = 'OvsVanilla'
- VSWITCH_VANILLA_PHY_PORT_NAMES = ['$PORT1', '$PORT2']
Where $PORT1 and $PORT2 are the Linux interfaces you'd like to bind
to the vswitch.
Executing PVP and PVVP tests
-----------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
To run tests using vhost-user as guest access method:
1. Set VHOST_METHOD and VNF of your settings file to:
$ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
Executing PVP tests using Vanilla OVS
--------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
To run tests using Vanilla OVS:
1. Set the following variables:
or use --test-param
$ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
- --test-param "vanilla_tgen_tx_ip=n.n.n.n;
+ --test-params "vanilla_tgen_tx_ip=n.n.n.n;
vanilla_tgen_tx_mac=nn:nn:nn:nn:nn:nn"
$ ./vsperf --conf-file<path_to_custom_conf>/10_custom.conf
+.. _vfio-pci:
+
+Using vfio_pci with DPDK
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To use vfio with DPDK instead of igb_uio edit 'conf/02_vswitch.conf'
+with the following parameters:
+
+.. code-block:: console
+
+ DPDK_MODULES = [
+ ('vfio-pci'),
+ ]
+ SYS_MODULES = ['cuse']
+
+**NOTE:** Please ensure that Intel VT-d is enabled in BIOS.
+
+**NOTE:** Please ensure your boot/grub parameters include
+the following:
+
+.. code-block:: console
+
+ iommu=pt intel_iommu=on
+
+To check that IOMMU is enabled on your platform:
+
+.. code-block:: console
+
+ $ dmesg | grep IOMMU
+ [ 0.000000] Intel-IOMMU: enabled
+ [ 0.139882] dmar: IOMMU 0: reg_base_addr fbffe000 ver 1:0 cap d2078c106f0466 ecap f020de
+ [ 0.139888] dmar: IOMMU 1: reg_base_addr ebffc000 ver 1:0 cap d2078c106f0466 ecap f020de
+ [ 0.139893] IOAPIC id 2 under DRHD base 0xfbffe000 IOMMU 0
+ [ 0.139894] IOAPIC id 0 under DRHD base 0xebffc000 IOMMU 1
+ [ 0.139895] IOAPIC id 1 under DRHD base 0xebffc000 IOMMU 1
+ [ 3.335744] IOMMU: dmar0 using Queued invalidation
+ [ 3.335746] IOMMU: dmar1 using Queued invalidation
+ ....
+
+.. _SRIOV-support:
+
+Using SRIOV support
+^^^^^^^^^^^^^^^^^^^
+
+To use virtual functions of NIC with SRIOV support, use extended form
+of NIC PCI slot definition:
+
+.. code-block:: python
+
+ WHITELIST_NICS = ['0000:05:00.0|vf0', '0000:05:00.1|vf3']
+
+Where 'vf' is an indication of virtual function usage and following
+number defines a VF to be used. In case that VF usage is detected,
+then vswitchperf will enable SRIOV support for given card and it will
+detect PCI slot numbers of selected VFs.
+
+So in example above, one VF will be configured for NIC '0000:05:00.0'
+and four VFs will be configured for NIC '0000:05:00.1'. Vswitchperf
+will detect PCI addresses of selected VFs and it will use them during
+test execution.
+
+At the end of vswitchperf execution, SRIOV support will be disabled.
+
+SRIOV support is generic and it can be used in different testing scenarios.
+For example:
+
+* vSwitch tests with DPDK or without DPDK support to verify impact
+ of VF usage on vSwitch performance
+* tests without vSwitch, where traffic is forwared directly
+ between VF interfaces by packet forwarder (e.g. testpmd application)
+* tests without vSwitch, where VM accesses VF interfaces directly
+ by PCI-passthrough_ to measure raw VM throughput performance.
+
+.. _PCI-passthrough:
+
+Using QEMU with PCI passthrough support
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Raw virtual machine throughput performance can be measured by execution of PVP
+test with direct access to NICs by PCI passthrough. To execute VM with direct
+access to PCI devices, enable vfio-pci_. In order to use virtual functions,
+SRIOV-support_ must be enabled.
+
+Execution of test with PCI passthrough with vswitch disabled:
+
+.. code-block:: console
+
+ $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
+ --vswtich none --vnf QemuPciPassthrough pvp_tput
+
+Any of supported guest-loopback-application_ can be used inside VM with
+PCI passthrough support.
+
+Note: Qemu with PCI passthrough support can be used only with PVP test
+deployment.
+
+.. _guest-loopback-application:
+
Selection of loopback application for PVP and PVVP tests
---------------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
To select loopback application, which will perform traffic forwarding
inside VM, following configuration parameter should be configured:
.. code-block:: console
$ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
- --test-param "guest_loopback=testpmd"
+ --test-params "guest_loopback=testpmd"
Supported loopback applications are:
will not be forwarded by VM and testcases with PVP and PVVP deployments
will fail. Guest loopback application is set to 'testpmd' by default.
+Multi-Queue Configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+VSPerf currently supports multi-queue with the following limitations:
+
+ 1. Execution of pvp/pvvp tests require testpmd as the loopback if multi-queue
+ is enabled at the guest.
+
+ 2. Requires QemuDpdkVhostUser as the vnf.
+
+ 3. Requires switch to be set to OvsDpdkVhost.
+
+ 4. Requires QEMU 2.5 or greater and any OVS version higher than 2.5. The
+ default upstream package versions installed by VSPerf satisfy this
+ requirement.
+
+To enable multi-queue modify the ''02_vswitch.conf'' file to enable multi-queue
+on the switch.
+
+ .. code-block:: console
+
+ VSWITCH_MULTI_QUEUES = 2
+
+**NOTE:** you should consider using the switch affinity to set a pmd cpu mask
+that can optimize your performance. Consider the numa of the NIC in use if this
+applies by checking /sys/class/net/<eth_name>/device/numa_node and setting an
+appropriate mask to create PMD threads on the same numa node.
+
+When multi-queue is enabled, each dpdk or dpdkvhostuser port that is created
+on the switch will set the option for multiple queues.
+
+To enable multi-queue on the guest modify the ''04_vnf.conf'' file.
+
+ .. code-block:: console
+
+ GUEST_NIC_QUEUES = 2
+
+Enabling multi-queue at the guest will add multiple queues to each NIC port when
+qemu launches the guest.
+
+Testpmd should be configured to take advantage of multi-queue on the guest. This
+can be done by modifying the ''04_vnf.conf'' file.
+
+ .. code-block:: console
+
+ GUEST_TESTPMD_CPU_MASK = '-l 0,1,2,3,4'
+
+ GUEST_TESTPMD_NB_CORES = 4
+ GUEST_TESTPMD_TXQ = 2
+ GUEST_TESTPMD_RXQ = 2
+
+**NOTE:** The guest SMP cores must be configured to allow for testpmd to use the
+optimal number of cores to take advantage of the multiple guest queues.
+
+**NOTE:** For optimal performance guest SMPs should be on the same numa as the
+NIC in use if possible/applicable. Testpmd should be assigned at least
+(nb_cores +1) total cores with the cpu mask.
+
Executing Packet Forwarding tests
------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To select application, which will perform packet forwarding,
following configuration parameter should be configured:
1. Update your ''10_custom.conf'' file to use the appropriate variables
for selected Packet Forwarder:
+
.. code-block:: console
# testpmd configuration
$ ./vsperf --conf-file <path_to_settings_py>
VSPERF modes of operation
--------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
VSPERF can be run in different modes. By default it will configure vSwitch,
traffic generator and VNF. However it can be used just for configuration
and execution of traffic generator. Another option is execution of all
"trafficgen-pause" - execute vSwitch and VNF but wait before traffic transmission
In case, that VSPERF is executed in "trafficgen" mode, then configuration
-of traffic generator should be configured through --test-param option.
+of traffic generator should be configured through --test-params option.
Supported CLI options useful for traffic generator configuration are:
.. code-block:: console
--test-params "traffic_type=continuous;bidirectional=True;iload=60"
Code change verification by pylint
-----------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
Every developer participating in VSPERF project should run
pylint before his python code is submitted for review. Project
specific configuration for pylint is available at 'pylint.rc'.
$ pylint --rcfile ./pylintrc ./vsperf
GOTCHAs:
---------
+^^^^^^^^
OVS with DPDK and QEMU
~~~~~~~~~~~~~~~~~~~~~~~
+
If you encounter the following error: "before (last 100 chars):
'-path=/dev/hugepages,share=on: unable to map backing store for
hugepages: Cannot allocate memory\r\n\r\n" with the PVP or PVVP
.. code-block:: console
VSWITCHD_DPDK_ARGS = ['-c', '0x4', '-n', '4', '--socket-mem 1024,0']
+ VSWITCHD_DPDK_CONFIG = {
+ 'dpdk-init' : 'true',
+ 'dpdk-lcore-mask' : '0x4',
+ 'dpdk-socket-mem' : '1024,0',
+ }
+
+Note: Option VSWITCHD_DPDK_ARGS is used for vswitchd, which supports --dpdk
+parameter. In recent vswitchd versions, option VSWITCHD_DPDK_CONFIG will be
+used to configure vswitchd via ovs-vsctl calls.
+
More information
-----------------
+^^^^^^^^^^^^^^^^
For more information and details refer to the vSwitchPerf user guide at:
http://artifacts.opnfv.org/vswitchperf/brahmaputra/userguide/index.html
Code Review / vswitchperf.git / blobdiff