Traffic generator modules.
- Moongen software traffic generator. Requires a separate machine running
moongen to execute packet generation.
+- T-Rex software traffic generator. Requires a separate machine running T-Rex
+ Server to execute packet generation.
If you want to use another traffic generator, please select the :ref:`trafficgen-dummy`
generator.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If your ``10_custom.conf`` doesn't reside in the ``./conf`` directory
-of if you want to use an alternative configuration file, the file can
+or if you want to use an alternative configuration file, the file can
be passed to ``vsperf`` via the ``--conf-file`` argument.
.. code-block:: console
$ ./vsperf --conf-file <path_to_custom_conf> ...
-Note that configuration passed in via the environment (``--load-env``)
-or via another command line argument will override both the default and
-your custom configuration files. This "priority hierarchy" can be
-described like so (1 = max priority):
-
-1. Testcase definition section ``Parameters``
-2. Command line arguments
-3. Environment variables
-4. Configuration file(s)
-
-Further details about configuration files evaluation and special behaviour
+Evaluation of configuration parameters
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The value of configuration parameter can be specified at various places,
+e.g. at the test case definition, inside configuration files, by the command
+line argument, etc. Thus it is important to understand the order of configuration
+parameter evaluation. This "priority hierarchy" can be described like so
+(1 = max priority):
+
+1. Testcase definition keywords ``vSwitch``, ``Trafficgen``, ``VNF`` and ``Tunnel Type``
+2. Parameters inside testcase definition section ``Parameters``
+3. Command line arguments (e.g. ``--test-params``, ``--vswitch``, ``--trafficgen``, etc.)
+4. Environment variables (see ``--load-env`` argument)
+5. Custom configuration file specified via ``--conf-file`` argument
+6. Standard configuration files, where higher prefix number means higher
+ priority.
+
+For example, if the same configuration parameter is defined in custom configuration
+file (specified via ``--conf-file`` argument), via ``--test-params`` argument
+and also inside ``Parameters`` section of the testcase definition, then parameter
+value from the ``Parameters`` section will be used.
+
+Further details about order of configuration files evaluation and special behaviour
of options with ``GUEST_`` prefix could be found at :ref:`design document
<design-configuration>`.
$ ./vsperf --test-params "TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,);" \
"GUEST_LOOPBACK=['testpmd','l2fwd']" pvvp_tput
+The ``--test-params`` command line argument can also be used to override default
+configuration values for multiple tests. Providing a list of parameters will apply each
+element of the list to the test with the same index. If more tests are run than
+parameters provided the last element of the list will repeat.
+
+.. code:: console
+
+ $ ./vsperf --test-params "['TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)',"
+ "'TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(64,)']" \
+ pvvp_tput pvvp_tput
+
The second option is to override configuration items by ``Parameters`` section
of the test case definition. The configuration items can be added into ``Parameters``
dictionary with their new values. These values will override values defined in
section of test case definition. It is also forbidden to redefine a value of
``TEST_PARAMS`` configuration item via CLI or ``Parameters`` section.
+**NOTE:** The new definition of the dictionary parameter, specified via ``--test-params``
+or inside ``Parameters`` section, will not override original dictionary values. Instead
+the original dictionary will be updated with values from the new dictionary definition.
+
+Referencing parameter values
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It is possible to use a special macro ``#PARAM()`` to refer to the value of
+another configuration parameter. This reference is evaluated during
+access of the parameter value (by ``settings.getValue()`` call), so it
+can refer to parameters created during VSPERF runtime, e.g. NICS dictionary.
+It can be used to reflect DUT HW details in the testcase definition.
+
+Example:
+
+.. code:: python
+
+ {
+ ...
+ "Name": "testcase",
+ "Parameters" : {
+ "TRAFFIC" : {
+ 'l2': {
+ # set destination MAC to the MAC of the first
+ # interface from WHITELIST_NICS list
+ 'dstmac' : '#PARAM(NICS[0]["mac"])',
+ },
+ },
+ ...
+
vloop_vnf
^^^^^^^^^
support for Destination Network Address Translation (DNAT) for both the MAC and
IP addresses. l2fwd can be found in <vswitchperf_dir>/src/l2fwd
+Additional Tools Setup
+^^^^^^^^^^^^^^^^^^^^^^
+
+Follow the `Additional tools instructions <additional-tools-configuration>` to
+install and configure additional tools such as collectors and loadgens.
+
Executing tests
^^^^^^^^^^^^^^^
Where $TESTNAME is the name of the vsperf test you would like to run.
+To run a test multiple times, repeat it:
+
+.. code-block:: console
+
+ $ ./vsperf $TESTNAME $TESTNAME $TESTNAME
+
To run a group of tests, for example all tests with a name containing
'RFC2544':
--tests RFC2544Tput \
--test-params "TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)"
+To specify configurable parameters for multiple tests, use a list of
+parameters. One element for each test.
+
+.. code:: console
+
+ $ ./vsperf --conf-file user_settings.py \
+ --test-params "['TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)',"\
+ "'TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(64,)']" \
+ phy2phy_cont phy2phy_cont
+
+If the ``CUMULATIVE_PARAMS`` setting is set to True and there are different parameters
+provided for each test using ``--test-params``, each test will take the parameters of
+the previous test before appyling it's own.
+With ``CUMULATIVE_PARAMS`` set to True the following command will be equivalent to the
+previous example:
+
+.. code:: console
+
+ $ ./vsperf --conf-file user_settings.py \
+ --test-params "['TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)',"\
+ "'TRAFFICGEN_PKT_SIZES=(64,)']" \
+ phy2phy_cont phy2phy_cont
+ "
+
For all available options, check out the help dialog:
.. code-block:: console
To run tests using vhost-user as guest access method:
-1. Set VHOST_METHOD and VNF of your settings file to:
+1. Set VSWITCH and VNF of your settings file to:
.. code-block:: python
$ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
+**NOTE:** By default vSwitch is acting as a server for dpdk vhost-user sockets.
+In case, that QEMU should be a server for vhost-user sockets, then parameter
+``VSWITCH_VHOSTUSER_SERVER_MODE`` should be set to ``False``.
+
Executing tests with VMs using Vanilla OVS
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 3.335746] IOMMU: dmar1 using Queued invalidation
....
+**NOTE:** In case of VPP, it is required to explicitly define, that vfio-pci
+DPDK driver should be used. It means to update dpdk part of VSWITCH_VPP_ARGS
+dictionary with uio-driver section, e.g. VSWITCH_VPP_ARGS['dpdk'] = 'uio-driver vfio-pci'
+
.. _SRIOV-support:
Using SRIOV support
* vSwitch tests with DPDK or without DPDK support to verify impact
of VF usage on vSwitch performance
-* tests without vSwitch, where traffic is forwared directly
+* tests without vSwitch, where traffic is forwarded directly
between VF interfaces by packet forwarder (e.g. testpmd application)
* tests without vSwitch, where VM accesses VF interfaces directly
by PCI-passthrough_ to measure raw VM throughput performance.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Raw virtual machine throughput performance can be measured by execution of PVP
-test with direct access to NICs by PCI passthrough. To execute VM with direct
+test with direct access to NICs by PCI pass-through. To execute VM with direct
access to PCI devices, enable vfio-pci_. In order to use virtual functions,
SRIOV-support_ must be enabled.
-Execution of test with PCI passthrough with vswitch disabled:
+Execution of test with PCI pass-through with vswitch disabled:
.. code-block:: console
--vswitch none --vnf QemuPciPassthrough pvp_tput
Any of supported guest-loopback-application_ can be used inside VM with
-PCI passthrough support.
+PCI pass-through support.
-Note: Qemu with PCI passthrough support can be used only with PVP test
+Note: Qemu with PCI pass-through support can be used only with PVP test
deployment.
.. _guest-loopback-application:
.. code-block:: console
- 'uio_pci_generic' - Use uio_pci_generic driver
+ 'uio_pci_generic' - Use uio_pci_generic driver
'igb_uio_from_src' - Build and use the igb_uio driver from the dpdk src
files
'vfio_no_iommu' - Use vfio with no iommu option. This requires custom
Please refer to the dpdk documents at http://dpdk.org/doc/guides for more
information on these drivers.
+Guest Core and Thread Binding
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+VSPERF provides options to achieve better performance by guest core binding and
+guest vCPU thread binding as well. Core binding is to bind all the qemu threads.
+Thread binding is to bind the house keeping threads to some CPU and vCPU thread to
+some other CPU, this helps to reduce the noise from qemu house keeping threads.
+
+
+.. code-block:: python
+
+ GUEST_CORE_BINDING = [('#EVAL(6+2*#VMINDEX)', '#EVAL(7+2*#VMINDEX)')]
+
+**NOTE** By default the GUEST_THREAD_BINDING will be none, which means same as
+the GUEST_CORE_BINDING, i.e. the vcpu threads are sharing the physical CPUs with
+the house keeping threads. Better performance using vCPU thread binding can be
+achieved by enabling affinity in the custom configuration file.
+
+For example, if an environment requires 32,33 to be core binded and 29,30&31 for
+guest thread binding to achieve better performance.
+
+.. code-block:: python
+
+ VNF_AFFINITIZATION_ON = True
+ GUEST_CORE_BINDING = [('32','33')]
+ GUEST_THREAD_BINDING = [('29', '30', '31')]
+
+Qemu CPU features
+^^^^^^^^^^^^^^^^^
+
+QEMU default to a compatible subset of performance enhancing cpu features.
+To pass all available host processor features to the guest.
+
+.. code-block:: python
+
+ GUEST_CPU_OPTIONS = ['host,migratable=off']
+
+**NOTE** To enhance the performance, cpu features tsc deadline timer for guest,
+the guest PMU, the invariant TSC can be provided in the custom configuration file.
+
Multi-Queue Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^
on the same numa as the NIC in use if possible/applicable. Testpmd should be
assigned at least (nb_cores +1) total cores with the cpu mask.
+Jumbo Frame Testing
+^^^^^^^^^^^^^^^^^^^
+
+VSPERF provides options to support jumbo frame testing with a jumbo frame supported
+NIC and traffic generator for the following vswitches:
+
+1. OVSVanilla
+
+2. OvsDpdkVhostUser
+
+3. TestPMD loopback with or without a guest
+
+**NOTE:** There is currently no support for SR-IOV or VPP at this time with jumbo
+frames.
+
+All packet forwarding applications for pxp testing is supported.
+
+To enable jumbo frame testing simply enable the option in the conf files and set the
+maximum size that will be used.
+
+.. code-block:: python
+
+ VSWITCH_JUMBO_FRAMES_ENABLED = True
+ VSWITCH_JUMBO_FRAMES_SIZE = 9000
+
+To enable jumbo frame testing with OVSVanilla the NIC in test on the host must have
+its mtu size changed manually using ifconfig or applicable tools:
+
+.. code-block:: console
+
+ ifconfig eth1 mtu 9000 up
+
+**NOTE:** To make the setting consistent across reboots you should reference the OS
+documents as it differs from distribution to distribution.
+
+To start a test for jumbo frames modify the conf file packet sizes or pass the option
+through the VSPERF command line.
+
+.. code-block:: python
+
+ TEST_PARAMS = {'TRAFFICGEN_PKT_SIZES':(2000,9000)}
+
+.. code-block:: python
+
+ ./vsperf --test-params "TRAFFICGEN_PKT_SIZES=2000,9000"
+
+It is recommended to increase the memory size for OvsDpdkVhostUser testing from the default
+1024. Your size required may vary depending on the number of guests in your testing. 4096
+appears to work well for most typical testing scenarios.
+
+.. code-block:: python
+
+ DPDK_SOCKET_MEM = ['4096', '0']
+
+**NOTE:** For Jumbo frames to work with DpdkVhostUser, mergable buffers will be enabled by
+default. If testing with mergable buffers in QEMU is desired, disable Jumbo Frames and only
+test non jumbo frame sizes. Test Jumbo Frames sizes separately to avoid this collision.
+
+
Executing Packet Forwarding tests
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
$ ./vsperf -m trafficgen --trafficgen IxNet --conf-file vsperf.conf \
--test-params "TRAFFIC={'traffic_type':'rfc2544_continuous','bidir':'False','framerate':60}"
+Performance Matrix
+^^^^^^^^^^^^^^^^^^
+
+The ``--matrix`` command line argument analyses and displays the performance of
+all the tests run. Using the metric specified by ``MATRIX_METRIC`` in the conf-file,
+the first test is set as the baseline and all the other tests are compared to it.
+The ``MATRIX_METRIC`` must always refer to a numeric value to enable comparision.
+A table, with the test ID, metric value, the change of the metric in %, testname
+and the test parameters used for each test, is printed out as well as saved into the
+results directory.
+
+Example of 2 tests being compared using Performance Matrix:
+
+.. code-block:: console
+
+ $ ./vsperf --conf-file user_settings.py \
+ --test-params "['TRAFFICGEN_PKT_SIZES=(64,)',"\
+ "'TRAFFICGEN_PKT_SIZES=(128,)']" \
+ phy2phy_cont phy2phy_cont --matrix
+
+Example output:
+
+.. code-block:: console
+
+ +------+--------------+---------------------+----------+---------------------------------------+
+ | ID | Name | throughput_rx_fps | Change | Parameters, CUMULATIVE_PARAMS = False |
+ +======+==============+=====================+==========+=======================================+
+ | 0 | phy2phy_cont | 23749000.000 | 0 | 'TRAFFICGEN_PKT_SIZES': [64] |
+ +------+--------------+---------------------+----------+---------------------------------------+
+ | 1 | phy2phy_cont | 16850500.000 | -29.048 | 'TRAFFICGEN_PKT_SIZES': [128] |
+ +------+--------------+---------------------+----------+---------------------------------------+
+
+
Code change verification by pylint
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Code Review / vswitchperf.git / blobdiff