^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If your ``10_custom.conf`` doesn't reside in the ``./conf`` directory
-of if you want to use an alternative configuration file, the file can
+or if you want to use an alternative configuration file, the file can
be passed to ``vsperf`` via the ``--conf-file`` argument.
.. code-block:: console
$ ./vsperf --conf-file <path_to_custom_conf> ...
-Note that configuration passed in via the environment (``--load-env``)
-or via another command line argument will override both the default and
-your custom configuration files. This "priority hierarchy" can be
-described like so (1 = max priority):
-
-1. Testcase definition section ``Parameters``
-2. Command line arguments
-3. Environment variables
-4. Configuration file(s)
-
-Further details about configuration files evaluation and special behaviour
+Evaluation of configuration parameters
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The value of configuration parameter can be specified at various places,
+e.g. at the test case definition, inside configuration files, by the command
+line argument, etc. Thus it is important to understand the order of configuration
+parameter evaluation. This "priority hierarchy" can be described like so
+(1 = max priority):
+
+1. Testcase definition keywords ``vSwitch``, ``Trafficgen``, ``VNF`` and ``Tunnel Type``
+2. Parameters inside testcase definition section ``Parameters``
+3. Command line arguments (e.g. ``--test-params``, ``--vswitch``, ``--trafficgen``, etc.)
+4. Environment variables (see ``--load-env`` argument)
+5. Custom configuration file specified via ``--conf-file`` argument
+6. Standard configuration files, where higher prefix number means higher
+ priority.
+
+For example, if the same configuration parameter is defined in custom configuration
+file (specified via ``--conf-file`` argument), via ``--test-params`` argument
+and also inside ``Parameters`` section of the testcase definition, then parameter
+value from the ``Parameters`` section will be used.
+
+Further details about order of configuration files evaluation and special behaviour
of options with ``GUEST_`` prefix could be found at :ref:`design document
<design-configuration>`.
.. _overriding-parameters-documentation:
-Referencing parameter values
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-It is possible to use a special macro ``#PARAM()`` to refer to the value of
-another configuration parameter. This reference is evaluated during
-access of the parameter value (by ``settings.getValue()`` call), so it
-can refer to parameters created during VSPERF runtime, e.g. NICS dictionary.
-It can be used to reflect DUT HW details in the testcase definition.
-
-Example:
-
-.. code:: python
-
- {
- ...
- "Name": "testcase",
- "Parameters" : {
- "TRAFFIC" : {
- 'l2': {
- # set destination MAC to the MAC of the first
- # interface from WHITELIST_NICS list
- 'dstmac' : '#PARAM(NICS[0]["mac"])',
- },
- },
- ...
-
Overriding values defined in configuration files
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
$ ./vsperf --test-params "TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,);" \
"GUEST_LOOPBACK=['testpmd','l2fwd']" pvvp_tput
+The ``--test-params`` command line argument can also be used to override default
+configuration values for multiple tests. Providing a list of parameters will apply each
+element of the list to the test with the same index. If more tests are run than
+parameters provided the last element of the list will repeat.
+
+.. code:: console
+
+ $ ./vsperf --test-params "['TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)',"
+ "'TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(64,)']" \
+ pvvp_tput pvvp_tput
+
The second option is to override configuration items by ``Parameters`` section
of the test case definition. The configuration items can be added into ``Parameters``
dictionary with their new values. These values will override values defined in
section of test case definition. It is also forbidden to redefine a value of
``TEST_PARAMS`` configuration item via CLI or ``Parameters`` section.
+**NOTE:** The new definition of the dictionary parameter, specified via ``--test-params``
+or inside ``Parameters`` section, will not override original dictionary values. Instead
+the original dictionary will be updated with values from the new dictionary definition.
+
+Referencing parameter values
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It is possible to use a special macro ``#PARAM()`` to refer to the value of
+another configuration parameter. This reference is evaluated during
+access of the parameter value (by ``settings.getValue()`` call), so it
+can refer to parameters created during VSPERF runtime, e.g. NICS dictionary.
+It can be used to reflect DUT HW details in the testcase definition.
+
+Example:
+
+.. code:: python
+
+ {
+ ...
+ "Name": "testcase",
+ "Parameters" : {
+ "TRAFFIC" : {
+ 'l2': {
+ # set destination MAC to the MAC of the first
+ # interface from WHITELIST_NICS list
+ 'dstmac' : '#PARAM(NICS[0]["mac"])',
+ },
+ },
+ ...
+
vloop_vnf
^^^^^^^^^
support for Destination Network Address Translation (DNAT) for both the MAC and
IP addresses. l2fwd can be found in <vswitchperf_dir>/src/l2fwd
+Additional Tools Setup
+^^^^^^^^^^^^^^^^^^^^^^
+
+Follow the `Additional tools instructions <additional-tools-configuration>` to
+install and configure additional tools such as collectors and loadgens.
+
Executing tests
^^^^^^^^^^^^^^^
Where $TESTNAME is the name of the vsperf test you would like to run.
+To run a test multiple times, repeat it:
+
+.. code-block:: console
+
+ $ ./vsperf $TESTNAME $TESTNAME $TESTNAME
+
To run a group of tests, for example all tests with a name containing
'RFC2544':
--tests RFC2544Tput \
--test-params "TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)"
+To specify configurable parameters for multiple tests, use a list of
+parameters. One element for each test.
+
+.. code:: console
+
+ $ ./vsperf --conf-file user_settings.py \
+ --test-params "['TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)',"\
+ "'TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(64,)']" \
+ phy2phy_cont phy2phy_cont
+
+If the ``CUMULATIVE_PARAMS`` setting is set to True and there are different parameters
+provided for each test using ``--test-params``, each test will take the parameters of
+the previous test before appyling it's own.
+With ``CUMULATIVE_PARAMS`` set to True the following command will be equivalent to the
+previous example:
+
+.. code:: console
+
+ $ ./vsperf --conf-file user_settings.py \
+ --test-params "['TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)',"\
+ "'TRAFFICGEN_PKT_SIZES=(64,)']" \
+ phy2phy_cont phy2phy_cont
+ "
+
For all available options, check out the help dialog:
.. code-block:: console
**NOTE:** Please ensure your boot/grub parameters include
the following:
-**NOTE:** In case of VPP, it is required to explicitly define, that vfio-pci
-DPDK driver should be used. It means to update dpdk part of VSWITCH_VPP_ARGS
-dictionary with uio-driver section, e.g. VSWITCH_VPP_ARGS['dpdk'] = 'uio-driver vfio-pci'
-
.. code-block:: console
iommu=pt intel_iommu=on
[ 3.335746] IOMMU: dmar1 using Queued invalidation
....
+**NOTE:** In case of VPP, it is required to explicitly define, that vfio-pci
+DPDK driver should be used. It means to update dpdk part of VSWITCH_VPP_ARGS
+dictionary with uio-driver section, e.g. VSWITCH_VPP_ARGS['dpdk'] = 'uio-driver vfio-pci'
+
.. _SRIOV-support:
Using SRIOV support
.. code-block:: console
- 'uio_pci_generic' - Use uio_pci_generic driver
+ 'uio_pci_generic' - Use uio_pci_generic driver
'igb_uio_from_src' - Build and use the igb_uio driver from the dpdk src
files
'vfio_no_iommu' - Use vfio with no iommu option. This requires custom
$ ./vsperf -m trafficgen --trafficgen IxNet --conf-file vsperf.conf \
--test-params "TRAFFIC={'traffic_type':'rfc2544_continuous','bidir':'False','framerate':60}"
+Performance Matrix
+^^^^^^^^^^^^^^^^^^
+
+The ``--matrix`` command line argument analyses and displays the performance of
+all the tests run. Using the metric specified by ``MATRIX_METRIC`` in the conf-file,
+the first test is set as the baseline and all the other tests are compared to it.
+The ``MATRIX_METRIC`` must always refer to a numeric value to enable comparision.
+A table, with the test ID, metric value, the change of the metric in %, testname
+and the test parameters used for each test, is printed out as well as saved into the
+results directory.
+
+Example of 2 tests being compared using Performance Matrix:
+
+.. code-block:: console
+
+ $ ./vsperf --conf-file user_settings.py \
+ --test-params "['TRAFFICGEN_PKT_SIZES=(64,)',"\
+ "'TRAFFICGEN_PKT_SIZES=(128,)']" \
+ phy2phy_cont phy2phy_cont --matrix
+
+Example output:
+
+.. code-block:: console
+
+ +------+--------------+---------------------+----------+---------------------------------------+
+ | ID | Name | throughput_rx_fps | Change | Parameters, CUMULATIVE_PARAMS = False |
+ +======+==============+=====================+==========+=======================================+
+ | 0 | phy2phy_cont | 23749000.000 | 0 | 'TRAFFICGEN_PKT_SIZES': [64] |
+ +------+--------------+---------------------+----------+---------------------------------------+
+ | 1 | phy2phy_cont | 16850500.000 | -29.048 | 'TRAFFICGEN_PKT_SIZES': [128] |
+ +------+--------------+---------------------+----------+---------------------------------------+
+
+
Code change verification by pylint
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Code Review / vswitchperf.git / blobdiff