1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
3 .. (c) OPNFV, Intel Corporation, AT&T and others.
5 Execution of vswitchperf testcases by Yardstick
6 -----------------------------------------------
11 Yardstick is a generic framework for a test execution, which is used for
12 validation of installation of OPNFV platform. In the future, Yardstick will
13 support two options of vswitchperf testcase execution:
15 - plugin mode, which will execute native vswitchperf testcases; Tests will
16 be executed natively by vsperf, and test results will be processed and
17 reported by yardstick.
18 - traffic generator mode, which will run vswitchperf in **trafficgen**
19 mode only; Yardstick framework will be used to launch VNFs and to configure
20 flows to ensure, that traffic is properly routed. This mode will allow to
21 test OVS performance in real world scenarios.
23 In Colorado release only the traffic generator mode is supported.
25 Yardstick Installation
26 ^^^^^^^^^^^^^^^^^^^^^^
28 In order to run Yardstick testcases, you will need to prepare your test
29 environment. Please follow the `installation instructions
30 <http://artifacts.opnfv.org/yardstick/brahmaputra/docs/user_guides_framework/index.html>`__
31 to install the yardstick.
33 Please note, that yardstick uses OpenStack for execution of testcases.
34 OpenStack must be installed with Heat and Neutron services. Otherwise
35 vswitchperf testcases cannot be executed.
37 Vswitchperf VM image preparation
38 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
40 In general, any Linux distribution supported by vswitchperf can be used as
41 a base image for vswitchperf. One of the possibilities is to modify vloop-vnf
42 image, which can be downloaded from `<http://artifacts.opnfv.org/>`__.
44 .. code-block:: console
46 $ wget http://artifacts.opnfv.org/vswitchperf/vloop-vnf-ubuntu-14.04_20151216.qcow2
48 Please follow the `installation instructions
49 <http://artifacts.opnfv.org/vswitchperf/docs/configguide/installation.html>`__ to
50 install vswitchperf inside vloop-vnf image. As vswitchperf will be run in
51 trafficgen mode, it is possible to skip installation and compilation of OVS,
52 QEMU and DPDK to keep image size smaller.
54 In case, that selected traffic generator requires installation of additional
55 client software, please follow appropriate documentation. For example in case
56 of IXIA, you would need to install IxOS and IxNetowrk TCL API.
58 Final image with vswitchperf must be uploaded into the glance service and
59 vswitchperf specific flavor configured, e.g.:
61 .. code-block:: console
63 $ glance --os-username admin --os-image-api-version 1 image-create --name
64 vsperf --is-public true --disk-format qcow2 --container-format bare --file
67 $ nova --os-username admin flavor-create vsperf-flavor 100 2048 25 1
69 Testcase customization
70 ^^^^^^^^^^^^^^^^^^^^^^
72 Yardstick testcases are described by YAML files. vswitchperf specific testcases
73 are part of the vswitchperf repository and their yaml files can be found at
74 ``yardstick/tests`` directory. For detailed description of yaml file structure,
75 please see yardstick documentation and testcase samples. Only vswitchperf specific
76 parts will be discussed here.
87 testname: 'rfc2544_p2p_tput'
88 traffic_type: 'rfc2544'
93 trafficgen_port1: 'eth1'
94 trafficgen_port2: 'eth3'
95 external_bridge: 'br-ex'
96 conf-file: '~/vsperf-yardstick.conf'
102 scenario_option_name: pkt_sizes
110 metrics: 'throughput_rx_fps'
111 throughput_rx_fps: 500000
120 Section **option** defines details of vswitchperf test scenario. Lot of options
121 are identical to the vswitchperf parameters passed through ``--test-params``
122 argument. Following options are supported:
124 - **traffic_type** - specifies the type of traffic executed by traffic generator;
125 valid values are "rfc2544", "continuous" and "back2back"; Default: 'rfc2544'
126 - **pkt_sizes** - a packet size for which test should be executed;
127 Multiple packet sizes can be tested by modification of Sequence runner
128 section inside YAML definition. Default: '64'
129 - **duration** - sets duration for which traffic will be generated; Default: 30
130 - **bidirectional** - specifies if traffic will be uni (False) or bi-directional
131 (True); Default: False
132 - **iload** - specifies frame rate; Default: 100
133 - **rfc2544_trials** - specifies the number of trials performed for each packet
135 - **multistream** - specifies the number of simulated streams; Default: 0 (i.e.
136 multistream feature is disabled)
137 - **stream_type** - specifies network layer used for multistream simulation
138 the valid values are "L4", "L3" and "L2"; Default: 'L4'
139 - **conf-file** - sets path to the vswitchperf configuration file, which will be
140 uploaded to VM; Default: '~/vsperf-yardstick.conf'
141 - **setup-script** - sets path to the setup script, which will be executed
142 during setup and teardown phases
143 - **trafficgen_port1** - specifies device name of 1st interface connected to
145 - **trafficgen_port2** - specifies device name of 2nd interface connected to
147 - **external_bridge** - specifies name of external bridge configured in OVS;
150 In case that **trafficgen_port1** and/or **trafficgen_port2** are defined, then
151 these interfaces will be inserted into the **external_bridge** of OVS. It is
152 expected, that OVS runs at the same node, where the testcase is executed. In case
153 of more complex OpenStack installation or a need of additional OVS configuration,
154 **setup-script** can be used.
156 Note: It is essential to prepare customized configuration file for the vsperf
157 and to specify its name by **conf-file** option. Config file must specify, which
158 traffic generator will be used and configure traffic generator specific options.
163 Yardstick supports several `runner types
164 <http://artifacts.opnfv.org/yardstick/brahmaputra/docs/userguide/architecture.html#runner-types>`__.
165 In case of vswitchperf specific TCs, **Sequence** runner type can be used to
166 execute the testcase for given list of packet sizes.
172 In case that sla section is not defined, then testcase will be always
173 considered as successful. On the other hand, it is possible to define a set of
174 test metrics and their minimal values to evaluate test success. Any numeric
175 value, reported by vswitchperf inside CSV result file, can be used.
176 Multiple metrics can be defined as a coma separated list of items. Minimal
177 value must be set separately for each metric.
184 metrics: 'throughput_rx_fps,throughput_rx_mbps'
185 throughput_rx_fps: 500000
186 throughput_rx_mbps: 1000
188 In case that any of defined metrics will be lower than defined value, then
189 testcase will be marked as failed. Based on ``action`` policy, yardstick
190 will either stop test execution (value ``assert``) or it will run next test
196 After installation, yardstick is available as python package within yardstick
197 specific virtual environment. It means, that before test execution yardstick
198 environment must be enabled, e.g.:
200 .. code-block:: console
202 source ~/yardstick_venv/bin/activate
205 Next step is configuration of OpenStack environment, e.g. in case of devstack:
207 .. code-block:: console
209 source /opt/openstack/devstack/openrc
210 export EXTERNAL_NETWORK=public
212 Vswitchperf testcases executable by yardstick are located at vswitchperf
213 repository inside ``yardstick/tests`` directory. Example of their download
214 and execution follows:
216 .. code-block:: console
218 git clone https://gerrit.opnfv.org/gerrit/vswitchperf
221 yardstick -d task start yardstick/tests/p2p_cont.yaml
223 Note: Optional argument ``-d`` shows debug output.