1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
5 .. _Dashboard006: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc006
6 .. _Dashboard007: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc007
7 .. _Dashboard020: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc020
8 .. _Dashboard021: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc021
9 .. _DashboardVTC: http://testresults.opnfv.org/grafana/dashboard/db/vtc-dashboard
10 ====================================
11 Test Results for yardstick-opnfv-vtc
12 ====================================
21 .. after this doc is filled, remove all comments and include the scenario in
22 .. results.rst by removing the comment on the file name.
25 Overview of test results
26 ------------------------
28 .. general on metrics collected, number of iterations
30 The virtual Traffic Classifier (vtc) Scenario supported by Yardstick is used by 4 Test Cases:
40 TC006 is the Virtual Traffic Classifier Data Plane Throughput Benchmarking Test.
41 It collects measures about the end-to-end throughput supported by the
42 virtual Traffic Classifier (vTC).
43 Results of the test are shown in the Dashboard006_
44 The throughput is expressed as percentage of the available bandwidth on the NIC.
49 TC007 is the Virtual Traffic Classifier Data Plane Throughput Benchmarking in presence of
51 It collects measures about the end-to-end throughput supported by the
52 virtual Traffic Classifier when a user-defined number of noisy neighbors is deployed.
53 Results of the test are shown in the Dashboard007_
54 The throughput is expressed as percentage of the available bandwidth on the NIC.
59 TC020 is the Virtual Traffic Classifier Instantiation Test.
60 It verifies that a newly instantiated vTC is alive and functional and its instantiation
61 is correctly supported by the underlying infrastructure.
62 Results of the test are shown in the Dashboard020_
67 TC021 is the Virtual Traffic Classifier Instantiation in presence of noisy neighbors Test.
68 It verifies that a newly instantiated vTC is alive and functional and its instantiation
69 is correctly supported by the underlying infrastructure when noisy neighbors are present.
70 Results of the test are shown in the Dashboard021_
74 In the Generic scenario the Virtual Traffic Classifier is running on a standard Openstack
75 setup and traffic is being replayed from a neighbor VM. The traffic sent contains
76 various protocols and applications, and the VTC identifies them and exports the data.
77 Results of the test are shown in the DashboardVTC.
84 The results for TC006 have been obtained using the following test case
88 - Scenario: vtc_throughput
89 - Network Techology: SR-IOV
90 - vTC Flavor: m1.large
95 The results for TC007 have been obtained using the following test case
99 - Scenario: vtc_throughput_noisy
100 - Network Techology: SR-IOV
101 - vTC Flavor: m1.large
102 - Number of noisy neighbors: 2
103 - Number of cores per neighbor: 2
104 - Amount of RAM per neighbor: 1G
109 The results for TC020 have been obtained using the following test case
112 The results listed in previous section have been obtained using the following
113 test case configuration:
116 - Scenario: vtc_instantiation_validation
117 - Network Techology: SR-IOV
118 - vTC Flavor: m1.large
123 The results listed in previous section have been obtained using the following
124 test case configuration:
127 - Scenario: vtc_instantiation_validation
128 - Network Techology: SR-IOV
129 - vTC Flavor: m1.large
130 - Number of noisy neighbors: 2
131 - Number of cores per neighbor: 2
132 - Amount of RAM per neighbor: 1G
135 For all the test cases, the user can specify different values for the parameters.
139 The results listed in the previous section have been obtained, using a
140 standard Openstack setup.
141 The user can replay his/her own traffic and see the corresponding results.
143 Rationale for decisions
144 -----------------------
148 The result of the test is a number between 0 and 100 which represents the percentage of bandwidth
149 available on the NIC that corresponds to the supported throughput by the vTC.
154 The result of the test is a number between 0 and 100 which represents the percentage of bandwidth
155 available on the NIC that corresponds to the supported throughput by the vTC.
159 The execution of the test is done as described in the following:
161 - The vTC is deployed on the OpenStack testbed;
162 - Some traffic is sent to the vTC;
163 - The vTC changes the header of the packets and sends them back to the packet generator;
164 - The packet generator checks that all the packets are received correctly and have been changed
165 correctly by the vTC.
167 The test is declared as PASSED if all the packets are correcly received by the packet generator
168 and they have been modified by the virtual Traffic Classifier as required.
173 The execution of the test is done as described in the following:
175 - The vTC is deployed on the OpenStack testbed;
176 - The noisy neighbors are deployed as requested by the user;
177 - Some traffic is sent to the vTC;
178 - The vTC change the header of the packets and sends them back to the packet generator;
179 - The packet generator checks that all the packets are received correctly and have been changed
182 The test is declared as PASSED if all the packets are correcly received by the packet generator
183 and they have been modified by the virtual Traffic Classifier as required.
187 The execution of the test consists of the following actions:
189 - The vTC is deployed on the OpenStack testbed;
190 - The traffic generator VM is deployed on the Openstack Testbed;
191 - Traffic data are relevant to the network setup;
192 - Traffic is sent to the vTC;
196 Conclusions and recommendations
197 -------------------------------
201 The obtained results show that the virtual Traffic Classifier can support up to 4 Gbps
202 (40% of the available bandwidth) correspond to the expected behaviour of the virtual
204 Using the configuration with SR-IOV and large flavor, the expected throughput should
205 generally be in the range between 3 and 4 Gbps.
210 These results correspond to the configuration in which the virtual Traffic Classifier uses SR-IOV
211 Virtual Functions and the flavor is set to large for the virtual machine.
212 The throughput is in the range between 2.5 Gbps and 3.7 Gbps.
213 This shows that the effect of 2 noisy neighbors reduces the throughput of
214 the service between 10 and 20%.
215 Increasing number of neihbours would have a higher impact on the performance.
220 The obtained results correspond to the expected behaviour of the virtual Traffic Classifier.
221 Using the configuration with SR-IOV and large flavor, the expected result is that the vTC is
222 correctly instantiated, it is able to receive and send packets using SR-IOV technology
223 and to forward packets back to the packet generator changing the TCP/IP header as required.
228 The obtained results correspond to the expected behaviour of the virtual Traffic Classifier.
229 Using the configuration with SR-IOV and large flavor, the expected result is that the vTC is
230 correctly instantiated, it is able to receive and send packets using SR-IOV technology
231 and to forward packets back to the packet generator changing the TCP/IP header as required,
232 also in presence of noisy neighbors.
236 The obtained results correspond to the expected behaviour of the virtual Traffic Classifier.
237 Using the aforementioned configuration the expected application protocols are identified
238 and their traffic statistics are demonstrated in the DashboardVTC, a group of popular
239 applications is selected to demonstrate the sound operation of the vTC.
240 The demonstrated application protocols are: