1 # CHARACTERIZE VSWITCH PERFORMANCE FOR TELCO NFV USE CASES LEVEL TEST REPORT
5 - [1. Introduction](#Introduction)
6 - [1.1. Document identifier](#DocId)
8 - [1.3. References](#References)
9 - [2. Details of the Level Test Report](#DetailsoftheLevelTestReport)
10 - [2.1. Overview of test results](#OverviewOfTestResults)
11 - [2.2. Detailed test results](#DetailedTestResults)
12 - [2.3. Rationale for decisions](#RationaleForDecisions)
13 - [2.4. Conclusions and recommendations](#ConclusionsandRecommendations)
14 - [3. General](#General)
15 - [3.1. Glossary](#Glossary)
16 - [3.2. Document change procedures and history](#DocChangeProceduresandHistory)
20 <a name="Introduction"></a>
23 The objective of the OPNFV project titled **"Characterise vSwitch Performance
24 for Telco NFV Use Cases"**, is to evaluate a virtual switch to identify its
25 suitability for a Telco Network Function Virtualization (NFV) environment. As
26 well as this, the project aims to identify any gaps or bottlenecks in order to
27 drive architectural changes to improve virtual switch performance and
28 determinism. The purpose of this document is to summarize the results of the
29 tests carried out on the virtual switch in the Network Function Virtualization
30 Infrastructure (NFVI) and, from these results, provide evaluations and
31 recommendations for the virtual switch. Test results will be outlined in
32 [Details of the Level Test Report](#DetailsoftheLevelTestReport), preceded by
33 the [Document Identifier](#DocId), [Scope](#Scope) and
34 [References](#References).
36 This document is currently in draft form.
39 ### 1.1. Document identifier
41 The document id will be used to uniquely identify versions of the LTR. The
42 format for the document id will be:
43 `OPNFV_vswitchperf_LTR_ver_NUM_MONTH_YEAR_AUTHOR_STATUS`, where by the AUTHOR
44 field should be replaced with the initials of the author and the status is one
45 of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of
46 the LTR is: `OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT`.
51 The scope of this report is to detail the results of the tests that have been
52 performed on the virtual switch. This report will also evaluate the results of
53 these tests and, based on these evaluations, provide recommendations on the
54 suitability of the virtual switch for use in a Telco NFV environment.
56 <a name="References"></a>
59 - `OPNFV_vswitchperf_LTD_ver_1.6_Jan_15_DRAFT`
63 <a name="DetailsoftheLevelTestReport"></a>
64 ## 2. Details of the Level Test Report
66 This section provides an overview of the test results ([Section
67 2.1.](#OveriewOfTestResults)) as well as detailed test results for each test
68 ([Section 2.2.](#DetailedTestResults)). Also included are the rationale used to
69 evaluate each test ([Section 2.3.](#RationaleForDecisions)) and the conclusions
70 and recommendations for each test ([Section
71 2.4.](#ConclusionsandRecommendations)).
73 <a name="OverviewOfTestResults"></a>
74 ### 2.1. Overview of test results
76 ##### Test Environment
78 Below is the environment that the test was performed in:
80 - OS: {{tests[0].env.os}}
81 - Kernel Version: {{tests[0].env.kernel}}
82 - NIC(s):{% for nic in tests[0].env.nics %}
84 - Board: {{tests[0].env.platform}}
85 - CPU: {{tests[0].env.cpu}}
86 - CPU cores: {{tests[0].env.cpu_cores}}
87 - Memory: {{tests[0].env.memory}}
88 - Virtual Switch Set-up: {{tests[0].deployment}}
89 - vswitchperf: GIT tag: {{tests[0].env.vsperf['git_tag']}}
90 - Traffic Generator: {{tests[0].env.traffic_gen['name']}}, Version: {{tests[0].env.traffic_gen['version']}}, GIT tag: {{tests[0].env.traffic_gen['git_tag']}}
91 - vSwitch: {{tests[0].env.vswitch['name']}}, Version: {{tests[0].env.vswitch['version']}}, GIT tag: {{tests[0].env.vswitch['git_tag']}}
92 {%- if 'dpdk' in tests[0].env %}
93 - DPDK Version: {{tests[0].env.dpdk['version']}}, GIT tag: {{tests[0].env.dpdk['git_tag']}}
95 {%- if 'vnf' in tests[0].env %}
96 - VNF: {{tests[0].env.vnf['name']}}, Version: {{tests[0].env.vnf['version']}}, GIT tag: {{tests[0].env.vnf['git_tag']}}
97 - VM images:{% for guest_image in tests[0].env.guest_image %}
98 - {{guest_image}}{% endfor %}
99 - VM loopback apps:{% for loopback_app in tests[0].env.loopback_app %}
100 - {{loopback_app['name']}}, Version: {{loopback_app['version']}}, GIT tag: {{loopback_app['git_tag']}}{% endfor %}
103 For each test, a summary of the key test results is provided.
104 {% for test in tests %}
105 #### Test ID: {{ test.ID }}
107 Below are test details:
109 - Test ID: {{ "%s"|format(test.id) }}
110 - Description: {{ "%s"|format(test.conf['Description']) }}
111 - Deployment: {{ "%s"|format(test.deployment) }}
112 - Traffic type: {{ "%s"|format(test.result['type']) }}
113 - Packet size: {{ "%s"|format(test.result['packet_size']) }}
114 - Bidirectional : {{ "%s"|format(test.conf['biDirectional']) }}
117 <a name="DetailedTestResults"></a>
118 ### 2.2. Detailed test results
120 A detailed summary of the main results for each test is outlined below.
121 {% for test in tests %}
122 #### Test ID: {{ test.ID }}
124 ##### Results/Metrics Collected
126 The following are the metrics obtained during this test:
129 | ------------------------------ | ------------------------------ |
130 {%- for item, value in test.result.items() %}
131 | {{ "%-30s | %30s |"|format(item,value)}}
134 ##### Statistics collected
136 The following system statistics were collected during testcase execution:
137 {% for process in test.stats %}
138 | --------------------------------------------------------------- |
139 | Process: {{ "%-54s |"|format('_'.join(process.split('_')[:-1])) }}
140 | ------------------------------ | ------------------------------ |
141 | Statistic | Value |
142 | ------------------------------ | ------------------------------ |
143 {%- for item, value in test.stats[process].items() %}
144 | {{ "%-30s | %30s |"|format(item,value)}}
151 No anomalies were detected during the course of this test.
153 ##### Testing Activities/Events
155 There were no significant testing activities for this test.
157 <a name="RationaleForDecisions"></a>
158 ### 2.3. Rationale for decisions
160 The tests conducted do not have pass/fail/conditional-pass criteria. The test
161 is simply conducted and the results are reported.
164 <a name="ConclusionsandRecommendations"></a>
165 ### 2.4. Conclusions and recommendations
167 The test results are stable. The vsperf CI jobs that were used to obtain the
168 results can be found at https://wiki.opnfv.org/wiki/vsperf_results.
172 <a name="General"></a>
175 <a name="Glossary"></a>
178 - NFV - Network Function Virtualization
179 - Mbps - 1,000,000bps
181 <a name="DocChangeProceduresandHistory"></a>
182 ### 3.2. Document change procedures and history
184 | Document ID | Author | Date Modified |
185 | ----------- |------- | ------------- |
186 | `OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT` | Christopher Nolan | 23/01/2015
187 | `OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT` | Christopher Nolan | 28/01/2015