1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, 2016-2017 Intel Corporation.
6 Network Services Benchmarking (NSB)
7 ===================================
12 .. _Yardstick: https://wiki.opnfv.org/yardstick
14 This chapter provides an overview of the NSB, a contribution to OPNFV
15 Yardstick_ from Intel.
20 The goal of NSB is to Extend Yardstick to perform real world VNFs and NFVi Characterization and
21 benchmarking with repeatable and deterministic methods.
23 The Network Service Benchmarking (NSB) extends the yardstick framework to do
24 VNF characterization and benchmarking in three different execution
25 environments - bare metal i.e. native Linux environment, standalone virtual
26 environment and managed virtualized environment (e.g. Open stack etc.).
27 It also brings in the capability to interact with external traffic generators
28 both hardware & software based for triggering and validating the traffic
29 according to user defined profiles.
31 NSB extension includes:
33 - Generic data models of Network Services, based on ETSI spec `ETSI GS NFV-TST 001 <http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_nfv-tst001v010101p.pdf>`_
35 - New Standalone context for VNF testing like SRIOV, OVS, OVS-DPDK etc
37 - Generic VNF configuration models and metrics implemented with Python
40 - Traffic generator features and traffic profiles
42 - L1-L3 state-less traffic profiles
44 - L4-L7 state-full traffic profiles
46 - Tunneling protocol / network overlay support
54 - vPE,vCGNAT, vFirewall etc - ipv4 throughput, latency etc
56 - Traffic generators like Trex, ab/nginx, ixia, iperf etc
58 - KPIs for a given use case:
60 - System agent support for collecting NFVi KPI. This includes:
68 - Network KPIs, e.g., inpackets, outpackets, thoughput, latency etc
70 - VNF KPIs, e.g., packet_in, packet_drop, packet_fwd etc
75 The Network Service (NS) defines a set of Virtual Network Functions (VNF)
76 connected together using NFV infrastructure.
78 The Yardstick NSB extension can support multiple VNFs created by different
79 vendors including traffic generators. Every VNF being tested has its
80 own data model. The Network service defines a VNF modelling on base of performed
81 network functionality. The part of the data model is a set of the configuration
82 parameters, number of connection points used and flavor including core and
85 The ETSI defines a Network Service as a set of configurable VNFs working in
86 some NFV Infrastructure connecting each other using Virtual Links available
87 through Connection Points. The ETSI MANO specification defines a set of
88 management entities called Network Service Descriptors (NSD) and
89 VNF Descriptors (VNFD) that define real Network Service. The picture below
90 makes an example how the real Network Operator use-case can map into ETSI
91 Network service definition
93 Network Service framework performs the necessary test steps. It may involve
95 - Interacting with traffic generator and providing the inputs on traffic
96 type / packet structure to generate the required traffic as per the
97 test case. Traffic profiles will be used for this.
99 - Executing the commands required for the test procedure and analyses the
100 command output for confirming whether the command got executed correctly
101 or not. E.g. As per the test case, run the traffic for the given
102 time period / wait for the necessary time delay
104 - Verify the test result.
106 - Validate the traffic flow from SUT
108 - Fetch the table / data from SUT and verify the value as per the test case
110 - Upload the logs from SUT onto the Test Harness server
112 - Read the KPI's provided by particular VNF
114 Components of Network Service
115 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
117 * *Models for Network Service benchmarking*: The Network Service benchmarking
118 requires the proper modelling approach. The NSB provides models using Python
119 files and defining of NSDs and VNFDs.
121 The benchmark control application being a part of OPNFV yardstick can call
122 that python models to instantiate and configure the VNFs. Depending on
123 infrastructure type (bare-metal or fully virtualized) that calls could be
124 made directly or using MANO system.
126 * *Traffic generators in NSB*: Any benchmark application requires a set of
127 traffic generator and traffic profiles defining the method in which traffic
130 The Network Service benchmarking model extends the Network Service
131 definition with a set of Traffic Generators (TG) that are treated
132 same way as other VNFs being a part of benchmarked network service.
133 Same as other VNFs the traffic generator are instantiated and terminated.
135 Every traffic generator has own configuration defined as a traffic profile and
136 a set of KPIs supported. The python models for TG is extended by specific calls
137 to listen and generate traffic.
139 * *The stateless TREX traffic generator*: The main traffic generator used as
140 Network Service stimulus is open source TREX tool.
142 The TREX tool can generate any kind of stateless traffic.
144 .. code-block:: console
146 +--------+ +-------+ +--------+
148 | Trex | ---> | VNF | ---> | Trex |
150 +--------+ +-------+ +--------+
152 Supported testcases scenarios:
154 - Correlated UDP traffic using TREX traffic generator and replay VNF.
156 - using different IMIX configuration like pure voice, pure video traffic etc
158 - using different number IP flows like 1 flow, 1K, 16K, 64K, 256K, 1M flows
160 - Using different number of rules configured like 1 rule, 1K, 10K rules
162 For UDP correlated traffic following Key Performance Indicators are collected
163 for every combination of test case parameters:
165 - RFC2544 throughput for various loss rate defined (1% is a default)
170 NSB Testing with yardstick framework facilitate performance testing of various
173 .. code-block:: console
177 | vPE | ->|TGen Port 0|
178 | TestCase | | +-----------+
180 +-----------+ +------------------+ +-------+ |
181 | | -- API --> | VNF | <--->
182 +-----------+ | Yardstick | +-------+ |
183 | Test Case | --> | NSB Testing | |
186 | +------------------+ |
187 +-----------+ | +-----------+
188 | Traffic | ->|TGen Port 1|
189 | patterns | +-----------+
192 Figure 1: Network Service - 2 server configuration
194 VNFs supported for chracterization:
195 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
197 1. CGNAPT - Carrier Grade Network Address and port Translation
198 2. vFW - Virtual Firewall
199 3. vACL - Access Control List
200 4. Prox - Packet pROcessing eXecution engine:
201 - VNF can act as Drop, Basic Forwarding (no touch),
202 L2 Forwarding (change MAC), GRE encap/decap, Load balance based on
203 packet fields, Symmetric load balancing
204 - QinQ encap/decap IPv4/IPv6, ARP, QoS, Routing, Unmpls, Policing, ACL