4 This document describes the steps to create a new NSB PROX test based on
5 existing PROX functionalities. NSB PROX provides is a simple approximation
6 of an operation and can be used to develop best practices and TCO models
7 for Telco customers, investigate the impact of new Intel compute,
8 network and storage technologies, characterize performance, and develop
9 optimal system architectures and configurations.
16 In order to integrate PROX tests into NSB, the following prerequisites are required.
18 .. _`dpdk wiki page`: http://dpdk.org/
19 .. _`yardstick wiki page`: https://wiki.opnfv.org/display/yardstick/
20 .. _`Prox documentation`: https://01.org/intel-data-plane-performance-demonstrators/documentation/prox-documentation
21 .. _`openstack wiki page`: https://wiki.openstack.org/wiki/Main_Page
22 .. _`grafana getting started`: http://docs.grafana.org/guides/gettingstarted/
23 .. _`opnfv grafana dashboard`: https://wiki.opnfv.org/display/yardstick/How+to+work+with+grafana+dashboard
24 .. _`Prox command line`: https://01.org/intel-data-plane-performance-demonstrators/documentation/prox-documentation#Command_line_options
25 .. _`grafana deployment`: https://wiki.opnfv.org/display/yardstick/How+to+deploy+InfluxDB+and+Grafana+locally
26 .. _`Prox options`: https://01.org/intel-data-plane-performance-demonstrators/documentation/prox-documentation#.5Beal_options.5D
27 .. _`NSB Installation`: http://artifacts.opnfv.org/yardstick/docs/userguide/index.html#document-09-installation
29 * A working knowledge of Yardstick. See `yardstick wiki page`_.
30 * A working knowledge of PROX. See `Prox documentation`_.
31 * Knowledge of Openstack. See `openstack wiki page`_.
32 * Knowledge of how to use Grafana. See `grafana getting started`_.
33 * How to Deploy InfluxDB & Grafana. See `grafana deployment`_.
34 * How to use Grafana in OPNFV/Yardstick. See `opnfv grafana dashboard`_.
35 * How to install NSB. See `NSB Installation`_
37 Sample Prox Test Hardware Architecture
38 ======================================
40 The following is a diagram of a sample NSB PROX Hardware Architecture
41 for both NSB PROX on Bare metal and on Openstack.
43 In this example when running yardstick on baremetal, yardstick will
44 run on the deployment node, the generator will run on the deployment node
45 and the SUT(SUT) will run on the Controller Node.
48 .. image:: images/PROX_Hardware_Arch.png
50 :alt: Sample NSB PROX Hard Architecture
52 Prox Test Architecture
53 ======================
55 In order to create a new test, one must understand the architecture of
58 A NSB Prox test architecture is composed of:
60 * A traffic generator. This provides blocks of data on 1 or more ports
62 The traffic generator also consumes the result packets from the system
64 * A SUT consumes the packets generated by the packet
65 generator, and applies one or more tasks to the packets and return the
66 modified packets to the traffic generator.
68 This is an example of a sample NSB PROX test architecture.
70 .. image:: images/PROX_Software_Arch.png
72 :alt: NSB PROX test Architecture
74 This diagram is of a sample NSB PROX test application.
78 * Generator Tasks - Composted of 1 or more tasks (It is possible to
79 have multiple tasks sending packets to same port No. See Tasks Ai and Aii
82 * Task Ai - Generates Packets on Port 0 of Traffic Generator
83 and send to Port 0 of SUT Port 0
84 * Task Aii - Generates Packets on Port 0 of Traffic Generator
85 and send to Port 0 of SUT Port 0
86 * Task B - Generates Packets on Port 1 of Traffic Generator
87 and send to Port 1 of SUT Port 1
88 * Task C - Generates Packets on Port 2 of Traffic Generator
89 and send to Port 2 of SUT Port 2
90 * Task Di - Generates Packets on Port 3 of Traffic Generator
91 and send to Port 3 of SUT Port 3
92 * Task Dii - Generates Packets on Port 0 of Traffic Generator
93 and send to Port 0 of SUT Port 0
95 * Verifier Tasks - Composed of 1 or more tasks which receives
98 * Task E - Receives packets on Port 0 of Traffic Generator sent
99 from Port 0 of SUT Port 0
100 * Task F - Receives packets on Port 1 of Traffic Generator sent
101 from Port 1 of SUT Port 1
102 * Task G - Receives packets on Port 2 of Traffic Generator sent
103 from Port 2 of SUT Port 2
104 * Task H - Receives packets on Port 3 of Traffic Generator sent
105 from Port 3 of SUT Port 3
109 * Receiver Tasks - Receives packets from generator - Composed on 1 or
110 more tasks which consume the packs sent from Traffic Generator
112 * Task A - Receives Packets on Port 0 of System-Under-Test from
113 Traffic Generator Port 0, and forwards packets to Task E
114 * Task B - Receives Packets on Port 1 of System-Under-Test from
115 Traffic Generator Port 1, and forwards packets to Task E
116 * Task C - Receives Packets on Port 2 of System-Under-Test from
117 Traffic Generator Port 2, and forwards packets to Task E
118 * Task D - Receives Packets on Port 3 of System-Under-Test from
119 Traffic Generator Port 3, and forwards packets to Task E
121 * Processing Tasks - Composed of multiple tasks in series which carry
122 out some processing on received packets before forwarding to the
125 * Task E - This receives packets from the Receiver Tasks,
126 carries out some operation on the data and forwards to result
127 packets to the next task in the sequence - Task F
128 * Task F - This receives packets from the previous Task - Task
129 E, carries out some operation on the data and forwards to result
130 packets to the next task in the sequence - Task G
131 * Task G - This receives packets from the previous Task - Task F
132 and distributes the result packages to the Transmitter tasks
134 * Transmitter Tasks - Composed on 1 or more tasks which send the
135 processed packets back to the Traffic Generator
137 * Task H - Receives Packets from Task G of System-Under-Test and
138 sends packets to Traffic Generator Port 0
139 * Task I - Receives Packets from Task G of System-Under-Test and
140 sends packets to Traffic Generator Port 1
141 * Task J - Receives Packets from Task G of System-Under-Test and
142 sends packets to Traffic Generator Port 2
143 * Task K - Receives Packets From Task G of System-Under-Test and
144 sends packets to Traffic Generator Port 3
149 A NSB Prox test is composed of the following components :-
151 * Test Description File. Usually called
152 ``tc_prox_<context>_<test>-<ports>.yaml`` where
154 * <context> is either ``baremetal`` or ``heat_context``
155 * <test> is the a one or 2 word description of the test.
156 * <ports> is the number of ports used
158 Example tests ``tc_prox_baremetal_l2fwd-2.yaml`` or
159 ``tc_prox_heat_context_vpe-4.yaml``. This file describes the components
160 of the test, in the case of openstack the network description and
161 server descriptions, in the case of baremetal the hardware
162 description location. It also contains the name of the Traffic Generator, the SUT config file
163 and the traffic profile description, all described below. See nsb-test-description-label_
165 * Traffic Profile file. Example ``prox_binsearch.yaml``. This describes the packet size, tolerated
166 loss, initial line rate to start traffic at, test interval etc See nsb-traffic-profile-label_
168 * Traffic Generator Config file. Usually called ``gen_<test>-<ports>.cfg``.
170 This describes the activity of the traffic generator
172 * What each core of the traffic generator does,
173 * The packet of data sent by a core on a port of the traffic generator
174 to the system under test
175 * What core is used to wait on what port for data from the system
178 Example traffic generator config file ``gen_l2fwd-4.cfg``
179 See nsb-traffic-generator-label_
181 * SUT Config file. Usually called ``handle_<test>-<ports>.cfg``.
183 This describes the activity of the SUTs
185 * What each core of the does,
186 * What cores receives packets from what ports
187 * What cores perform operations on the packets and pass the packets onto
189 * What cores receives packets from what cores and transmit the packets on
190 the ports to the Traffic Verifier tasks of the Traffic Generator.
192 Example traffic generator config file ``handle_l2fwd-4.cfg``
193 See nsb-sut-generator-label_
195 * NSB PROX Baremetal Configuration file. Usually called
196 ``prox-baremetal-<ports>.yaml``
198 * <ports> is the number of ports used
200 This is required for baremetal only. This describes hardware, NICs,
201 IP addresses, Network drivers, usernames and passwords.
202 See baremetal-config-label_
204 * Grafana Dashboard. Usually called
205 ``Prox_<context>_<test>-<port>-<DateAndTime>.json`` where
207 * <context> Is either ``BM`` or ``heat``
208 * <test> Is the a one or 2 word description of the test.
209 * <port> is the number of ports used express as ``2Port`` or ``4Port``
210 * <DateAndTime> is the Date and Time expressed as a string.
212 Example grafana dashboard ``Prox_BM_L2FWD-4Port-1507804504588.json``
214 Other files may be required. These are test specific files and will be
217 .. _nsb-test-description-label:
219 **Test Description File**
221 Here we will discuss the test description for both
222 baremetal and openstack.
224 *Test Description File for Baremetal*
225 -------------------------------------
227 This section will introduce the meaning of the Test case description
228 file. We will use ``tc_prox_baremetal_l2fwd-2.yaml`` as an example to
229 show you how to understand the test description file.
231 .. image:: images/PROX_Test_BM_Script.png
233 :alt: NSB PROX Test Description File
235 Now let's examine the components of the file in detail
237 1. ``traffic_profile`` - This specifies the traffic profile for the
238 test. In this case ``prox_binsearch.yaml`` is used. See nsb-traffic-profile-label_
240 2. ``topology`` - This is either ``prox-tg-topology-1.yaml`` or
241 ``prox-tg-topology-2.yaml`` or ``prox-tg-topology-4.yaml``
242 depending on number of ports required.
244 3. ``nodes`` - This names the Traffic Generator and the System
245 under Test. Does not need to change.
247 4. ``interface_speed_gbps`` - This is an optional parameter. If not present
248 the system defaults to 10Gbps. This defines the speed of the interfaces.
250 5. ``prox_path`` - Location of the Prox executable on the traffic
251 generator (Either baremetal or Openstack Virtual Machine)
253 6. ``prox_config`` - This is the ``SUT Config File``.
254 In this case it is ``handle_l2fwd-2.cfg``
256 A number of additional parameters can be added. This example
260 interface_speed_gbps: 10
263 prox_path: /opt/nsb_bin/prox
264 prox_config: ``configs/handle_vpe-4.cfg``
268 ``configs/vpe_ipv4.lua`` : ````
269 ``configs/vpe_dscp.lua`` : ````
270 ``configs/vpe_cpe_table.lua`` : ````
271 ``configs/vpe_user_table.lua`` : ````
272 ``configs/vpe_rules.lua`` : ````
273 prox_generate_parameter: True
275 ``interface_speed_gbps`` - this specifies the speed of the interface
276 in Gigabits Per Second. This is used to calculate pps(packets per second).
277 If the interfaces are of different speeds, then this specifies the speed
278 of the slowest interface. This parameter is optional. If omitted the
279 interface speed defaults to 10Gbps.
281 ``prox_files`` - this specified that a number of addition files
282 need to be provided for the test to run correctly. This files
283 could provide routing information,hashing information or a
284 hashing algorithm and ip/mac information.
286 ``prox_generate_parameter`` - this specifies that the NSB application
287 is required to provide information to the nsb Prox in the form
288 of a file called ``parameters.lua``, which contains information
289 retrieved from either the hardware or the openstack configuration.
291 7. ``prox_args`` - this specifies the command line arguments to start
292 prox. See `prox command line`_.
294 8. ``prox_config`` - This specifies the Traffic Generator config file.
296 9. ``runner`` - This is set to ``ProxDuration`` - This specifies that the
297 test runs for a set duration. Other runner types are available
298 but it is recommend to use ``ProxDuration``
300 The following parrameters are supported
302 ``interval`` - (optional) - This specifies the sampling interval.
305 ``sampled`` - (optional) - This specifies if sampling information is
306 required. Default ``no``
308 ``duration`` - This is the length of the test in seconds. Default
311 ``confirmation`` - This specifies the number of confirmation retests to
312 be made before deciding to increase or decrease line speed. Default 0.
314 10. ``context`` - This is ``context`` for a 2 port Baremetal configuration.
316 If a 4 port configuration was required then file
317 ``prox-baremetal-4.yaml`` would be used. This is the NSB Prox
318 baremetal configuration file.
320 .. _nsb-traffic-profile-label:
322 *Traffic Profile file*
323 ----------------------
325 This describes the details of the traffic flow. In this case
326 ``prox_binsearch.yaml`` is used.
328 .. image:: images/PROX_Traffic_profile.png
330 :alt: NSB PROX Traffic Profile
333 1. ``name`` - The name of the traffic profile. This name should match the name specified in the
334 ``traffic_profile`` field in the Test Description File.
336 2. ``traffic_type`` - This specifies the type of traffic pattern generated, This name matches
337 class name of the traffic generator See::
339 network_services/traffic_profile/prox_binsearch.py class ProxBinSearchProfile(ProxProfile)
341 In this case it lowers the traffic rate until the number of packets
342 sent is equal to the number of packets received (plus a
343 tolerated loss). Once it achieves this it increases the traffic
344 rate in order to find the highest rate with no traffic loss.
346 Custom traffic types can be created by creating a new traffic profile class.
348 3. ``tolerated_loss`` - This specifies the percentage of packets that
349 can be lost/dropped before
350 we declare success or failure. Success is Transmitted-Packets from
351 Traffic Generator is greater than or equal to
352 packets received by Traffic Generator plus tolerated loss.
354 4. ``test_precision`` - This specifies the precision of the test
355 results. For some tests the success criteria may never be
356 achieved because the test precision may be greater than the
357 successful throughput. For finer results increase the precision
358 by making this value smaller.
360 5. ``packet_sizes`` - This specifies the range of packets size this
363 6. ``duration`` - This specifies the sample duration that the test
364 uses to check for success or failure.
366 7. ``lower_bound`` - This specifies the test initial lower bound sample rate.
367 On success this value is increased.
369 8. ``upper_bound`` - This specifies the test initial upper bound sample rate.
370 On success this value is decreased.
372 Other traffic profiles exist eg prox_ACL.yaml which does not
373 compare what is received with what is transmitted. It just
374 sends packet at max rate.
376 It is possible to create custom traffic profiles with by
377 creating new file in the same folder as prox_binsearch.yaml.
378 See this prox_vpe.yaml as example::
380 schema: ``nsb:traffic_profile:0.1``
383 description: Prox vPE traffic profile
386 traffic_type: ProxBinSearchProfile
387 tolerated_loss: 100.0 #0.001
389 # The minimum size of the Ethernet frame for the vPE test is 68 bytes.
395 *Test Description File for Openstack*
396 -------------------------------------
398 We will use ``tc_prox_heat_context_l2fwd-2.yaml`` as a example to show
399 you how to understand the test description file.
401 .. image:: images/PROX_Test_HEAT_Script1.png
403 :alt: NSB PROX Test Description File - Part 1
405 .. image:: images/PROX_Test_HEAT_Script2.png
407 :alt: NSB PROX Test Description File - Part 2
409 Now lets examine the components of the file in detail
411 Sections 1 to 9 are exactly the same in Baremetal and in Heat. Section
412 ``10`` is replaced with sections A to F. Section 10 was for a baremetal
413 configuration file. This has no place in a heat configuration.
415 A. ``image`` - yardstick-samplevnfs. This is the name of the image
416 created during the installation of NSB. This is fixed.
418 B. ``flavor`` - The flavor is created dynamically. However we could
419 use an already existing flavor if required. In that case the
420 flavor would be named::
422 flavor: yardstick-flavor
424 C. ``extra_specs`` - This allows us to specify the number of
425 cores sockets and hyperthreading assigned to it. In this case
426 we have 1 socket with 10 codes and no hyperthreading enabled.
428 D. ``placement_groups`` - default. Do not change for NSB PROX.
430 E. ``servers`` - ``tg_0`` is the traffic generator and ``vnf_0``
431 is the system under test.
433 F. ``networks`` - is composed of a management network labeled ``mgmt``
434 and one uplink network labeled ``uplink_0`` and one downlink
435 network labeled ``downlink_0`` for 2 ports. If this was a 4 port
436 configuration there would be 2 extra downlink ports. See this
437 example from a 4 port l2fwd test.::
445 port_security_enabled: False
450 port_security_enabled: False
455 port_security_enabled: False
460 port_security_enabled: False
463 .. _nsb-traffic-generator-label:
465 *Traffic Generator Config file*
466 -------------------------------
468 This section will describe the traffic generator config file.
469 This is the same for both baremetal and heat. See this example
470 of ``gen_l2fwd_multiflow-2.cfg`` to explain the options.
472 .. image:: images/PROX_Gen_2port_cfg.png
474 :alt: NSB PROX Gen Config File
476 The configuration file is divided into multiple sections, each
477 of which is used to define some parameters and options.::
493 See `prox options`_ for details
495 Now let's examine the components of the file in detail
497 1. ``[eal options]`` - This specified the EAL (Environmental
498 Abstraction Layer) options. These are default values and
499 are not changed. See `dpdk wiki page`_.
501 2. ``[variables]`` - This section contains variables, as
502 the name suggests. Variables for Core numbers, mac
503 addresses, ip addresses etc. They are assigned as a
504 ``key = value`` where the key is used in place of the value.
507 A special case for valuables with a value beginning with
508 ``@@``. These values are dynamically updated by the NSB
509 application at run time. Values like MAC address,
512 3. ``[port 0]`` - This section describes the DPDK Port. The number
513 following the keyword ``port`` usually refers to the DPDK Port
514 Id. usually starting from ``0``. Because you can have multiple
515 ports this entry usually repeated. Eg. For a 2 port setup
516 ``[port0]`` and ``[port 1]`` and for a 4 port setup ``[port 0]``,
517 ``[port 1]``, ``[port 2]`` and ``[port 3]``::
526 a. In this example ``name = p0`` assigned the name ``p0`` to the
527 port. Any name can be assigned to a port.
528 b. ``mac=hardware`` sets the MAC address assigned by the hardware
529 to data from this port.
530 c. ``rx desc=2048`` sets the number of available descriptors to
531 allocate for receive packets. This can be changed and can
533 d. ``tx desc=2048`` sets the number of available descriptors to
534 allocate for transmit packets. This can be changed and can
536 e. ``promiscuous=yes`` this enables promiscuous mode for this port.
538 4. ``[defaults]`` - Here default operations and settings can be over
539 written. In this example ``mempool size=4K`` the number of mbufs
540 per task is altered. Altering this value could effect
541 performance. See `prox options`_ for details.
543 5. ``[global]`` - Here application wide setting are supported. Things
544 like application name, start time, duration and memory
545 configurations can be set here. In this example.::
551 a. ``start time=5`` Time is seconds after which average
552 stats will be started.
553 b. ``name=Basic Gen`` Name of the configuration.
555 6. ``[core 0]`` - This core is designated the master core. Every
556 Prox application must have a master core. The master mode must
557 be assigned to exactly one task, running alone on one core.::
562 7. ``[core 1]`` - This describes the activity on core 1. Cores can
563 be configured by means of a set of [core #] sections, where
566 a. an absolute core number: e.g. on a 10-core, dual socket
567 system with hyper-threading,
568 cores are numbered from 0 to 39.
570 b. PROX allows a core to be identified by a core number, the
571 letter 's', and a socket number.
573 It is possible to write a baremetal and an openstack test which use
574 the same traffic generator config file and SUT config file.
575 In this case it is advisable not to use physical
578 However it is also possible to write NSB Prox tests that
579 have been optimized for a particular hardware configuration.
580 In this case it is advisable to use the core numbering.
581 It is up to the user to make sure that cores from
582 the right sockets are used (i.e. from the socket on which the NIC
583 is attached to), to ensure good performance (EPA).
585 Each core can be assigned with a set of tasks, each running
586 one of the implemented packet processing modes.::
594 ; Ethernet + IP + UDP
595 pkt inline=${sut_mac0} 70 00 00 00 00 01 08 00 45 00 00 1c 00 01 00 00 40 11 f7 7d 98 10 64 01 98 10 64 02 13 88 13 88 00 08 55 7b
596 ; src_ip: 152.16.100.0/8
599 ; dst_ip: 152.16.100.0/8
602 random=0001001110001XXX0001001110001XXX
605 a. ``name=p0`` - Name assigned to the core.
606 b. ``task=0`` - Each core can run a set of tasks. Starting with ``0``.
607 Task 1 can be defined later in this core or
608 can be defined in another ``[core 1]`` section with ``task=1``
609 later in configuration file. Sometimes running
610 multiple task related to the same packet on the same physical
611 core improves performance, however sometimes it
612 is optimal to move task to a separate core. This is best
613 decided by checking performance.
614 c. ``mode=gen`` - Specifies the action carried out by this task on
615 this core. Supported modes are: classify, drop, gen, lat, genl4, nop, l2fwd, gredecap,
616 greencap, lbpos, lbnetwork, lbqinq, lb5tuple, ipv6_decap, ipv6_encap,
617 qinqdecapv4, qinqencapv4, qos, routing, impair,
618 mirror, unmpls, tagmpls, nat, decapnsh, encapnsh, police, acl
623 * Basic Forwarding (no touch)
624 * L2 Forwarding (change MAC)
626 * Load balance based on packet fields
627 * Symmetric load balancing
628 * QinQ encap/decap IPv4/IPv6
637 In the traffic generator we expect a core to generate packets (``gen``)
638 and to receive packets & calculate latency (``lat``)
639 This core does ``gen`` . ie it is a traffic generator.
641 To understand what each of the modes support please see
642 `prox documentation`_.
644 d. ``tx port=p0`` - This specifies that the packets generated are
645 transmitted to port ``p0``
646 e. ``bps=1250000000`` - This indicates Bytes Per Second to
648 f. ``; Ethernet + IP + UDP`` - This is a comment. Items starting with
650 g. ``pkt inline=${sut_mac0} 70 00 00 00 ...`` - Defines the packet
651 format as a sequence of bytes (each
652 expressed in hexadecimal notation). This defines the packet
653 that is generated. This packets begins
654 with the hexadecimal sequence assigned to ``sut_mac`` and the
655 remainder of the bytes in the string.
656 This packet could now be sent or modified by ``random=..``
657 described below before being sent to target.
658 h. ``; src_ip: 152.16.100.0/8`` - Comment
659 i. ``random=0000XXX1`` - This describes a field of the packet
660 containing random data. This string can be
661 8,16,24 or 32 character long and represents 1,2,3 or 4
662 bytes of data. In this case it describes a byte of
663 data. Each character in string can be 0,1 or ``X``. 0 or 1
664 are fixed bit values in the data packet and ``X`` is a
665 random bit. So random=0000XXX1 generates 00000001(1),
666 00000011(3), 00000101(5), 00000111(7),
667 00001001(9), 00001011(11), 00001101(13) and 00001111(15)
669 j. ``rand_offset=29`` - Defines where to place the previously
670 defined random field.
671 k. ``; dst_ip: 152.16.100.0/8`` - Comment
672 l. ``random=0000XXX0`` - This is another random field which
673 generates a byte of 00000000(0), 00000010(2),
674 00000100(4), 00000110(6), 00001000(8), 00001010(10),
675 00001100(12) and 00001110(14) combinations.
676 m. ``rand_offset=33`` - Defines where to place the previously
677 defined random field.
678 n. ``random=0001001110001XXX0001001110001XXX`` - This is
679 another random field which generates 4 bytes.
680 o. ``rand_offset=34`` - Defines where to place the previously
681 defined 4 byte random field.
683 Core 2 executes same scenario as Core 1. The only difference
684 in this case is that the packets are generated
687 8. ``[core 3]`` - This defines the activities on core 3. The purpose
688 of ``core 3`` and ``core 4`` is to receive packets
698 a. ``name=rec 0`` - Name assigned to the core.
699 b. ``task=0`` - Each core can run a set of tasks. Starting with
700 ``0``. Task 1 can be defined later in this core or
701 can be defined in another ``[core 1]`` section with
702 ``task=1`` later in configuration file. Sometimes running
703 multiple task related to the same packet on the same
704 physical core improves performance, however sometimes it
705 is optimal to move task to a separate core. This is
706 best decided by checking performance.
707 c. ``mode=lat`` - Specifies the action carried out by this task on this core. Supported modes are: acl,
708 classify, drop, gredecap, greencap, ipv6_decap, ipv6_encap, l2fwd, lbnetwork, lbpos, lbqinq, nop,
709 police, qinqdecapv4, qinqencapv4, qos, routing, impair, lb5tuple, mirror, unmpls, tagmpls,
710 nat, decapnsh, encapnsh, gen, genl4 and lat. This task(0) per core(3) receives packets on port.
711 d. ``rx port=p0`` - The port to receive packets on ``Port 0``. Core 4 will receive packets on ``Port 1``.
712 e. ``lat pos=42`` - Describes where to put a 4-byte timestamp in the packet. Note that the packet length should
713 be longer than ``lat pos`` + 4 bytes to avoid truncation of the timestamp. It defines where the timestamp is
714 to be read from. Note that the SUT workload might cause the position of the timestamp to change
715 (i.e. due to encapsulation).
717 .. _nsb-sut-generator-label:
720 -------------------------------
722 This section will describes the SUT(VNF) config file. This is the same for both
723 baremetal and heat. See this example of ``handle_l2fwd_multiflow-2.cfg`` to explain the options.
725 .. image:: images/PROX_Handle_2port_cfg.png
727 :alt: NSB PROX Handle Config File
729 See `prox options`_ for details
731 Now let's examine the components of the file in detail
733 1. ``[eal options]`` - same as the Generator config file. This specified the EAL (Environmental Abstraction Layer)
734 options. These are default values and are not changed.
735 See `dpdk wiki page`_.
737 2. ``[port 0]`` - This section describes the DPDK Port. The number following the keyword ``port`` usually refers to the DPDK Port Id. usually starting from ``0``.
738 Because you can have multiple ports this entry usually repeated. Eg. For a 2 port setup ``[port0]`` and ``[port 1]`` and for a 4 port setup ``[port 0]``, ``[port 1]``,
739 ``[port 2]`` and ``[port 3]``::
748 a. In this example ``name =if0`` assigned the name ``if0`` to the port. Any name can be assigned to a port.
749 b. ``mac=hardware`` sets the MAC address assigned by the hardware to data from this port.
750 c. ``rx desc=2048`` sets the number of available descriptors to allocate for receive packets. This can be changed and can effect performance.
751 d. ``tx desc=2048`` sets the number of available descriptors to allocate for transmit packets. This can be changed and can effect performance.
752 e. ``promiscuous=yes`` this enables promiscuous mode for this port.
754 3. ``[defaults]`` - Here default operations and settings can be over written.::
760 a. In this example ``mempool size=8K`` the number of mbufs per task is altered. Altering this value could effect performance. See `prox options`_ for details.
761 b. ``memcache size=512`` - number of mbufs cached per core, default is 256 this is the cache_size. Altering this value could effect performance.
763 4. ``[global]`` - Here application wide setting are supported. Things like application name, start time, duration and memory configurations can be set here.
770 a. ``start time=5`` Time is seconds after which average stats will be started.
771 b. ``name=Handle L2FWD Multiflow (2x)`` Name of the configuration.
773 5. ``[core 0]`` - This core is designated the master core. Every Prox application must have a master core. The master mode must be assigned to
774 exactly one task, running alone on one core.::
779 6. ``[core 1]`` - This describes the activity on core 1. Cores can be configured by means of a set of [core #] sections, where # represents either:
781 a. an absolute core number: e.g. on a 10-core, dual socket system with hyper-threading,
782 cores are numbered from 0 to 39.
784 b. PROX allows a core to be identified by a core number, the letter 's', and a socket number.
785 However NSB PROX is hardware agnostic (physical and virtual configurations are the same) it
786 is advisable no to use physical core numbering.
788 Each core can be assigned with a set of tasks, each running one of the implemented packet processing modes.::
794 dst mac=@@tester_mac1
798 a. ``name=none`` - No name assigned to the core.
799 b. ``task=0`` - Each core can run a set of tasks. Starting with ``0``. Task 1 can be defined later in this core or
800 can be defined in another ``[core 1]`` section with ``task=1`` later in configuration file. Sometimes running
801 multiple task related to the same packet on the same physical core improves performance, however sometimes it
802 is optimal to move task to a separate core. This is best decided by checking performance.
803 c. ``mode=l2fwd`` - Specifies the action carried out by this task on this core. Supported modes are: acl,
804 classify, drop, gredecap, greencap, ipv6_decap, ipv6_encap, l2fwd, lbnetwork, lbpos, lbqinq, nop,
805 police, qinqdecapv4, qinqencapv4, qos, routing, impair, lb5tuple, mirror, unmpls, tagmpls,
806 nat, decapnsh, encapnsh, gen, genl4 and lat. This code does ``l2fwd`` .. ie it does the L2FWD.
808 d. ``dst mac=@@tester_mac1`` - The destination mac address of the packet will be set to the MAC address of ``Port 1`` of destination device. (The Traffic Generator/Verifier)
809 e. ``rx port=if0`` - This specifies that the packets are received from ``Port 0`` called if0
810 f. ``tx port=if1`` - This specifies that the packets are transmitted to ``Port 1`` called if1
812 If this example we receive a packet on core on a port, carry out operation on the packet on the core and transmit it on on another port still using the same task on the same core.
814 On some implementation you may wish to use multiple tasks, like this.::
832 In this example you can see Core 1/Task 0 called ``rx_task`` receives the packet from if0 and perform the l2fwd. However instead of sending the packet to a
833 port it sends it to a core see ``tx cores=1t1``. In this case it sends it to Core 1/Task 1.
835 Core 1/Task 1 called ``l2fwd_if0``, receives the packet, not from a port but from the ring. See ``rx ring=yes``. It does not perform any operation on the packet See ``mode=none``
836 and sends the packets to ``if0`` see ``tx port=if0``.
838 It is also possible to implement more complex operations be chaining multiple operations in sequence and using rings to pass packets from one core to another.
840 In thus example we show a Broadband Network Gateway (BNG) with Quality of Service (QoS). Communication from task to task is via rings.
842 .. image:: images/PROX_BNG_QOS.png
844 :alt: NSB PROX Config File for BNG_QOS
846 *Baremetal Configuration file*
847 ------------------------------
849 .. _baremetal-config-label:
851 This is required for baremetal testing. It describes the IP address of the various ports, the Network devices drivers and MAC addresses and the network
854 In this example we will describe a 2 port configuration. This file is the same for all 2 port NSB Prox tests on the same platforms/configuration.
856 .. image:: images/PROX_Baremetal_config.png
858 :alt: NSB PROX Yardstick Config
860 Now lets describe the sections of the file.
862 1. ``TrafficGen`` - This section describes the Traffic Generator node of the test configuration. The name of the node ``trafficgen_1`` must match the node name
863 in the ``Test Description File for Baremetal`` mentioned earlier. The password attribute of the test needs to be configured. All other parameters
864 can remain as default settings.
865 2. ``interfaces`` - This defines the DPDK interfaces on the Traffic Generator.
866 3. ``xe0`` is DPDK Port 0. ``lspci`` and `` ./dpdk-devbind.py -s`` can be used to provide the interface information. ``netmask`` and ``local_ip`` should not be changed
867 4. ``xe1`` is DPDK Port 1. If more than 2 ports are required then ``xe1`` section needs to be repeated and modified accordingly.
868 5. ``vnf`` - This section describes the SUT of the test configuration. The name of the node ``vnf`` must match the node name in the
869 ``Test Description File for Baremetal`` mentioned earlier. The password attribute of the test needs to be configured. All other parameters
870 can remain as default settings
871 6. ``interfaces`` - This defines the DPDK interfaces on the SUT
872 7. ``xe0`` - Same as 3 but for the ``SUT``.
873 8. ``xe1`` - Same as 4 but for the ``SUT`` also.
874 9. ``routing_table`` - All parameters should remain unchanged.
875 10. ``nd_route_tbl`` - All parameters should remain unchanged.
880 The grafana dashboard visually displays the results of the tests. The steps required to produce a grafana dashboard are described here.
882 .. _yardstick-config-label:
884 a. Configure ``yardstick`` to use influxDB to store test results. See file ``/etc/yardstick/yardstick.conf``.
886 .. image:: images/PROX_Yardstick_config.png
888 :alt: NSB PROX Yardstick Config
890 1. Specify the dispatcher to use influxDB to store results.
891 2. "target = .. " - Specify location of influxDB to store results.
892 "db_name = yardstick" - name of database. Do not change
893 "username = root" - username to use to store result. (Many tests are run as root)
894 "password = ... " - Please set to root user password
896 b. Deploy InfludDB & Grafana. See how to Deploy InfluxDB & Grafana. See `grafana deployment`_.
897 c. Generate the test data. Run the tests as follows .::
899 yardstick --debug task start tc_prox_<context>_<test>-ports.yaml
903 yardstick --debug task start tc_prox_heat_context_l2fwd-4.yaml
905 d. Now build the dashboard for the test you just ran. The easiest way to do this is to copy an existing dashboard and rename the
906 test and the field names. The procedure to do so is described here. See `opnfv grafana dashboard`_.
908 How to run NSB Prox Test on an baremetal environment
909 ====================================================
911 In order to run the NSB PROX test.
913 1. Install NSB on Traffic Generator node and Prox in SUT. See `NSB Installation`_
915 2. To enter container::
917 docker exec -it yardstick /bin/bash
919 3. Install baremetal configuration file (POD files)
921 a. Go to location of PROX tests in container ::
923 cd /home/opnfv/repos/yardstick/samples/vnf_samples/nsut/prox
925 b. Install prox-baremetal-2.yam and prox-baremetal-4.yaml for that topology
926 into this directory as per baremetal-config-label_
928 c. Install and configure ``yardstick.conf`` ::
932 Modify /etc/yardstick/yardstick.conf as per yardstick-config-label_
934 4. Execute the test. Eg.::
936 yardstick --debug task start ./tc_prox_baremetal_l2fwd-4.yaml
938 How to run NSB Prox Test on an Openstack environment
939 ====================================================
941 In order to run the NSB PROX test.
943 1. Install NSB on Openstack deployment node. See `NSB Installation`_
945 2. To enter container::
947 docker exec -it yardstick /bin/bash
949 3. Install configuration file
951 a. Goto location of PROX tests in container ::
953 cd /home/opnfv/repos/yardstick/samples/vnf_samples/nsut/prox
955 b. Install and configure ``yardstick.conf`` ::
959 Modify /etc/yardstick/yardstick.conf as per yardstick-config-label_
962 4. Execute the test. Eg.::
964 yardstick --debug task start ./tc_prox_heat_context_l2fwd-4.yaml
966 Frequently Asked Questions
967 ==========================
969 Here is a list of frequently asked questions.
971 *NSB Prox does not work on Baremetal, How do I resolve this?*
972 -------------------------------------------------------------
974 If PROX NSB does not work on baremetal, problem is either in network configuration or test file.
978 1. Verify network configuration. Execute existing baremetal test.::
980 yardstick --debug task start ./tc_prox_baremetal_l2fwd-4.yaml
982 If test does not work then error in network configuration.
984 a. Check DPDK on Traffic Generator and SUT via:- ::
986 /root/dpdk-17./usertools/dpdk-devbind.py
988 b. Verify MAC addresses match ``prox-baremetal-<ports>.yaml`` via ``ifconfig`` and ``dpdk-devbind``
990 c. Check your eth port is what you expect. You would not be the first person to think that
991 the port your cable is plugged into is ethX when in fact it is ethY. Use
992 ethtool to visually confirm that the eth is where you expect.::
996 A led should start blinking on port. (On both System-Under-Test and Traffic Generator)
1000 Install Linux kernel network driver and ensure your ports are
1001 ``bound`` to the driver via ``dpdk-devbind``. Bring up port on both
1002 SUT and Traffic Generator and check connection.
1004 i) On SUT and on Traffic Generator::
1006 ifconfig ethX/enoX up
1012 See ``Link detected`` if ``yes`` .... Cable is good. If ``no`` you have an issue with your cable/port.
1014 2. If existing baremetal works then issue is with your test. Check the traffic generator gen_<test>-<ports>.cfg to ensure
1015 it is producing a valid packet.
1017 *How do I debug NSB Prox on Baremetal?*
1018 ---------------------------------------
1022 1. Execute the test as follows::
1024 yardstick --debug task start ./tc_prox_baremetal_l2fwd-4.yaml
1026 2. Login to Traffic Generator as ``root``.::
1029 /opt/nsb_bin/prox -f /tmp/gen_<test>-<ports>.cfg
1031 3. Login to SUT as ``root``.::
1034 /opt/nsb_bin/prox -f /tmp/handle_<test>-<ports>.cfg
1036 4. Now let's examine the Generator Output. In this case the output of gen_l2fwd-4.cfg.
1038 .. image:: images/PROX_Gen_GUI.png
1040 :alt: NSB PROX Traffic Generator GUI
1042 Now let's examine the output
1044 1. Indicates the amount of data successfully transmitted on Port 0
1045 2. Indicates the amount of data successfully received on port 1
1046 3. Indicates the amount of data successfully handled for port 1
1048 It appears what is transmitted is received.
1051 The number of packets MAY not exactly match because the ports are read in sequence.
1054 What is transmitted on PORT X may not always be received on same port. Please check the Test scenario.
1056 5. Now lets examine the SUT Output
1058 .. image:: images/PROX_SUT_GUI.png
1060 :alt: NSB PROX SUT GUI
1062 Now lets examine the output
1064 1. What is received on 0 is transmitted on 1, received on 1 transmitted on 0,
1065 received on 2 transmitted on 3 and received on 3 transmitted on 2.
1066 2. No packets are Failed.
1067 3. No packets are discarded.
1069 We can also dump the packets being received or transmitted via the following commands. ::
1071 dump Arguments: <core id> <task id> <nb packets>
1072 Create a hex dump of <nb_packets> from <task_id> on <core_id> showing how
1073 packets have changed between RX and TX.
1074 dump_rx Arguments: <core id> <task id> <nb packets>
1075 Create a hex dump of <nb_packets> from <task_id> on <core_id> at RX
1076 dump_tx Arguments: <core id> <task id> <nb packets>
1077 Create a hex dump of <nb_packets> from <task_id> on <core_id> at TX
1083 *NSB Prox works on Baremetal but not in Openstack. How do I resolve this?*
1084 --------------------------------------------------------------------------
1086 NSB Prox on Baremetal is a lot more forgiving than NSB Prox on Openstack. A badly
1087 formed packed may still work with PROX on Baremetal. However on
1088 Openstack the packet must be correct and all fields of the header correct.
1089 Eg A packet with an invalid Protocol ID would still work in Baremetal
1090 but this packet would be rejected by openstack.
1094 1. Check the validity of the packet.
1095 2. Use a known good packet in your test
1096 3. If using ``Random`` fields in the traffic generator, disable them and retry.
1099 *How do I debug NSB Prox on Openstack?*
1100 ---------------------------------------
1104 1. Execute the test as follows::
1106 yardstick --debug task start --keep-deploy ./tc_prox_heat_context_l2fwd-4.yaml
1108 2. Access docker image if required via::
1110 docker exec -it yardstick /bin/bash
1112 3. Install openstack credentials.
1114 Depending on your openstack deployment, the location of these credentials may vary.
1115 On this platform I do this via::
1117 scp root@10.237.222.55:/etc/kolla/admin-openrc.sh .
1118 source ./admin-openrc.sh
1120 4. List Stack details
1122 a. Get the name of the Stack.
1124 .. image:: images/PROX_Openstack_stack_list.png
1126 :alt: NSB PROX openstack stack list
1128 b. Get the Floating IP of the Traffic Generator & SUT
1130 This generates a lot of information. Please not the floating IP of the VNF and
1131 the Traffic Generator.
1133 .. image:: images/PROX_Openstack_stack_show_a.png
1135 :alt: NSB PROX openstack stack show (Top)
1137 From here you can see the floating IP Address of the SUT / VNF
1139 .. image:: images/PROX_Openstack_stack_show_b.png
1141 :alt: NSB PROX openstack stack show (Top)
1143 From here you can see the floating IP Address of the Traffic Generator
1145 c. Get ssh identity file
1147 In the docker container locate the identity file.::
1149 cd /home/opnfv/repos/yardstick/yardstick/resources/files
1152 5. Login to SUT as ``Ubuntu``.::
1154 ssh -i ./yardstick_key-01029d1d ubuntu@172.16.2.158
1160 Now continue as baremetal.
1162 6. Login to SUT as ``Ubuntu``.::
1164 ssh -i ./yardstick_key-01029d1d ubuntu@172.16.2.156
1170 Now continue as baremetal.
1172 *How do I resolve "Quota exceeded for resources"*
1173 -------------------------------------------------
1177 This usually occurs due to 2 reasons when executing an openstack test.
1179 1. One or more stacks already exists and are consuming all resources. To resolve ::
1181 openstack stack list
1185 +--------------------------------------+--------------------+-----------------+----------------------+--------------+
1186 | ID | Stack Name | Stack Status | Creation Time | Updated Time |
1187 +--------------------------------------+--------------------+-----------------+----------------------+--------------+
1188 | acb559d7-f575-4266-a2d4-67290b556f15 | yardstick-e05ba5a4 | CREATE_COMPLETE | 2017-12-06T15:00:05Z | None |
1189 | 7edf21ce-8824-4c86-8edb-f7e23801a01b | yardstick-08bda9e3 | CREATE_COMPLETE | 2017-12-06T14:56:43Z | None |
1190 +--------------------------------------+--------------------+-----------------+----------------------+--------------+
1192 In this case 2 stacks already exist.
1196 openstack stack delete yardstick-08bda9e3
1197 Are you sure you want to delete this stack(s) [y/N]? y
1199 2. The openstack configuration quotas are too small.
1201 The solution is to increase the quota. Use below to query existing quotas::
1203 openstack quota show
1207 openstack quota set <resource>
1209 *Openstack Cli fails or hangs. How do I resolve this?*
1210 ------------------------------------------------------
1214 If it fails due to ::
1216 Missing value auth-url required for auth plugin password
1218 Check your shell environment for Openstack variables. One of them should contain the authentication URL ::
1221 OS_AUTH_URL=``https://192.168.72.41:5000/v3``
1223 Or similar. Ensure that openstack configurations are exported. ::
1225 cat /etc/kolla/admin-openrc.sh
1229 export OS_PROJECT_DOMAIN_NAME=default
1230 export OS_USER_DOMAIN_NAME=default
1231 export OS_PROJECT_NAME=admin
1232 export OS_TENANT_NAME=admin
1233 export OS_USERNAME=admin
1234 export OS_PASSWORD=BwwSEZqmUJA676klr9wa052PFjNkz99tOccS9sTc
1235 export OS_AUTH_URL=http://193.168.72.41:35357/v3
1236 export OS_INTERFACE=internal
1237 export OS_IDENTITY_API_VERSION=3
1238 export EXTERNAL_NETWORK=yardstick-public
1242 If the Openstack Cli appears to hang, then verify the proxys and no_proxy are set correctly.
1243 They should be similar to ::
1245 FTP_PROXY="http://proxy.ir.intel.com:911/"
1246 HTTPS_PROXY="http://proxy.ir.intel.com:911/"
1247 HTTP_PROXY="http://proxy.ir.intel.com:911/"
1248 NO_PROXY="localhost,127.0.0.1,10.237.222.55,10.237.223.80,10.237.222.134,.ir.intel.com"
1249 ftp_proxy="http://proxy.ir.intel.com:911/"
1250 http_proxy="http://proxy.ir.intel.com:911/"
1251 https_proxy="http://proxy.ir.intel.com:911/"
1252 no_proxy="localhost,127.0.0.1,10.237.222.55,10.237.223.80,10.237.222.134,.ir.intel.com"
1256 1) 10.237.222.55 = IP Address of deployment node
1257 2) 10.237.223.80 = IP Address of Controller node
1258 3) 10.237.222.134 = IP Address of Compute Node
1259 4) ir.intel.com = local no proxy
1262 *How to Understand the Grafana output?*
1263 ---------------------------------------
1265 .. image:: images/PROX_Grafana_1.png
1267 :alt: NSB PROX Grafana_1
1269 .. image:: images/PROX_Grafana_2.png
1271 :alt: NSB PROX Grafana_2
1273 .. image:: images/PROX_Grafana_3.png
1275 :alt: NSB PROX Grafana_3
1277 .. image:: images/PROX_Grafana_4.png
1279 :alt: NSB PROX Grafana_4
1281 A. Test Parameters - Test interval, Duartion, Tolerated Loss and Test Precision
1283 B. Overall No of packets send and received during test
1285 C. Generator Stats - packets sent, received and attempted by Generator
1289 E. No of packets received by SUT
1291 F. No of packets forwarded by SUT
1293 G. This is the number of packets sent by the generator per port, for each interval.
1295 H. This is the number of packets received by the generator per port, for each interval.
1297 I. This is the number of packets send and received by the generator and lost by the SUT
1298 that meet the success criteria
1300 J. This is the changes the Percentage of Line Rate used over a test, The MAX and the
1301 MIN should converge to within the interval specified as the ``test-precision``.
1303 K. This is the packets Size supported during test. If "N/A" appears in any field the result has not been decided.
1305 L. This is the calculated throughput in MPPS(Million Packets Per second) for this line rate.
1307 M. This is the actual No, of packets sent by the generator in MPPS
1309 N. This is the actual No. of packets received by the generator in MPPS
1311 O. This is the total No. of packets sent by SUT.
1313 P. This is the total No. of packets received by the SUT
1315 Q. This is the total No. of packets dropped. (These packets were sent by the generator but not
1316 received back by the generator, these may be dropped by the SUT or the Generator)
1318 R. This is the tolerated no of packets that can be dropped.
1320 S. This is the test Throughput in Gbps
1322 T. This is the Latencey per Port
1324 U. This is the CPU Utilization