1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, Intel Corporation and others.
14 The device under test (DUT) consists of a system following;
15 * A single or dual processor and PCH chip, except for System on Chip (SoC) cases
16 * DRAM memory size and frequency (normally single DIMM per channel)
17 * Specific Intel Network Interface Cards (NICs)
18 * BIOS settings noting those that updated from the basic settings
19 * DPDK build configuration settings, and commands used for tests
20 Connected to the DUT is an IXIA* or Software Traffic generator like pktgen or TRex,
21 simulation platform to generate packet traffic to the DUT ports and
22 determine the throughput/latency at the tester side.
24 Below are the supported/tested (:term:`VNF`) deployment type.
26 .. image:: images/deploy_type.png
28 :alt: SampleVNF supported topology
30 Hardware & Software Ingredients
31 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
35 +-----------+------------------+
36 | Item | Description |
37 +-----------+------------------+
39 +-----------+------------------+
41 +-----------+------------------+
42 | OS | Ubuntu 16.04 LTS |
43 +-----------+------------------+
44 | kernel | 4.4.0-34-generic |
45 +-----------+------------------+
47 +-----------+------------------+
49 Boot and BIOS settings:
51 +------------------+---------------------------------------------------+
52 | Boot settings | default_hugepagesz=1G hugepagesz=1G hugepages=16 |
53 | | hugepagesz=2M hugepages=2048 isolcpus=1-11,22-33 |
54 | | nohz_full=1-11,22-33 rcu_nocbs=1-11,22-33 |
55 | | Note: nohz_full and rcu_nocbs is to disable Linux*|
56 | | kernel interrupts, and it’s import |
57 +------------------+---------------------------------------------------+
58 | BIOS | CPU Power and Performance Policy <Performance> |
59 | | CPU C-state Disabled |
60 | | CPU P-state Disabled |
61 | | Enhanced Intel® Speedstep® Tech Disabled |
62 | | Hyper-Threading Technology (If supported) Enable |
63 | | Virtualization Techology Enable |
64 | | Coherency Enable |
65 | | Turbo Boost Disabled |
66 +------------------+---------------------------------------------------+
68 Network Topology for testing VNFs
69 ---------------------------------
70 The ethernet cables should be connected between traffic generator and the VNF server (BM,
71 SRIOV or OVS) setup based on the test profile.
73 The connectivity could be
75 1) Single port pair : One pair ports used for traffic
79 e.g. Single port pair link0 and link1 of VNF are used
80 TG:port 0 <------> VNF:Port 0
81 TG:port 1 <------> VNF:Port 1
83 For correalted traffic, use below configuration
84 TG_1:port 0 <------> VNF:Port 0
85 VNF:Port 1 <------> TG_2:port 0 (UDP Replay)
86 (TG_2(UDP_Replay) reflects all the traffic on the given port)
88 2) Multi port pair : More than one pair of traffic
92 e.g. Two port pair link 0, link1, link2 and link3 of VNF are used
93 TG:port 0 <------> VNF:Port 0
94 TG:port 1 <------> VNF:Port 1
95 TG:port 2 <------> VNF:Port 2
96 TG:port 3 <------> VNF:Port 3
98 For correalted traffic, use below configuration
99 TG_1:port 0 <------> VNF:Port 0
100 VNF:Port 1 <------> TG_2:port 0 (UDP Replay)
101 TG_1:port 1 <------> VNF:Port 2
102 VNF:Port 3 <------> TG_2:port 1 (UDP Replay)
103 (TG_2(UDP_Replay) reflects all the traffic on the given port)
106 Refer: http://fast.dpdk.org/doc/pdf-guides/ to setup the DUT for VNF to run
108 * Standalone Virtualization - PHY-VM-PHY
110 Refer below link to setup sriov
111 https://software.intel.com/en-us/articles/using-sr-iov-to-share-an-ethernet-port-among-multiple-vms
114 Refer below link to setup ovs-dpdk
115 http://docs.openvswitch.org/en/latest/intro/install/general/
116 http://docs.openvswitch.org/en/latest/intro/install/dpdk/
119 Use any OPNFV installer to deploy the openstack.
121 Setup Traffic generator
122 -----------------------
124 Step 0: Preparing hardware connection
126 Connect Traffic generator and VNF system back to back as shown in previous section
130 TRex port 0 ↔ (VNF Port 0) ↔ (VNF Port 1) ↔ TRex port 1
132 Step 1: Setting up Traffic generator (TRex)
134 TRex Software preparations
135 **************************
136 * Install the OS (Bare metal Linux, not VM!)
137 * Obtain the latest TRex package: wget https://trex-tgn.cisco.com/trex/release/latest
138 * Untar the package: tar -xzf latest
139 * Change dir to unzipped TRex
140 * Create config file using command: sudo python dpdk_setup_ports.py -i
141 In case of Ubuntu 16 need python3
142 See paragraph config creation for detailed step-by-step
143 (Refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html)
149 Step 2: Procedure to build SampleVNFs
151 a) Clone sampleVNF project repository - git clone https://git.opnfv.org/samplevnf
156 * Interactive options:
160 ./tools/vnf_build.sh -i
161 Follow the steps in the screen from option [1] –> [10] and select option [9] to build the vnfs.
162 It will automatically download selected DPDK version and any required patches and will setup everything and build VNFs.
164 Options [8], If RestAPI feature is needed install 'civetweb'
166 Following are the options for setup:
167 ----------------------------------------------------------
168 Step 1: Environment setup.
169 ----------------------------------------------------------
170 [1] Check OS and network connection
171 [2] Select DPDK RTE version
173 ----------------------------------------------------------
174 Step 2: Download and Install
175 ----------------------------------------------------------
176 [3] Agree to download
177 [4] Download packages
178 [5] Download DPDK zip
179 [6] Build and Install DPDK
181 [8] Download and Build civetweb
183 ----------------------------------------------------------
185 ----------------------------------------------------------
186 [9] Build all VNFs (vACL, vCGNAPT, vFW, UDP_Replay, DPPD-PROX)
191 * Non-Interactive options:
195 ./tools/vnf_build.sh -s -d=<dpdk version eg 17.02>
196 if system is behind the proxy
197 ./tools/vnf_build.sh -s -d=<dpdk version eg 17.02> -p=<proxy>
204 1) Download DPDK supported version from dpdk.org
205 * http://dpdk.org/browse/dpdk/snapshot/dpdk-$DPDK_RTE_VER.zip
206 * unzip dpdk-$DPDK_RTE_VER.zip and apply dpdk patches only in case of 16.04 (Not required for other DPDK versions)
208 * make config T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc
209 * cd x86_64-native-linuxapp-gcc
212 2) Download civetweb 1.9 version from the following link
213 * https://sourceforge.net/projects/civetweb/files/1.9/CivetWeb_V1.9.zip
214 * unzip CivetWeb_V1.9.zip
215 * mv civetweb-master civetweb
219 3) Add this to Go to /etc/default/grub configuration file to setup higepages.
220 * Append “default_hugepagesz=1G hugepagesz=1G hugepages=8 hugepagesz=2M hugepages=2048” to the GRUB_CMDLINE_LINUX entry.
221 * execute update-grub
222 * Reboot after grub setup
224 4) Setup Environment Variable
225 * export RTE_SDK=<samplevnf>/dpdk
226 * export RTE_TARGET=x86_64-native-linuxapp-gcc
227 * export VNF_CORE=<samplevnf> or using ./tools/setenv.sh
232 * or To build individual VNFs
233 * cd <samplevnf>/VNFs/
236 * The vFW executable will be created at the following location
237 * <samplevnf>/VNFs/vFW/build/vFW
240 Virtual Firewall - How to run
241 -----------------------------
243 Step 3: Bind the datapath ports to DPDK
245 a) Bind ports to DPDK
249 For DPDK versions 17.xx
250 1) cd <samplevnf>/dpdk
251 2) ./usertools/dpdk-devbind.py --status <--- List the network device
252 3) ./usertools/dpdk-devbind.py -b igb_uio <PCI Port 0> <PCI Port 1>
253 .. _More details: http://dpdk.org/doc/guides-17.05/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
256 b) Prepare script to enalble VNF to route the packets
260 cd <samplevnf>/VNFs/vFW/config
261 Open -> VFW_SWLB_SinglePortPair_script.tc. Replace the bold items based on your setting.
263 link 0 config <VNF port 0 IP eg 202.16.100.10> 8
266 link 1 config <VNF port 0 IP eg 172.16.40.10> 8
269 ; routeadd <net/host> <port #> <ipv4 nhip address in decimal> <Mask>
270 routeadd net 0 <traffic generator port 0 IP eg 202.16.100.20> 0xff000000
271 routeadd net 1 <traffic generator port 1 IP eg 172.16.40.20> 0xff000000
273 ; IPv4 static ARP; disable if dynamic arp is enabled.
274 p 1 arpadd 0 <traffic generator port 0 IP eg 202.16.100.20> <traffic generator port 0 MAC>
275 p 1 arpadd 1 <traffic generator port 1 IP eg 172.16.40.20> <traffic generator port 1 MAC>
276 p action add 0 accept
279 p action add 1 accept
284 p action add 0 conntrack
285 p action add 1 conntrack
286 p action add 2 conntrack
287 p action add 3 conntrack
289 p vfw add 1 <traffic generator port 0 IP eg 202.16.100.20> 8 <traffic generator port 1 IP eg 172.16.40.20> 8 0 65535 67 69 0 0 2
290 p vfw add 2 <traffic generator port 0 IP eg 202.16.100.20> 8 <traffic generator port 1 IP eg 172.16.40.20> 8 0 65535 0 65535 0 0 1
291 p vfw add 2 <traffic generator port 1 IP eg 172.16.40.20> 8 <traffic generator port 0 IP eg 202.16.100.20> 8 0 65535 0 65535 0 0 0
294 c) Run below cmd to launch the VNF. Please make sure both hugepages and ports to be used are bind to dpdk.
298 cd <samplevnf>/VNFs/vFW/
299 ./build/vFW -p 0x3 -f ./config/VFW_SWLB_SinglePortPair_4Thread.cfg -s ./config/VFW_SWLB_SinglePortPair_script.tc
302 step 4: Run Test using traffic geneator
306 On traffic generator system:
307 cd <trex eg v2.28/stl>
308 Update the bench.py to generate the traffic.
310 class STLBench(object):
312 ip_range['src'] = {'start': '<traffic generator port 0 IP eg 202.16.100.20>', 'end': '<traffic generator port 0 IP eg 202.16.100.20>'}
313 ip_range['dst'] = {'start': '<traffic generator port 1 IP eg 172.16.40.20>', 'end': '<traffic generator port 1 IP eg 172.16.40.20>'}
315 Run the TRex server: sudo ./t-rex-64 -i -c 7
316 In another shell run TRex console: trex-console
317 The console can be run from another computer with -s argument, --help for more info.
318 Other options for TRex client are automation or GUI
319 In the console, run "tui" command, and then send the traffic with commands like:
320 start -f stl/bench.py -m 50% --port 0 3 -t size=590,vm=var1
321 For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html
324 Virtual Access Control list - How to run
325 ----------------------------------------
327 Step 3: Bind the datapath ports to DPDK
329 a) Bind ports to DPDK
333 For DPDK versions 17.xx
334 1) cd <samplevnf>/dpdk
335 2) ./usertools/dpdk-devbind.py --status <--- List the network device
336 3) ./usertools/dpdk-devbind.py -b igb_uio <PCI Port 0> <PCI Port 1>
337 .. _More details: http://dpdk.org/doc/guides-17.05/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
340 b) Prepare script to enalble VNF to route the packets
344 cd <samplevnf>/VNFs/vACL/config
345 Open -> IPv4_swlb_acl.tc. Replace the bold items based on your setting.
347 link 0 config <VNF port 0 IP eg 202.16.100.10> 8
350 link 1 config <VNF port 0 IP eg 172.16.40.10> 8
353 ; routeadd <port #> <ipv4 nhip address in decimal> <Mask>
354 routeadd net 0 <traffic generator port 0 IP eg 202.16.100.20> 0xff000000
355 routeadd net 1 <traffic generator port 1 IP eg 172.16.40.20> 0xff000000
357 ; IPv4 static ARP; disable if dynamic arp is enabled.
358 p 1 arpadd 0 <traffic generator port 0 IP eg 202.16.100.20> <traffic generator port 0 MAC>
359 p 1 arpadd 1 <traffic generator port 1 IP eg 172.16.40.20> <traffic generator port 1 MAC>
360 p action add 0 accept
363 p action add 1 accept
368 p action add 0 conntrack
369 p action add 1 conntrack
370 p action add 2 conntrack
371 p action add 3 conntrack
373 p acl add 1 <traffic generator port 0 IP eg 202.16.100.20> 8 <traffic generator port 1 IP eg 172.16.40.20> 8 0 65535 67 69 0 0 2
374 p acl add 2 <traffic generator port 0 IP eg 202.16.100.20> 8 <traffic generator port 1 IP eg 172.16.40.20> 8 0 65535 0 65535 0 0 1
375 p acl add 2 <traffic generator port 1 IP eg 172.16.40.20> 8 <traffic generator port 0 IP eg 202.16.100.20> 8 0 65535 0 65535 0 0 0
379 c) Run below cmd to launch the VNF. Please make sure both hugepages and ports to be used are bind to dpdk.
383 cd <samplevnf>/VNFs/vFW/
384 ./build/vFW -p 0x3 -f ./config/IPv4_swlb_acl_1LB_1t.cfg -s ./config/IPv4_swlb_acl.tc.
387 step 4: Run Test using traffic geneator
391 On traffic generator system:
392 cd <trex eg v2.28/stl>
393 Update the bench.py to generate the traffic.
395 class STLBench(object):
397 ip_range['src'] = {'start': '<traffic generator port 0 IP eg 202.16.100.20>', 'end': '<traffic generator port 0 IP eg 202.16.100.20>'}
398 ip_range['dst'] = {'start': '<traffic generator port 1 IP eg 172.16.40.20>', 'end': '<traffic generator port 1 IP eg 172.16.40.20>'}
400 Run the TRex server: sudo ./t-rex-64 -i -c 7
401 In another shell run TRex console: trex-console
402 The console can be run from another computer with -s argument, --help for more info.
403 Other options for TRex client are automation or GUI
404 In the console, run "tui" command, and then send the traffic with commands like:
405 start -f stl/bench.py -m 50% --port 0 3 -t size=590,vm=var1
406 For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html
412 Step 3: Bind the datapath ports to DPDK
414 a) Bind ports to DPDK
418 For DPDK versions 17.xx
419 1) cd <samplevnf>/dpdk
420 2) ./usertools/dpdk-devbind.py --status <--- List the network device
421 3) ./usertools/dpdk-devbind.py -b igb_uio <PCI Port 0> <PCI Port 1>
422 .. _More details: http://dpdk.org/doc/guides-17.05/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
425 b) Prepare script to enalble VNF to route the packets
429 cd <samplevnf>/VNFs/vCGNAPT/config
430 Open -> sample_swlb_2port_2WT.tc Replace the bold items based on your setting.
432 link 0 config <VNF port 0 IP eg 202.16.100.10> 8
435 link 1 config <VNF port 0 IP eg 172.16.40.10> 8
438 ; uncomment to enable static NAPT
439 ;p <cgnapt pipeline id> entry addm <prv_ipv4/6> prvport> <pub_ip> <pub_port> <phy_port> <ttl> <no_of_entries> <end_prv_port> <end_pub_port>
440 ;p 5 entry addm 202.16.100.20 1234 152.16.40.10 1 0 500 65535 1234 65535
442 ; routeadd <net/host> <port #> <ipv4 nhip address in decimal> <Mask>
443 routeadd net 0 <traffic generator port 0 IP eg 202.16.100.20> 0xff000000
444 routeadd net 1 <traffic generator port 1 IP eg 172.16.40.20> 0xff000000
446 ; IPv4 static ARP; disable if dynamic arp is enabled.
447 p 1 arpadd 0 <traffic generator port 0 IP eg 202.16.100.20> <traffic generator port 0 MAC>
448 p 1 arpadd 1 <traffic generator port 1 IP eg 172.16.40.20> <traffic generator port 1 MAC>
450 For dynamic cgnapt. Please use UDP_Replay as one of the traffic generator
451 (TG1) (port 0) --> (port 0) VNF (CGNAPT) (Port 1) --> (port0)(UDPReplay)
453 c) Run below cmd to launch the VNF. Please make sure both hugepages and ports to be used are bind to dpdk.
457 cd <samplevnf>/VNFs/vCGNAPT/
458 ./build/vCGNAPT -p 0x3 -f ./config/sample_swlb_2port_2WT.cfg -s ./config/sample_swlb_2port_2WT.tc
460 d) Run UDP_replay to reflect the traffic on public side.
464 cmd: ./build/UDP_Replay -c 0x7 -n 4 -w <pci> -w <pci> -- --no-hw-csum -p <portmask> --config='(port, queue, cpucore)'
465 e.g ./build/UDP_Replay -c 0x7 -n 4 -w 0000:07:00.0 -w 0000:07:00.1 -- --no-hw-csum -p 0x3 --config='(0, 0, 1)(1, 0, 2)'
467 step 4: Run Test using traffic geneator
469 On traffic generator system:
472 cd <trex eg v2.28/stl>
473 Update the bench.py to generate the traffic.
475 class STLBench(object):
477 ip_range['src'] = {'start': '<traffic generator port 0 IP eg 202.16.100.20>', 'end': '<traffic generator port 0 IP eg 202.16.100.20>'}
478 ip_range['dst'] = {'start': '<traffic generator port 1 IP eg 172.16.40.20>', 'end': '<public ip e.g 152.16.40.10>'}
480 Run the TRex server: sudo ./t-rex-64 -i -c 7
481 In another shell run TRex console: trex-console
482 The console can be run from another computer with -s argument, --help for more info.
483 Other options for TRex client are automation or GUI
484 In the console, run "tui" command, and then send the traffic with commands like:
485 start -f stl/bench.py -m 50% --port 0 3 -t size=590,vm=var1
486 For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html
489 UDP_Replay - How to run
490 ----------------------------------------
492 Step 3: Bind the datapath ports to DPDK
494 a) Bind ports to DPDK
498 For DPDK versions 17.xx
499 1) cd <samplevnf>/dpdk
500 2) ./usertools/dpdk-devbind.py --status <--- List the network device
501 3) ./usertools/dpdk-devbind.py -b igb_uio <PCI Port 0> <PCI Port 1>
502 .. _More details: http://dpdk.org/doc/guides-17.05/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
504 b) Run below cmd to launch the VNF. Please make sure both hugepages and ports to be used are bind to dpdk.
508 cd <samplevnf>/VNFs/UDP_Replay/
509 cmd: ./build/UDP_Replay -c 0x7 -n 4 -w <pci> -w <pci> -- --no-hw-csum -p <portmask> --config='(port, queue, cpucore)'
510 e.g ./build/UDP_Replay -c 0x7 -n 4 -w 0000:07:00.0 -w 0000:07:00.1 -- --no-hw-csum -p 0x3 --config='(0, 0, 1)(1, 0, 2)'
513 step 4: Run Test using traffic geneator
517 On traffic generator system:
518 cd <trex eg v2.28/stl>
519 Update the bench.py to generate the traffic.
521 class STLBench(object):
523 ip_range['src'] = {'start': '<traffic generator port 0 IP eg 202.16.100.20>', 'end': '<traffic generator port 0 IP eg 202.16.100.20>'}
524 ip_range['dst'] = {'start': '<traffic generator port 1 IP eg 172.16.40.20>', 'end': '<public ip e.g 152.16.40.10>'}
526 Run the TRex server: sudo ./t-rex-64 -i -c 7
527 In another shell run TRex console: trex-console
528 The console can be run from another computer with -s argument, --help for more info.
529 Other options for TRex client are automation or GUI
530 In the console, run "tui" command, and then send the traffic with commands like:
531 start -f stl/bench.py -m 50% --port 0 3 -t size=590,vm=var1
532 For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html
540 This is PROX, the Packet pROcessing eXecution engine, part of Intel(R)
541 Data Plane Performance Demonstrators, and formerly known as DPPD-BNG.
542 PROX is a DPDK-based application implementing Telco use-cases such as
543 a simplified BRAS/BNG, light-weight AFTR... It also allows configuring
544 finer grained network functions like QoS, Routing, load-balancing...
546 PROX COMMANDS AND SCREENS
547 -------------------------
549 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
550 | *RUNTIME COMMAND* | *DESCRIPTION* | *EXAMPLE* |
551 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
552 | quit | Stop all cores and quit | |
553 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
554 | help <substr> | Show list of commands that have <substr> as a substring. | |
555 | | If no substring is provided, all commands are shown. | |
556 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
557 | verbose <level> | Set the verbosity level of some printed messages. | |
558 | | Possible values are: 0 (default value, error messages only), | verbose 1 |
559 | | 1 (+ warnings), 2 (+ info) and 3 (+ debugging) | |
560 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
561 | thread info <core_id> <task_id> | Show task specific information | |
562 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
563 | update interval <value> | Update statistics refresh rate, in msec (must be >=10). | |
564 | | Default is 1 second | update interval 500 |
565 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
566 | rx tx info | Print connections between tasks on all cores | |
567 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
568 | start <core list>|all <task_id> | Start cores specified in <core list> or all cores. | start all |
569 | | If <task_id> is not specified, all tasks for the specified cores | start 1 |
570 | | will be started. | start 1s0-4s0 |
571 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
572 | stop <core list>|all <task_id> | Stop cores specified in <core list> or all cores. | |
573 | | If <task_id> is not specified, all tasks for the specified | stop 1 |
574 | | cores will be stopped. | |
575 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
576 | dump <coreid> <taskid> <nbpkts> | Create a hex dump of <nb_packets> from <task_id> on <core_id> | dump 2 1 5 |
577 | | showing how packets have changed between RX and TX. | |
578 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
579 | dump_rx <coreid> <taskid> <nbpkts> | Create a hex dump of <nb_packets> from <task_id> on <coreid> at RX | dump_rx 2 1 5 |
580 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
581 | dump_tx <coreid> <taskid> <nbpkts> | Create a hex dump of <nb_packets> from <task_id> on <coreid> at TX | dump_tx 2 1 5 |
582 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
583 | rx distr start | Start gathering statistical distribution of received packets | |
584 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
585 | rx distr stop | Stop gathering statistical distribution of received packets | |
586 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
587 | rx distr reset | Reset gathered statistical distribution of received packets | |
588 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
589 | rx distr show | Display gathered statistical distribution of received packets | |
590 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
591 | rate <port id> <queue id> <rate> | Set transmit rate in Mb/s. This does not include preamble, SFD and IFG | rate 0 0 1000 |
592 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
593 | count <core id> <task id> <count> | Generate <count> packets, then pause generating | count 1 0 5 |
594 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
595 | pkt_size <coreid> <taskid> <pktsize> | Set the packet size to <pkt_size> | pkt_size 1 3 255 |
596 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
597 | speed <core_id> <task_id> <speed percentage> | Change the speed to <speed percentage> of a | |
598 | | 10 Gbps line at which packets are being generated | speed 1 0 50 |
599 | | on core <core_id> in task <task_id> | |
600 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
601 | speed_byte <core_id> <task_id> <speed> | Change speed to <speed>. The speed is specified in units of bytes per sec | |
602 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
603 | set value <core_id> <task_id> <offset> | Set <value_len> bytes to <value> at offset <offset> in packets | |
604 | <value> <value_len> | generated on <core_id> <task_id> | set value 4 1 14 10 1 |
605 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
606 | reset values all | Undo all `set value` commands on all cores/tasks | |
607 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
608 | reset values <core id> <task id> | Undo all `set value` commands on specified core/task | |
609 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
610 | arp add <core id> <task id> <port id> | | |
611 | <gre id> <svlan> <cvlan> <ip addr> | | |
612 | <mac addr> <user> | Add a single ARP entry into a CPE table on <core id>/<task id> | |
613 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
614 | rule add <core id> <task id> svlan_id&mask | | |
615 | cvlan_id&mask ip_proto&mask | | |
616 | source_ip/prefix destination_ip/prefix | | |
617 | range dport_range action | Add a rule to the ACL table on <core id>/<task id> | |
618 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
619 | route add <core id> <task id> | | |
620 | <ip/prefix> <next hop id> | Add a route to the routing table on core <core id> <task id> | route add 10.0.16.0/24 9 |
621 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
622 | reset stats | Reset all statistics | |
623 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
624 | tot stats | Print total RX and TX packets | |
625 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
626 | tot ierrors per sec | Print total number of ierrors per second | |
627 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
628 | pps stats | Print RX and TX packet rate in unit of packet per second | |
629 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
630 | lat stats <core id> <task id> | Print min,max,avg latency as measured during last sampling interval | lat stats 1 0 |
631 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
632 | lat packets <core id> <task id> | Print the latency for each of the last set of packets | |
633 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
634 | core stats <core id> <task id> | Print rx/tx/drop for task <task id> running on core <core id> | |
635 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
636 | port_stats <port id> | Print rate for no_mbufs, ierrors, rx_bytes, tx_bytes, rx_pkts, | |
637 | | tx_pkts and totals for RX, TX, no_mbufs ierrors for port <port id> | |
638 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
639 | ring info all | Get information about ring, such as ring size and | |
640 | | number of elements in the ring | |
641 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
642 | ring info <core id> <task id> | Get information about ring on core <core id> | |
643 | | in task <task id>, such as ring size and number of elements in the ring | ring info 1 0 |
644 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
645 | port info <port id> [brief] | Get port related information, such as MAC address, socket, | |
646 | | number of descriptors..., . Adding `brief` after command | |
647 | | prints short version of output. | port info 1 |
648 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
649 | port up <port id> | Set the port up (all ports are up at startup) | port up 1 |
650 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
651 | port down <port id> | Set the port down | port down 1 |
652 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
653 | port xstats <port id> | Get extra statistics for the port | port xstats 1 |
654 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
655 | version | Show version | |
656 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
657 | port_stats <port id> | Print rate for no_mbufs, ierrors, rx_bytes, tx_bytes, rx_pkts, | |
658 | | tx_pkts and totals for RX, TX, no_mbufs ierrors for port <port id> | |
659 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
661 While PROX is running, F1 to F6 change the view on the system. Pressing F1 switches to the main screen showing per core statistics. When PROX is started,
662 this is the screen shown by default. Pressing F2 switches to show port-based information. Pressing F3 shows information (i.e. occupancy, memory usage, ...)
663 about memory pools. If there are tasks with mode=lat, F4 displays latency measurements made during the last second by each of those tasks.
664 F5 displays DPDK ring information. F6 is for L4 generation. If no command has been entered, numbers 1 to 6 can also be used to change the view on the system.
665 This is provided to allow changing screens in environments that do not pass function keys to PROX.
667 Page Up and Page Down can be used to view per core statistics that would otherwise not fit on the screen. Escape quits PROX.
668 The history of previously entered commands can be navigated using the Up and Down arrows. Statistics can be reset with F12.
672 Run PROX with the "--help" argument to display the usage text and the list of supported options as shown below.
673 PROX supports many compilation flags to enable or disable features. For these flags, refer to the Makefile.
674 Refer to the README file for more information on how to run PROX for specific use cases.
678 Usage: ./build/prox [-f CONFIG_FILE] [-l LOG_FILE] [-p] [-o DISPLAY] [-v] [-a|-e] [-m|-s|-i] [-n] [-w DEF] [-q] [-k] [-d] [-z] [-r VAL] [-u] [-t]
679 -f CONFIG_FILE : configuration file to load, ./prox.cfg by default
680 -l LOG_FILE : log file name, ./prox.log by default
681 -p : include PID in log file name if default log file is used
682 -o DISPLAY: Set display to use, can be 'curses' (default), 'cli' or 'none'
683 -v verbosity : initial logging verbosity
684 -a : autostart all cores (by default)
686 -n : Create NULL devices instead of using PCI devices, useful together with -i
687 -m : list supported task modes and exit
688 -s : check configuration file syntax and exit
689 -i : check initialization sequence and exit
690 -u : Listen on UDS /tmp/prox.sock
691 -t : Listen on TCP port 8474
692 -q : Pass argument to Lua interpreter, useful to define variables
693 -w : define variable using syntax varname=value
694 takes precedence over variables defined in CONFIG_FILE
695 -k : Log statistics to file "stats_dump" in current directory
696 -d : Run as daemon, the parent process will block until PROX is not initialized
697 -z : Ignore CPU topology, implies -i
698 -r : Change initial screen refresh rate. If set to a lower than 0.001 seconds,
699 screen refreshing will be disabled
701 CONFIGURATION FILE FORMAT
702 -------------------------
703 The configuration file is divided into multiple sections, each of which is used to define some parameters and options.
704 Sections are created using the [section name] syntax. The list of sections, where # represents an integer, is as follows:
715 In each section, entries are created using the key=value syntax.
716 Comments are created using the ; symbol: all characters from the ;
717 symbol to the end of line are ignored. A # symbol at the beginning of the section name comments
718 the whole section out: all entries in the section are treated as comments and are ignored. For example:
724 parameter name=parameter value ; this entry is ignored because the section is commented out
726 * [EAL OPTIONS]: The following parameters are supported:
730 -m ; Specifies the amount of memory used. If not provided, all hugepages will be used.
731 -n ; Specifies the number of memory channels. Use -n4 for latest Intel Xeon based platforms
732 -r ; Specifies the number of memory ranks.
733 eal ; Specifies DPDK EAL extra options. Those options will be passed blindly to DPDK.
735 * [PORT #]: DPDK ports are usually referenced by their port_id, i.e. an integer starting from 0.
736 Using port_id in the configuration file is tedious, since the same port_id can appear at
737 different places (rx port, tx port, routing tables), and those ports might change (e.g. if cables are swapped).
738 In order to make the configuration file easier to read and modify, DPDK ports are given a name with the name= option.
739 The name serves as the reference, and in addition, it will show up in the display at runtime.
743 PARAMETER EXAMPLE DESCRIPTION
744 ----------------------------------------------------------------------------
745 name inet0 Use inet0 to later refer to this port
746 mac hardware value can be: hardware, random or a literal MAC address
747 rx desc 256 number of descriptors to allocate for reception
748 tx desc 256 number of descriptors to allocate for transmission
749 promiscuous yes enable promiscuous mode
750 strip crc yes enable CRC stripping
752 lsc no While lsc is disabled for drivers known to not provide support,
753 this option explicitely overrides these settings.
754 rx_ring dpdk_ring_name use DPDK ring as an interface (receive side)
755 tx_ring dpdk_ring_name use DPDK ring as an interface (transmit side)
757 * [VARIABLES]: Variables can be defined in the configuration file using the $varname=value syntax.
758 Variables defined on the command line (-w varname=value) take precedence and do not create
759 conflicts with variables defined in the configuration file. Variables are used in the
760 configuration file using the $varname syntax: each instance of $varname is replaced by its
761 associated value. This is typically useful if the same parameter must be used at several places.
762 For instance, you might want to have multiple load balancers, all transmitting to the same set
763 of worker cores. The list of worker cores could then be defined once in a variable:
770 Then, a load balancer definition would use the variable:
781 And the section defining the worker cores would be:
791 * [DEFAULTS]: The default value of some options can be overridden using the [defaults] section:
795 PARAMETER EXAMPLE DESCRIPTION
796 -----------------------------------
797 mempool size 16K number of mbufs per task, relevant when task receives from a port.
798 this is the n argument provided to rte_mempool_create()
799 qinq tag 0xa888 Set qinq tag for all tasks. The result of adding this option is the
800 same as adding qinq tag= to each task
801 memcache size 128 number of mbufs cached per core, default is 256 this is the cache_size
802 argument provided to rte_mempool_create()
804 * [GLOBAL]: The following parameters are supported:
808 PARAMETER EXAMPLE DESCRIPTION
809 -------------------------------------------------
810 name BNG Name of the configuration, which will be shown in the title box at runtime.
811 start time 10 Time in seconds after which average statistics will be started.
813 duration time 30 Runtime duration in seconds, counted after start time.
814 This is typically useful to automate testing using
815 different parameters: PROX automatically exits when the
816 runtime duration has elapsed. Initialization and start time
817 are not included in this runtime duration.
818 For example, if start time is set to 10 and duration time is set to 30,
819 the total execution time (after initialization) will be 40 seconds.
820 Default value is 0, which means infinity and prevents PROX from automatically exiting.
821 shuffle yes When this parameter is set to yes, the order of mbufs
822 within mempools is randomized to simulate a system that has
823 been warmed up. Default value is no.
824 gre cfg /path/to/file.csv Path to CSV file that provides QinQ-to-GRE mapping.
825 Default value is gre_table.csv in same directory as
826 configuration file. Fields are GRE key and QinQ value (computed as SVLAN * 4096 + CVLAN).
827 pre cmd ls Arbitrary system commands to run while reading cfg. This option can occur multiple times.
828 user cfg /path/to/file.csv Path to CSV file that provides QinQ-to-User mapping.
829 Default value is user_table.csv in same directory as configuration file.
830 Fields are SVLAN, CVLAN and User-Id.
831 next hop cfg /path/to/file.csv Path to CSV file that provides Next-Hop details.
832 Default value is next_hop.csv in same directory as configuration file.
833 Fields are Next-Hop index (as returned by LPM lookup),
834 Out-Port index, Next-Hop IP (unused), Next-Hop MAC and MPLS label.
835 ipv4 cfg /path/to/file.csv Path to CSV file that provides IPv4 LPM routing table.
836 Default value is ipv4.csv in same directory as configuration file.
837 Fields are IPv4 subnet (in CIDR notation) and Next-Hop index.
838 dscp cfg /path/to/file.csv Path to CSV file that provides mapping for QoS classification,
839 from DSCP to Traffic Class and Queue.
840 Default value is dscp.csv in same directory as configuration file.
841 Fields are DSCP (0-63), Traffic Class (0-3) and Queue (0-3).
842 ipv6 tunnel cfg /path/to/file.csv Path to CSV file that provides lwAFTR binding table.
843 Default value is ipv6_tun_bind.csv in same directory as configuration file.
844 Fields are lwB4 IPv6 address, next hop MAC address towards lwB4,
845 IPv4 Public address and IPv4 Public Port Set.
846 acl cfg /path/to/file.csv Path to CSV file that provides ACL rules.
847 Default value is rules.csv in same directory as configuration file.
848 Fields are SVLAN value & mask, CVLAN value & mask, IP protocol value & mask,
849 source IPv4 subnet (in CIDR notation), destination IPv4 subnet (in CIDR notation),
850 source port range, destination port range, and action (drop, allow, rate limit).
854 * [CORE #]: Cores can be configured by means of a set of [core #] sections, where # represents either:
856 an absolute core number: e.g. on a 10-core, dual socket system with hyper-threading, cores are numbered from 0 to 39;
857 a core number, the letter 's', and a socket number: this allows selecting per-socket cores, independently from their interleaved numbering;
858 a core number and the letter 'h': this allows selecting the hyper-thread sibling of the specified core;
859 a dash-separated range of core numbers; a comma-separated list of core numbers; any combination of the above;
860 or a variable whose value complies with the above syntax.
861 The socket and hyper-thread syntax makes it easier to use the same configuration file on several platforms,
862 even if their core numbering differs (e.g. interleaving rule or number of cores per socket).
864 Each core can be assigned with a set of tasks, each running one of the implemented packet processing modes.
866 The following parameters are supported:
868 .. image:: images/prox_core.png
870 :alt: SampleVNF supported topology
872 Compiling and running this application
873 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
877 DPDK must be installed prior to running make in the PROX directory.
878 The README file shipped with PROX describes what versions of DPDK are supported,
879 and if any patches are needed for the chosen DPDK version.
881 The following packages need to be installed. (Example for destributions that are using rpm)
885 sudo yum install net-tools wget gcc unzip libpcap-devel ncurses-devel libedit-devel pciutils lua-devel kernel-devel
888 The following instructions are here to help customers to start using PROX.
889 It's by no means a complete guide, for detailed instructions on how to install and use
890 DPDK please refer to its documentation.
891 Your mileage may vary depending on a particular Linux distribution and hardware in use.
893 Edit grub default configuration:
899 Add the following to the kernel boot parameters
903 default_hugepagesz=1G hugepagesz=1G hugepages=8
905 Rebuild grub config and reboot the system:
909 grub2-mkconfig -o /boot/grub2/grub.cfg
912 Verify that hugepages are available
920 Hugepagesize: 1048576 kB
928 umount `awk '/hugetlbfs/ { print $2 }' /proc/mounts` >/dev/null 2>&1
929 mount -t hugetlbfs nodev /mnt/huge/
931 This application supports DPDK 16.04, 16.11, 17.02 and 17.05.
932 The following commands assume that the following variables have been set:
934 export RTE_SDK=/path/to/dpdk
935 export RTE_TARGET=x86_64-native-linuxapp-gcc
937 PROX Compiation installation
938 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
940 * git clone https://git.opnfv.org/samplevnf
942 * export RTE_SDK=`pwd`/dpdk
943 * export RTE_TARGET=x86_64-native-linuxapp-gcc
944 * git clone http://dpdk.org/git/dpdk
946 * git checkout v17.05
947 * make install T=$RTE_TARGET
948 * cd <samplevnf>/VNFs/DPPD-PROX
955 * git clone https://git.opnfv.org/samplevnf
957 * ./tools/vnf_build.sh -s -d='17.05' [-p=<proxy> if behind the proxy]
963 lsmod | grep -w "^uio" >/dev/null 2>&1 || sudo modprobe uio
970 lsmod | grep -w "^igb_uio" >/dev/null 2>&1 || sudo insmod $RTE_SDK/$RTE_TARGET/kmod/igb_uio.ko
972 Discover network devices available on the system:
976 lspci | grep Ethernet
978 Prior launching PROX, ports that are to be used by it must be bound to the igb_uio driver.
980 The following command will bind all Intel® Ethernet Converged Network Adapter X710 ports to igb_uio:
984 lspci | grep X710 | cut -d' ' -f 1 | sudo xargs -I {} python2.7 $RTE_UNBIND --bind=igb_uio {}
986 The following command will bind all Intel® 82599 10 Gigabit Ethernet Controller ports to igb_uio:
990 lspci | grep 82599 | cut -d' ' -f 1 | sudo xargs -I {} python2.7 $RTE_UNBIND --bind=igb_uio {}
995 The Makefile with this application expects RTE_SDK to point to the
996 root directory of DPDK (e.g. export RTE_SDK=/root/dpdk). If RTE_TARGET
997 has not been set, x86_64-native-linuxapp-gcc will be assumed.
1002 After DPDK has been set up, run make from the directory where you have
1003 extracted this application. A build directory will be created
1004 containing the PROX executable. The usage of the application is shown
1005 below. Note that this application assumes that all required ports have
1006 been bound to the DPDK provided igb_uio driver. Refer to the "Getting
1007 Started Guide - DPDK" document for more details.
1011 Usage: ./build/prox [-f CONFIG_FILE] [-l LOG_FILE] [-p] [-o DISPLAY] [-v] [-a|-e] [-m|-s|-i] [-n] [-w DEF] [-q] [-k] [-d] [-z] [-r VAL] [-u] [-t]
1012 -f CONFIG_FILE : configuration file to load, ./prox.cfg by default
1013 -l LOG_FILE : log file name, ./prox.log by default
1014 -p : include PID in log file name if default log file is used
1015 -o DISPLAY: Set display to use, can be 'curses' (default), 'cli' or 'none'
1016 -v verbosity : initial logging verbosity
1017 -a : autostart all cores (by default)
1018 -e : don't autostart
1019 -n : Create NULL devices instead of using PCI devices, useful together with -i
1020 -m : list supported task modes and exit
1021 -s : check configuration file syntax and exit
1022 -i : check initialization sequence and exit
1023 -u : Listen on UDS /tmp/prox.sock
1024 -t : Listen on TCP port 8474
1025 -q : Pass argument to Lua interpreter, useful to define variables
1026 -w : define variable using syntax varname=value
1027 takes precedence over variables defined in CONFIG_FILE
1028 -k : Log statistics to file "stats_dump" in current directory
1029 -d : Run as daemon, the parent process will block until PROX is not initialized
1030 -z : Ignore CPU topology, implies -i
1031 -r : Change initial screen refresh rate. If set to a lower than 0.001 seconds,
1032 screen refreshing will be disabled
1034 While applications using DPDK typically rely on the core mask and the
1035 number of channels to be specified on the command line, this
1036 application is configured using a .cfg file. The core mask and number
1037 of channels is derived from this config. For example, to run the
1038 application from the source directory execute:
1042 user@target:~$ ./build/prox -f ./config/nop.cfg
1044 Provided example configurations
1045 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1046 PROX can be configured either as the SUT (System Under Test) or as the
1047 Traffic Generator. Some example configuration files are provided, both
1048 in the config directory to run PROX as a SUT, and in the gen directory
1049 to run it as a Traffic Generator.
1050 A quick description of these example configurations is provided below.
1051 Additional details are provided in the example configuration files.
1053 Basic configurations, mostly used as sanity check:
1058 * config/nop-rings.cfg
1061 Simplified BNG (Border Network Gateway) configurations, using different
1062 number of ports, with and without QoS, running on the host or in a VM:
1066 * config/bng-4ports.cfg
1067 * config/bng-8ports.cfg
1068 * config/bng-qos-4ports.cfg
1069 * config/bng-qos-8ports.cfg
1070 * config/bng-1q-4ports.cfg
1071 * config/bng-ovs-usv-4ports.cfg
1072 * config/bng-no-cpu-topology-4ports.cfg
1073 * gen/bng-4ports-gen.cfg
1074 * gen/bng-8ports-gen.cfg
1075 * gen/bng-ovs-usv-4ports-gen.cfg
1077 Light-weight AFTR configurations:
1081 * config/lw_aftr.cfg
1082 * gen/lw_aftr-gen.cfg