1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, Intel Corporation and others.
14 The device under test (DUT) consists of a system following;
15 * A single or dual processor and PCH chip, except for System on Chip (SoC) cases
16 * DRAM memory size and frequency (normally single DIMM per channel)
17 * Specific Intel Network Interface Cards (NICs)
18 * BIOS settings noting those that updated from the basic settings
19 * DPDK build configuration settings, and commands used for tests
21 Connected to the DUT is an IXIA* or Software Traffic generator like pktgen or TRex,
22 simulation platform to generate packet traffic to the DUT ports and
23 determine the throughput/latency at the tester side.
25 Below are the supported/tested (:term:`VNF`) deployment type.
27 .. image:: images/deploy_type.png
29 :alt: SampleVNF supported topology
31 Hardware & Software Ingredients
32 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
36 +-----------+------------------+
37 | Item | Description |
38 +-----------+------------------+
40 +-----------+------------------+
42 +-----------+------------------+
43 | OS | Ubuntu 16.04 LTS |
44 +-----------+------------------+
45 | kernel | 4.4.0-34-generic |
46 +-----------+------------------+
48 +-----------+------------------+
50 Boot and BIOS settings:
52 +------------------+---------------------------------------------------+
53 | Boot settings | default_hugepagesz=1G hugepagesz=1G hugepages=16 |
54 | | hugepagesz=2M hugepages=2048 isolcpus=1-11,22-33 |
55 | | nohz_full=1-11,22-33 rcu_nocbs=1-11,22-33 |
56 | | Note: nohz_full and rcu_nocbs is to disable Linux*|
57 | | kernel interrupts, and it’s import |
58 +------------------+---------------------------------------------------+
59 | BIOS | CPU Power and Performance Policy <Performance> |
60 | | CPU C-state Disabled |
61 | | CPU P-state Disabled |
62 | | Enhanced Intel® Speedstep® Tech Disabled |
63 | | Hyper-Threading Technology (If supported) Enable |
64 | | Virtualization Techology Enable |
65 | | Coherency Enable |
66 | | Turbo Boost Disabled |
67 +------------------+---------------------------------------------------+
69 Network Topology for testing VNFs
70 ---------------------------------
71 The ethernet cables should be connected between traffic generator and the VNF server (BM,
72 SRIOV or OVS) setup based on the test profile.
74 The connectivity could be
76 1) Single port pair : One pair ports used for traffic
80 e.g. Single port pair link0 and link1 of VNF are used
81 TG:port 0 <------> VNF:Port 0
82 TG:port 1 <------> VNF:Port 1
84 For correalted traffic, use below configuration
85 TG_1:port 0 <------> VNF:Port 0
86 VNF:Port 1 <------> TG_2:port 0 (UDP Replay)
87 (TG_2(UDP_Replay) reflects all the traffic on the given port)
89 2) Multi port pair : More than one pair of traffic
93 e.g. Two port pair link 0, link1, link2 and link3 of VNF are used
94 TG:port 0 <------> VNF:Port 0
95 TG:port 1 <------> VNF:Port 1
96 TG:port 2 <------> VNF:Port 2
97 TG:port 3 <------> VNF:Port 3
99 For correalted traffic, use below configuration
100 TG_1:port 0 <------> VNF:Port 0
101 VNF:Port 1 <------> TG_2:port 0 (UDP Replay)
102 TG_1:port 1 <------> VNF:Port 2
103 VNF:Port 3 <------> TG_2:port 1 (UDP Replay)
104 (TG_2(UDP_Replay) reflects all the traffic on the given port)
107 Refer: http://fast.dpdk.org/doc/pdf-guides/ to setup the DUT for VNF to run
109 * Standalone Virtualization - PHY-VM-PHY
112 https://software.intel.com/en-us/articles/using-sr-iov-to-share-an-ethernet-port-among-multiple-vms
115 http://docs.openvswitch.org/en/latest/intro/install/general/
116 http://docs.openvswitch.org/en/latest/intro/install/dpdk/
119 Use any OPNFV installer to deploy the openstack.
121 Setup Traffic generator
122 -----------------------
124 Step 0: Preparing hardware connection
126 Connect Traffic generator and VNF system back to back as shown in previous section
130 TRex port 0 ↔ (VNF Port 0) ↔ (VNF Port 1) ↔ TRex port 1
132 Step 1: Setting up Traffic generator (TRex)
134 TRex Software preparations
135 * Install the OS (Bare metal Linux, not VM!)
136 * Obtain the latest TRex package: wget https://trex-tgn.cisco.com/trex/release/latest
137 * Untar the package: tar -xzf latest
138 * Change dir to unzipped TRex
139 * Create config file using command: sudo python dpdk_setup_ports.py -i
141 In case of Ubuntu 16 need python3
143 See paragraph config creation for detailed step-by-step
145 (Refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html)
151 Step 2: Procedure to build SampleVNFs
153 a) Clone sampleVNF project repository - git clone https://git.opnfv.org/samplevnf
158 * Interactive options:
162 ./tools/vnf_build.sh -i
163 Follow the steps in the screen from option [1] –> [10] and select option [9] to build the vnfs.
164 It will automatically download selected DPDK version and any required patches and will setup everything and build VNFs.
166 Options [8], If RestAPI feature is needed install 'civetweb'
168 Following are the options for setup:
169 ----------------------------------------------------------
170 Step 1: Environment setup.
171 ----------------------------------------------------------
172 [1] Check OS and network connection
173 [2] Select DPDK RTE version
175 ----------------------------------------------------------
176 Step 2: Download and Install
177 ----------------------------------------------------------
178 [3] Agree to download
179 [4] Download packages
180 [5] Download DPDK zip
181 [6] Build and Install DPDK
183 [8] Download and Build civetweb
185 ----------------------------------------------------------
187 ----------------------------------------------------------
188 [9] Build all VNFs (vACL, vCGNAPT, vFW, UDP_Replay, DPPD-PROX)
193 * Non-Interactive options:
197 ./tools/vnf_build.sh -s -d=<dpdk version eg 17.02>
198 if system is behind the proxy
199 ./tools/vnf_build.sh -s -d=<dpdk version eg 17.02> -p=<proxy>
206 1) Download DPDK supported version from dpdk.org
207 * http://dpdk.org/browse/dpdk/snapshot/dpdk-$DPDK_RTE_VER.zip
208 * unzip dpdk-$DPDK_RTE_VER.zip and apply dpdk patches only in case of 16.04 (Not required for other DPDK versions)
210 * make config T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc
211 * cd x86_64-native-linuxapp-gcc
214 2) Download civetweb 1.9 version from the following link
215 * https://sourceforge.net/projects/civetweb/files/1.9/CivetWeb_V1.9.zip
216 * unzip CivetWeb_V1.9.zip
217 * mv civetweb-master civetweb
221 3) Add this to Go to /etc/default/grub configuration file to setup higepages.
222 * Append “default_hugepagesz=1G hugepagesz=1G hugepages=8 hugepagesz=2M hugepages=2048” to the GRUB_CMDLINE_LINUX entry.
223 * execute update-grub
224 * Reboot after grub setup
226 4) Setup Environment Variable
227 * export RTE_SDK=<samplevnf>/dpdk
228 * export RTE_TARGET=x86_64-native-linuxapp-gcc
229 * export VNF_CORE=<samplevnf> or using ./tools/setenv.sh
234 * or To build individual VNFs
235 * cd <samplevnf>/VNFs/
238 * The vFW executable will be created at the following location
239 * <samplevnf>/VNFs/vFW/build/vFW
242 Virtual Firewall - How to run
243 -----------------------------
245 Step 3: Bind the datapath ports to DPDK
247 a) Bind ports to DPDK
251 For DPDK versions 17.xx
252 1) cd <samplevnf>/dpdk
253 2) ./usertools/dpdk-devbind.py --status <--- List the network device
254 3) ./usertools/dpdk-devbind.py -b igb_uio <PCI Port 0> <PCI Port 1>
255 .. _More details: http://dpdk.org/doc/guides-17.05/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
258 b) Prepare script to enalble VNF to route the packets
262 cd <samplevnf>/VNFs/vFW/config
263 Open -> VFW_SWLB_SinglePortPair_script.tc. Replace the bold items based on your setting.
265 link 0 config <VNF port 0 IP eg 202.16.100.10> 8
268 link 1 config <VNF port 0 IP eg 172.16.40.10> 8
271 ; routeadd <net/host> <port #> <ipv4 nhip address in decimal> <Mask>
272 routeadd net 0 <traffic generator port 0 IP eg 202.16.100.20> 0xff000000
273 routeadd net 1 <traffic generator port 1 IP eg 172.16.40.20> 0xff000000
275 ; IPv4 static ARP; disable if dynamic arp is enabled.
276 p 1 arpadd 0 <traffic generator port 0 IP eg 202.16.100.20> <traffic generator port 0 MAC>
277 p 1 arpadd 1 <traffic generator port 1 IP eg 172.16.40.20> <traffic generator port 1 MAC>
278 p action add 0 accept
281 p action add 1 accept
286 p action add 0 conntrack
287 p action add 1 conntrack
288 p action add 2 conntrack
289 p action add 3 conntrack
291 p vfw add 1 <traffic generator port 0 IP eg 202.16.100.20> 8 <traffic generator port 1 IP eg 172.16.40.20> 8 0 65535 67 69 0 0 2
292 p vfw add 2 <traffic generator port 0 IP eg 202.16.100.20> 8 <traffic generator port 1 IP eg 172.16.40.20> 8 0 65535 0 65535 0 0 1
293 p vfw add 2 <traffic generator port 1 IP eg 172.16.40.20> 8 <traffic generator port 0 IP eg 202.16.100.20> 8 0 65535 0 65535 0 0 0
296 c) Run below cmd to launch the VNF. Please make sure both hugepages and ports to be used are bind to dpdk.
300 cd <samplevnf>/VNFs/vFW/
301 ./build/vFW -p 0x3 -f ./config/VFW_SWLB_SinglePortPair_4Thread.cfg -s ./config/VFW_SWLB_SinglePortPair_script.tc
304 step 4: Run Test using traffic geneator
308 On traffic generator system:
309 cd <trex eg v2.28/stl>
310 Update the bench.py to generate the traffic.
312 class STLBench(object):
314 ip_range['src'] = {'start': '<traffic generator port 0 IP eg 202.16.100.20>', 'end': '<traffic generator port 0 IP eg 202.16.100.20>'}
315 ip_range['dst'] = {'start': '<traffic generator port 1 IP eg 172.16.40.20>', 'end': '<traffic generator port 1 IP eg 172.16.40.20>'}
317 Run the TRex server: sudo ./t-rex-64 -i -c 7
318 In another shell run TRex console: trex-console
319 The console can be run from another computer with -s argument, --help for more info.
320 Other options for TRex client are automation or GUI
321 In the console, run "tui" command, and then send the traffic with commands like:
322 start -f stl/bench.py -m 50% --port 0 3 -t size=590,vm=var1
323 For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html
326 Virtual Access Control list - How to run
327 ----------------------------------------
329 Step 3: Bind the datapath ports to DPDK
331 a) Bind ports to DPDK
335 For DPDK versions 17.xx
336 1) cd <samplevnf>/dpdk
337 2) ./usertools/dpdk-devbind.py --status <--- List the network device
338 3) ./usertools/dpdk-devbind.py -b igb_uio <PCI Port 0> <PCI Port 1>
339 .. _More details: http://dpdk.org/doc/guides-17.05/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
342 b) Prepare script to enalble VNF to route the packets
346 cd <samplevnf>/VNFs/vACL/config
347 Open -> IPv4_swlb_acl.tc. Replace the bold items based on your setting.
349 link 0 config <VNF port 0 IP eg 202.16.100.10> 8
352 link 1 config <VNF port 0 IP eg 172.16.40.10> 8
355 ; routeadd <port #> <ipv4 nhip address in decimal> <Mask>
356 routeadd net 0 <traffic generator port 0 IP eg 202.16.100.20> 0xff000000
357 routeadd net 1 <traffic generator port 1 IP eg 172.16.40.20> 0xff000000
359 ; IPv4 static ARP; disable if dynamic arp is enabled.
360 p 1 arpadd 0 <traffic generator port 0 IP eg 202.16.100.20> <traffic generator port 0 MAC>
361 p 1 arpadd 1 <traffic generator port 1 IP eg 172.16.40.20> <traffic generator port 1 MAC>
362 p action add 0 accept
365 p action add 1 accept
370 p action add 0 conntrack
371 p action add 1 conntrack
372 p action add 2 conntrack
373 p action add 3 conntrack
375 p acl add 1 <traffic generator port 0 IP eg 202.16.100.20> 8 <traffic generator port 1 IP eg 172.16.40.20> 8 0 65535 67 69 0 0 2
376 p acl add 2 <traffic generator port 0 IP eg 202.16.100.20> 8 <traffic generator port 1 IP eg 172.16.40.20> 8 0 65535 0 65535 0 0 1
377 p acl add 2 <traffic generator port 1 IP eg 172.16.40.20> 8 <traffic generator port 0 IP eg 202.16.100.20> 8 0 65535 0 65535 0 0 0
381 c) Run below cmd to launch the VNF. Please make sure both hugepages and ports to be used are bind to dpdk.
385 cd <samplevnf>/VNFs/vFW/
386 ./build/vFW -p 0x3 -f ./config/IPv4_swlb_acl_1LB_1t.cfg -s ./config/IPv4_swlb_acl.tc.
389 step 4: Run Test using traffic geneator
393 On traffic generator system:
394 cd <trex eg v2.28/stl>
395 Update the bench.py to generate the traffic.
397 class STLBench(object):
399 ip_range['src'] = {'start': '<traffic generator port 0 IP eg 202.16.100.20>', 'end': '<traffic generator port 0 IP eg 202.16.100.20>'}
400 ip_range['dst'] = {'start': '<traffic generator port 1 IP eg 172.16.40.20>', 'end': '<traffic generator port 1 IP eg 172.16.40.20>'}
402 Run the TRex server: sudo ./t-rex-64 -i -c 7
403 In another shell run TRex console: trex-console
404 The console can be run from another computer with -s argument, --help for more info.
405 Other options for TRex client are automation or GUI
406 In the console, run "tui" command, and then send the traffic with commands like:
407 start -f stl/bench.py -m 50% --port 0 3 -t size=590,vm=var1
408 For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html
414 Step 3: Bind the datapath ports to DPDK
416 a) Bind ports to DPDK
420 For DPDK versions 17.xx
421 1) cd <samplevnf>/dpdk
422 2) ./usertools/dpdk-devbind.py --status <--- List the network device
423 3) ./usertools/dpdk-devbind.py -b igb_uio <PCI Port 0> <PCI Port 1>
424 .. _More details: http://dpdk.org/doc/guides-17.05/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
427 b) Prepare script to enalble VNF to route the packets
431 cd <samplevnf>/VNFs/vCGNAPT/config
432 Open -> sample_swlb_2port_2WT.tc Replace the bold items based on your setting.
434 link 0 config <VNF port 0 IP eg 202.16.100.10> 8
437 link 1 config <VNF port 0 IP eg 172.16.40.10> 8
440 ; uncomment to enable static NAPT
441 ;p <cgnapt pipeline id> entry addm <prv_ipv4/6> prvport> <pub_ip> <pub_port> <phy_port> <ttl> <no_of_entries> <end_prv_port> <end_pub_port>
442 ;p 5 entry addm 202.16.100.20 1234 152.16.40.10 1 0 500 65535 1234 65535
444 ; routeadd <net/host> <port #> <ipv4 nhip address in decimal> <Mask>
445 routeadd net 0 <traffic generator port 0 IP eg 202.16.100.20> 0xff000000
446 routeadd net 1 <traffic generator port 1 IP eg 172.16.40.20> 0xff000000
448 ; IPv4 static ARP; disable if dynamic arp is enabled.
449 p 1 arpadd 0 <traffic generator port 0 IP eg 202.16.100.20> <traffic generator port 0 MAC>
450 p 1 arpadd 1 <traffic generator port 1 IP eg 172.16.40.20> <traffic generator port 1 MAC>
452 For dynamic cgnapt. Please use UDP_Replay as one of the traffic generator
453 (TG1) (port 0) --> (port 0) VNF (CGNAPT) (Port 1) --> (port0)(UDPReplay)
455 c) Run below cmd to launch the VNF. Please make sure both hugepages and ports to be used are bind to dpdk.
459 cd <samplevnf>/VNFs/vCGNAPT/
460 ./build/vCGNAPT -p 0x3 -f ./config/sample_swlb_2port_2WT.cfg -s ./config/sample_swlb_2port_2WT.tc
462 d) Run UDP_replay to reflect the traffic on public side.
466 cmd: ./build/UDP_Replay -c 0x7 -n 4 -w <pci> -w <pci> -- --no-hw-csum -p <portmask> --config='(port, queue, cpucore)'
467 e.g ./build/UDP_Replay -c 0x7 -n 4 -w 0000:07:00.0 -w 0000:07:00.1 -- --no-hw-csum -p 0x3 --config='(0, 0, 1)(1, 0, 2)'
469 step 4: Run Test using traffic geneator
471 On traffic generator system:
474 cd <trex eg v2.28/stl>
475 Update the bench.py to generate the traffic.
477 class STLBench(object):
479 ip_range['src'] = {'start': '<traffic generator port 0 IP eg 202.16.100.20>', 'end': '<traffic generator port 0 IP eg 202.16.100.20>'}
480 ip_range['dst'] = {'start': '<traffic generator port 1 IP eg 172.16.40.20>', 'end': '<public ip e.g 152.16.40.10>'}
482 Run the TRex server: sudo ./t-rex-64 -i -c 7
483 In another shell run TRex console: trex-console
484 The console can be run from another computer with -s argument, --help for more info.
485 Other options for TRex client are automation or GUI
486 In the console, run "tui" command, and then send the traffic with commands like:
487 start -f stl/bench.py -m 50% --port 0 3 -t size=590,vm=var1
488 For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html
491 UDP_Replay - How to run
492 -----------------------
494 Step 3: Bind the datapath ports to DPDK
496 a) Bind ports to DPDK
500 For DPDK versions 17.xx
501 1) cd <samplevnf>/dpdk
502 2) ./usertools/dpdk-devbind.py --status <--- List the network device
503 3) ./usertools/dpdk-devbind.py -b igb_uio <PCI Port 0> <PCI Port 1>
504 .. _More details: http://dpdk.org/doc/guides-17.05/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
506 b) Run below cmd to launch the VNF. Please make sure both hugepages and ports to be used are bind to dpdk.
510 cd <samplevnf>/VNFs/UDP_Replay/
511 cmd: ./build/UDP_Replay -c 0x7 -n 4 -w <pci> -w <pci> -- --no-hw-csum -p <portmask> --config='(port, queue, cpucore)'
512 e.g ./build/UDP_Replay -c 0x7 -n 4 -w 0000:07:00.0 -w 0000:07:00.1 -- --no-hw-csum -p 0x3 --config='(0, 0, 1)(1, 0, 2)'
515 step 4: Run Test using traffic geneator
519 On traffic generator system:
520 cd <trex eg v2.28/stl>
521 Update the bench.py to generate the traffic.
523 class STLBench(object):
525 ip_range['src'] = {'start': '<traffic generator port 0 IP eg 202.16.100.20>', 'end': '<traffic generator port 0 IP eg 202.16.100.20>'}
526 ip_range['dst'] = {'start': '<traffic generator port 1 IP eg 172.16.40.20>', 'end': '<public ip e.g 152.16.40.10>'}
528 Run the TRex server: sudo ./t-rex-64 -i -c 7
529 In another shell run TRex console: trex-console
530 The console can be run from another computer with -s argument, --help for more info.
531 Other options for TRex client are automation or GUI
532 In the console, run "tui" command, and then send the traffic with commands like:
533 start -f stl/bench.py -m 50% --port 0 3 -t size=590,vm=var1
534 For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html
542 This is PROX, the Packet pROcessing eXecution engine, part of Intel(R)
543 Data Plane Performance Demonstrators, and formerly known as DPPD-BNG.
544 PROX is a DPDK-based application implementing Telco use-cases such as
545 a simplified BRAS/BNG, light-weight AFTR... It also allows configuring
546 finer grained network functions like QoS, Routing, load-balancing...
548 PROX COMMANDS AND SCREENS
549 -------------------------
551 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
552 | *RUNTIME COMMAND* | *DESCRIPTION* | *EXAMPLE* |
553 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
554 | quit | Stop all cores and quit | |
555 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
556 | help <substr> | Show list of commands that have <substr> as a substring. | |
557 | | If no substring is provided, all commands are shown. | |
558 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
559 | verbose <level> | Set the verbosity level of some printed messages. | |
560 | | Possible values are: 0 (default value, error messages only), | verbose 1 |
561 | | 1 (+ warnings), 2 (+ info) and 3 (+ debugging) | |
562 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
563 | thread info <core_id> <task_id> | Show task specific information | |
564 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
565 | update interval <value> | Update statistics refresh rate, in msec (must be >=10). | |
566 | | Default is 1 second | update interval 500 |
567 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
568 | rx tx info | Print connections between tasks on all cores | |
569 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
570 | start <core list>|all <task_id> | Start cores specified in <core list> or all cores. | start all |
571 | | If <task_id> is not specified, all tasks for the specified cores | start 1 |
572 | | will be started. | start 1s0-4s0 |
573 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
574 | stop <core list>|all <task_id> | Stop cores specified in <core list> or all cores. | |
575 | | If <task_id> is not specified, all tasks for the specified | stop 1 |
576 | | cores will be stopped. | |
577 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
578 | dump <coreid> <taskid> <nbpkts> | Create a hex dump of <nb_packets> from <task_id> on <core_id> | dump 2 1 5 |
579 | | showing how packets have changed between RX and TX. | |
580 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
581 | dump_rx <coreid> <taskid> <nbpkts> | Create a hex dump of <nb_packets> from <task_id> on <coreid> at RX | dump_rx 2 1 5 |
582 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
583 | dump_tx <coreid> <taskid> <nbpkts> | Create a hex dump of <nb_packets> from <task_id> on <coreid> at TX | dump_tx 2 1 5 |
584 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
585 | rx distr start | Start gathering statistical distribution of received packets | |
586 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
587 | rx distr stop | Stop gathering statistical distribution of received packets | |
588 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
589 | rx distr reset | Reset gathered statistical distribution of received packets | |
590 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
591 | rx distr show | Display gathered statistical distribution of received packets | |
592 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
593 | rate <port id> <queue id> <rate> | Set transmit rate in Mb/s. This does not include preamble, SFD and IFG | rate 0 0 1000 |
594 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
595 | count <core id> <task id> <count> | Generate <count> packets, then pause generating | count 1 0 5 |
596 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
597 | pkt_size <coreid> <taskid> <pktsize> | Set the packet size to <pkt_size> | pkt_size 1 3 255 |
598 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
599 | speed <core_id> <task_id> <speed percentage> | Change the speed to <speed percentage> of a | |
600 | | 10 Gbps line at which packets are being generated | speed 1 0 50 |
601 | | on core <core_id> in task <task_id> | |
602 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
603 | speed_byte <core_id> <task_id> <speed> | Change speed to <speed>. The speed is specified in units of bytes per sec | |
604 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
605 | set value <core_id> <task_id> <offset> | Set <value_len> bytes to <value> at offset <offset> in packets | |
606 | <value> <value_len> | generated on <core_id> <task_id> | set value 4 1 14 10 1 |
607 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
608 | reset values all | Undo all `set value` commands on all cores/tasks | |
609 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
610 | reset values <core id> <task id> | Undo all `set value` commands on specified core/task | |
611 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
612 | arp add <core id> <task id> <port id> | | |
613 | <gre id> <svlan> <cvlan> <ip addr> | | |
614 | <mac addr> <user> | Add a single ARP entry into a CPE table on <core id>/<task id> | |
615 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
616 | rule add <core id> <task id> svlan_id&mask | | |
617 | cvlan_id&mask ip_proto&mask | | |
618 | source_ip/prefix destination_ip/prefix | | |
619 | range dport_range action | Add a rule to the ACL table on <core id>/<task id> | |
620 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
621 | route add <core id> <task id> | | |
622 | <ip/prefix> <next hop id> | Add a route to the routing table on core <core id> <task id> | route add 10.0.16.0/24 9 |
623 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
624 | reset stats | Reset all statistics | |
625 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
626 | tot stats | Print total RX and TX packets | |
627 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
628 | tot ierrors per sec | Print total number of ierrors per second | |
629 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
630 | pps stats | Print RX and TX packet rate in unit of packet per second | |
631 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
632 | lat stats <core id> <task id> | Print min,max,avg latency as measured during last sampling interval | lat stats 1 0 |
633 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
634 | lat packets <core id> <task id> | Print the latency for each of the last set of packets | |
635 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
636 | core stats <core id> <task id> | Print rx/tx/drop for task <task id> running on core <core id> | |
637 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
638 | port_stats <port id> | Print rate for no_mbufs, ierrors, rx_bytes, tx_bytes, rx_pkts, | |
639 | | tx_pkts and totals for RX, TX, no_mbufs ierrors for port <port id> | |
640 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
641 | ring info all | Get information about ring, such as ring size and | |
642 | | number of elements in the ring | |
643 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
644 | ring info <core id> <task id> | Get information about ring on core <core id> | |
645 | | in task <task id>, such as ring size and number of elements in the ring | ring info 1 0 |
646 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
647 | port info <port id> [brief] | Get port related information, such as MAC address, socket, | |
648 | | number of descriptors..., . Adding `brief` after command | |
649 | | prints short version of output. | port info 1 |
650 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
651 | port up <port id> | Set the port up (all ports are up at startup) | port up 1 |
652 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
653 | port down <port id> | Set the port down | port down 1 |
654 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
655 | port xstats <port id> | Get extra statistics for the port | port xstats 1 |
656 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
657 | version | Show version | |
658 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
659 | port_stats <port id> | Print rate for no_mbufs, ierrors, rx_bytes, tx_bytes, rx_pkts, | |
660 | | tx_pkts and totals for RX, TX, no_mbufs ierrors for port <port id> | |
661 +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+
663 While PROX is running, F1 to F6 change the view on the system. Pressing F1 switches to the main screen showing per core statistics. When PROX is started,
664 this is the screen shown by default. Pressing F2 switches to show port-based information. Pressing F3 shows information (i.e. occupancy, memory usage, ...)
665 about memory pools. If there are tasks with mode=lat, F4 displays latency measurements made during the last second by each of those tasks.
666 F5 displays DPDK ring information. F6 is for L4 generation. If no command has been entered, numbers 1 to 6 can also be used to change the view on the system.
667 This is provided to allow changing screens in environments that do not pass function keys to PROX.
669 Page Up and Page Down can be used to view per core statistics that would otherwise not fit on the screen. Escape quits PROX.
670 The history of previously entered commands can be navigated using the Up and Down arrows. Statistics can be reset with F12.
674 Run PROX with the "--help" argument to display the usage text and the list of supported options as shown below.
675 PROX supports many compilation flags to enable or disable features. For these flags, refer to the Makefile.
676 Refer to the README file for more information on how to run PROX for specific use cases.
680 Usage: ./build/prox [-f CONFIG_FILE] [-l LOG_FILE] [-p] [-o DISPLAY] [-v] [-a|-e] [-m|-s|-i] [-n] [-w DEF] [-q] [-k] [-d] [-z] [-r VAL] [-u] [-t]
681 -f CONFIG_FILE : configuration file to load, ./prox.cfg by default
682 -l LOG_FILE : log file name, ./prox.log by default
683 -p : include PID in log file name if default log file is used
684 -o DISPLAY: Set display to use, can be 'curses' (default), 'cli' or 'none'
685 -v verbosity : initial logging verbosity
686 -a : autostart all cores (by default)
688 -n : Create NULL devices instead of using PCI devices, useful together with -i
689 -m : list supported task modes and exit
690 -s : check configuration file syntax and exit
691 -i : check initialization sequence and exit
692 -u : Listen on UDS /tmp/prox.sock
693 -t : Listen on TCP port 8474
694 -q : Pass argument to Lua interpreter, useful to define variables
695 -w : define variable using syntax varname=value
696 takes precedence over variables defined in CONFIG_FILE
697 -k : Log statistics to file "stats_dump" in current directory
698 -d : Run as daemon, the parent process will block until PROX is not initialized
699 -z : Ignore CPU topology, implies -i
700 -r : Change initial screen refresh rate. If set to a lower than 0.001 seconds,
701 screen refreshing will be disabled
703 CONFIGURATION FILE FORMAT
704 -------------------------
705 The configuration file is divided into multiple sections, each of which is used to define some parameters and options.
706 Sections are created using the [section name] syntax. The list of sections, where # represents an integer, is as follows:
717 In each section, entries are created using the key=value syntax.
718 Comments are created using the ; symbol: all characters from the ;
719 symbol to the end of line are ignored. A # symbol at the beginning of the section name comments
720 the whole section out: all entries in the section are treated as comments and are ignored. For example:
726 parameter name=parameter value ; this entry is ignored because the section is commented out
728 * [EAL OPTIONS]: The following parameters are supported:
732 -m ; Specifies the amount of memory used. If not provided, all hugepages will be used.
733 -n ; Specifies the number of memory channels. Use -n4 for latest Intel Xeon based platforms
734 -r ; Specifies the number of memory ranks.
735 eal ; Specifies DPDK EAL extra options. Those options will be passed blindly to DPDK.
737 * [PORT #]: DPDK ports are usually referenced by their port_id, i.e. an integer starting from 0.
738 Using port_id in the configuration file is tedious, since the same port_id can appear at
739 different places (rx port, tx port, routing tables), and those ports might change (e.g. if cables are swapped).
740 In order to make the configuration file easier to read and modify, DPDK ports are given a name with the name= option.
741 The name serves as the reference, and in addition, it will show up in the display at runtime.
745 PARAMETER EXAMPLE DESCRIPTION
746 ----------------------------------------------------------------------------
747 name inet0 Use inet0 to later refer to this port
748 mac hardware value can be: hardware, random or a literal MAC address
749 rx desc 256 number of descriptors to allocate for reception
750 tx desc 256 number of descriptors to allocate for transmission
751 promiscuous yes enable promiscuous mode
752 strip crc yes enable CRC stripping
754 lsc no While lsc is disabled for drivers known to not provide support,
755 this option explicitely overrides these settings.
756 rx_ring dpdk_ring_name use DPDK ring as an interface (receive side)
757 tx_ring dpdk_ring_name use DPDK ring as an interface (transmit side)
759 * [VARIABLES]: Variables can be defined in the configuration file using the $varname=value syntax.
760 Variables defined on the command line (-w varname=value) take precedence and do not create
761 conflicts with variables defined in the configuration file. Variables are used in the
762 configuration file using the $varname syntax: each instance of $varname is replaced by its
763 associated value. This is typically useful if the same parameter must be used at several places.
764 For instance, you might want to have multiple load balancers, all transmitting to the same set
765 of worker cores. The list of worker cores could then be defined once in a variable:
772 Then, a load balancer definition would use the variable:
783 And the section defining the worker cores would be:
793 * [DEFAULTS]: The default value of some options can be overridden using the [defaults] section:
797 PARAMETER EXAMPLE DESCRIPTION
798 -----------------------------------
799 mempool size 16K number of mbufs per task, relevant when task receives from a port.
800 this is the n argument provided to rte_mempool_create()
801 qinq tag 0xa888 Set qinq tag for all tasks. The result of adding this option is the
802 same as adding qinq tag= to each task
803 memcache size 128 number of mbufs cached per core, default is 256 this is the cache_size
804 argument provided to rte_mempool_create()
806 * [GLOBAL]: The following parameters are supported:
810 PARAMETER EXAMPLE DESCRIPTION
811 -------------------------------------------------
812 name BNG Name of the configuration, which will be shown in the title box at runtime.
813 start time 10 Time in seconds after which average statistics will be started.
815 duration time 30 Runtime duration in seconds, counted after start time.
816 This is typically useful to automate testing using
817 different parameters: PROX automatically exits when the
818 runtime duration has elapsed. Initialization and start time
819 are not included in this runtime duration.
820 For example, if start time is set to 10 and duration time is set to 30,
821 the total execution time (after initialization) will be 40 seconds.
822 Default value is 0, which means infinity and prevents PROX from automatically exiting.
823 shuffle yes When this parameter is set to yes, the order of mbufs
824 within mempools is randomized to simulate a system that has
825 been warmed up. Default value is no.
826 gre cfg /path/to/file.csv Path to CSV file that provides QinQ-to-GRE mapping.
827 Default value is gre_table.csv in same directory as
828 configuration file. Fields are GRE key and QinQ value (computed as SVLAN * 4096 + CVLAN).
829 pre cmd ls Arbitrary system commands to run while reading cfg. This option can occur multiple times.
830 user cfg /path/to/file.csv Path to CSV file that provides QinQ-to-User mapping.
831 Default value is user_table.csv in same directory as configuration file.
832 Fields are SVLAN, CVLAN and User-Id.
833 next hop cfg /path/to/file.csv Path to CSV file that provides Next-Hop details.
834 Default value is next_hop.csv in same directory as configuration file.
835 Fields are Next-Hop index (as returned by LPM lookup),
836 Out-Port index, Next-Hop IP (unused), Next-Hop MAC and MPLS label.
837 ipv4 cfg /path/to/file.csv Path to CSV file that provides IPv4 LPM routing table.
838 Default value is ipv4.csv in same directory as configuration file.
839 Fields are IPv4 subnet (in CIDR notation) and Next-Hop index.
840 dscp cfg /path/to/file.csv Path to CSV file that provides mapping for QoS classification,
841 from DSCP to Traffic Class and Queue.
842 Default value is dscp.csv in same directory as configuration file.
843 Fields are DSCP (0-63), Traffic Class (0-3) and Queue (0-3).
844 ipv6 tunnel cfg /path/to/file.csv Path to CSV file that provides lwAFTR binding table.
845 Default value is ipv6_tun_bind.csv in same directory as configuration file.
846 Fields are lwB4 IPv6 address, next hop MAC address towards lwB4,
847 IPv4 Public address and IPv4 Public Port Set.
848 acl cfg /path/to/file.csv Path to CSV file that provides ACL rules.
849 Default value is rules.csv in same directory as configuration file.
850 Fields are SVLAN value & mask, CVLAN value & mask, IP protocol value & mask,
851 source IPv4 subnet (in CIDR notation), destination IPv4 subnet (in CIDR notation),
852 source port range, destination port range, and action (drop, allow, rate limit).
856 * [CORE #]: Cores can be configured by means of a set of [core #] sections, where # represents either:
858 an absolute core number: e.g. on a 10-core, dual socket system with hyper-threading, cores are numbered from 0 to 39;
859 a core number, the letter 's', and a socket number: this allows selecting per-socket cores, independently from their interleaved numbering;
860 a core number and the letter 'h': this allows selecting the hyper-thread sibling of the specified core;
861 a dash-separated range of core numbers; a comma-separated list of core numbers; any combination of the above;
862 or a variable whose value complies with the above syntax.
863 The socket and hyper-thread syntax makes it easier to use the same configuration file on several platforms,
864 even if their core numbering differs (e.g. interleaving rule or number of cores per socket).
866 Each core can be assigned with a set of tasks, each running one of the implemented packet processing modes.
868 The following parameters are supported:
870 .. image:: images/prox_core.png
872 :alt: SampleVNF supported topology
874 Compiling and running this application
875 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
879 DPDK must be installed prior to running make in the PROX directory.
880 The README file shipped with PROX describes what versions of DPDK are supported,
881 and if any patches are needed for the chosen DPDK version.
883 The following packages need to be installed. (Example for destributions that are using rpm)
887 sudo yum install net-tools wget gcc unzip libpcap-devel ncurses-devel libedit-devel pciutils lua-devel kernel-devel
890 The following instructions are here to help customers to start using PROX.
891 It's by no means a complete guide, for detailed instructions on how to install and use
892 DPDK please refer to its documentation.
893 Your mileage may vary depending on a particular Linux distribution and hardware in use.
895 Edit grub default configuration:
901 Add the following to the kernel boot parameters
905 default_hugepagesz=1G hugepagesz=1G hugepages=8
907 Rebuild grub config and reboot the system:
911 grub2-mkconfig -o /boot/grub2/grub.cfg
914 Verify that hugepages are available
922 Hugepagesize: 1048576 kB
930 umount `awk '/hugetlbfs/ { print $2 }' /proc/mounts` >/dev/null 2>&1
931 mount -t hugetlbfs nodev /mnt/huge/
933 This application supports DPDK 16.04, 16.11, 17.02 and 17.05.
934 The following commands assume that the following variables have been set:
936 export RTE_SDK=/path/to/dpdk
937 export RTE_TARGET=x86_64-native-linuxapp-gcc
939 PROX Compiation installation
940 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
942 * git clone https://git.opnfv.org/samplevnf
944 * export RTE_SDK=`pwd`/dpdk
945 * export RTE_TARGET=x86_64-native-linuxapp-gcc
946 * git clone git://dpdk.org/dpdk
948 * git checkout v17.05
949 * make install T=$RTE_TARGET
950 * cd <samplevnf>/VNFs/DPPD-PROX
957 * git clone https://git.opnfv.org/samplevnf
959 * ./tools/vnf_build.sh -s -d='17.05' [-p=<proxy> if behind the proxy]
965 lsmod | grep -w "^uio" >/dev/null 2>&1 || sudo modprobe uio
972 lsmod | grep -w "^igb_uio" >/dev/null 2>&1 || sudo insmod $RTE_SDK/$RTE_TARGET/kmod/igb_uio.ko
974 Discover network devices available on the system:
978 lspci | grep Ethernet
980 Prior launching PROX, ports that are to be used by it must be bound to the igb_uio driver.
982 The following command will bind all Intel® Ethernet Converged Network Adapter X710 ports to igb_uio:
986 lspci | grep X710 | cut -d' ' -f 1 | sudo xargs -I {} python2.7 $RTE_UNBIND --bind=igb_uio {}
988 The following command will bind all Intel® 82599 10 Gigabit Ethernet Controller ports to igb_uio:
992 lspci | grep 82599 | cut -d' ' -f 1 | sudo xargs -I {} python2.7 $RTE_UNBIND --bind=igb_uio {}
997 The Makefile with this application expects RTE_SDK to point to the
998 root directory of DPDK (e.g. export RTE_SDK=/root/dpdk). If RTE_TARGET
999 has not been set, x86_64-native-linuxapp-gcc will be assumed.
1004 After DPDK has been set up, run make from the directory where you have
1005 extracted this application. A build directory will be created
1006 containing the PROX executable. The usage of the application is shown
1007 below. Note that this application assumes that all required ports have
1008 been bound to the DPDK provided igb_uio driver. Refer to the "Getting
1009 Started Guide - DPDK" document for more details.
1013 Usage: ./build/prox [-f CONFIG_FILE] [-l LOG_FILE] [-p] [-o DISPLAY] [-v] [-a|-e] [-m|-s|-i] [-n] [-w DEF] [-q] [-k] [-d] [-z] [-r VAL] [-u] [-t]
1014 -f CONFIG_FILE : configuration file to load, ./prox.cfg by default
1015 -l LOG_FILE : log file name, ./prox.log by default
1016 -p : include PID in log file name if default log file is used
1017 -o DISPLAY: Set display to use, can be 'curses' (default), 'cli' or 'none'
1018 -v verbosity : initial logging verbosity
1019 -a : autostart all cores (by default)
1020 -e : don't autostart
1021 -n : Create NULL devices instead of using PCI devices, useful together with -i
1022 -m : list supported task modes and exit
1023 -s : check configuration file syntax and exit
1024 -i : check initialization sequence and exit
1025 -u : Listen on UDS /tmp/prox.sock
1026 -t : Listen on TCP port 8474
1027 -q : Pass argument to Lua interpreter, useful to define variables
1028 -w : define variable using syntax varname=value
1029 takes precedence over variables defined in CONFIG_FILE
1030 -k : Log statistics to file "stats_dump" in current directory
1031 -d : Run as daemon, the parent process will block until PROX is not initialized
1032 -z : Ignore CPU topology, implies -i
1033 -r : Change initial screen refresh rate. If set to a lower than 0.001 seconds,
1034 screen refreshing will be disabled
1036 While applications using DPDK typically rely on the core mask and the
1037 number of channels to be specified on the command line, this
1038 application is configured using a .cfg file. The core mask and number
1039 of channels is derived from this config. For example, to run the
1040 application from the source directory execute:
1044 user@target:~$ ./build/prox -f ./config/nop.cfg
1046 Provided example configurations
1047 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1048 PROX can be configured either as the SUT (System Under Test) or as the
1049 Traffic Generator. Some example configuration files are provided, both
1050 in the config directory to run PROX as a SUT, and in the gen directory
1051 to run it as a Traffic Generator.
1052 A quick description of these example configurations is provided below.
1053 Additional details are provided in the example configuration files.
1055 Basic configurations, mostly used as sanity check:
1060 * config/nop-rings.cfg
1063 Simplified BNG (Border Network Gateway) configurations, using different
1064 number of ports, with and without QoS, running on the host or in a VM:
1068 * config/bng-4ports.cfg
1069 * config/bng-8ports.cfg
1070 * config/bng-qos-4ports.cfg
1071 * config/bng-qos-8ports.cfg
1072 * config/bng-1q-4ports.cfg
1073 * config/bng-ovs-usv-4ports.cfg
1074 * config/bng-no-cpu-topology-4ports.cfg
1075 * gen/bng-4ports-gen.cfg
1076 * gen/bng-8ports-gen.cfg
1077 * gen/bng-ovs-usv-4ports-gen.cfg
1079 Light-weight AFTR configurations:
1083 * config/lw_aftr.cfg
1084 * gen/lw_aftr-gen.cfg