* ``--no-traffic`` or ``-0`` : sending traffic from traffic generator is skipped
+TRex force restart
+------------------
+
+NFVbench allows to restart TRex traffic generator between runs.
+It runs the whole test, but restart TRex instance before generating new traffic.
+
+To force restart, use the --restart option:
+
+.. code-block:: bash
+
+ nfvbench --restart
+
+Used parameters:
+
+* ``--restart`` : restart traffic generator (TRex)
+
+Rate Units
+^^^^^^^^^^
+
+Parameter ``--rate`` accepts different types of values:
+
+* packets per second (pps, kpps, mpps), e.g. ``1000pps`` or ``10kpps``
+* load percentage (%), e.g. ``50%``
+* bits per second (bps, kbps, Mbps, Gbps), e.g. ``1Gbps``, ``1000bps``
+* NDR/PDR (ndr, pdr, ndr_pdr), e.g. ``ndr_pdr``
+
+NDR/PDR is the default rate when not specified.
Fixed Rate Run
--------------
.. note:: When parameter ``--inter-node`` is not used or there aren't enough compute nodes, VMs are on the same compute node.
+A fixed rate run can also be used to check the running drop rate while traffic is being generated. In that case the --interval option can be used
+to specify the reporting interval in seconds (minimum is 1 second). This can be useful for example to see how packet drop rate
+evolves over time. One common use case is to see the drop rate when there is a network degradation (e.g. one of the 2 links in a bond
+goes down).
+The console output will show at every reporting interval the number of packets transmitted, received and estimated drop rate for the last reporting interval.
+The smaller is the interval the more precise is the drop rate.
-Rate Units
-^^^^^^^^^^
+Example of output where the reporting interval is set to 1 (second):
-Parameter ``--rate`` accepts different types of values:
+.. code-block:: bash
-* packets per second (pps, kpps, mpps), e.g. ``1000pps`` or ``10kpps``
-* load percentage (%), e.g. ``50%``
-* bits per second (bps, kbps, Mbps, Gbps), e.g. ``1Gbps``, ``1000bps``
-* NDR/PDR (ndr, pdr, ndr_pdr), e.g. ``ndr_pdr``
+ 2020-04-25 12:59:16,618 INFO TX: 1,878,719,266; RX: 1,666,641,890; (Est.) Dropped: 2; Drop rate: 0.0000%
+ 2020-04-25 12:59:17,625 INFO TX: 1,883,740,078; RX: 1,671,662,706; (Est.) Dropped: -4; Drop rate: -0.0001%
+ 2020-04-25 12:59:18,632 INFO TX: 1,888,764,404; RX: 1,676,686,993; (Est.) Dropped: 39; Drop rate: 0.0008%
+ 2020-04-25 12:59:19,639 INFO TX: 1,893,785,063; RX: 1,681,276,714; (Est.) Dropped: 430,938; Drop rate: 8.5833%
+ 2020-04-25 12:59:20,645 INFO TX: 1,898,805,769; RX: 1,683,782,636; (Est.) Dropped: 2,514,784; Drop rate: 50.0883%
+ 2020-04-25 12:59:21,652 INFO TX: 1,903,829,191; RX: 1,686,289,860; (Est.) Dropped: 2,516,198; Drop rate: 50.0893%
+ 2020-04-25 12:59:22,658 INFO TX: 1,908,850,478; RX: 1,691,283,008; (Est.) Dropped: 28,139; Drop rate: 0.5604%
+ 2020-04-25 12:59:23,665 INFO TX: 1,913,870,692; RX: 1,696,301,242; (Est.) Dropped: 1,980; Drop rate: 0.0394%
+ 2020-04-25 12:59:24,672 INFO TX: 1,918,889,696; RX: 1,698,806,224; (Est.) Dropped: 2,514,022; Drop rate: 50.0901%
+ 2020-04-25 12:59:25,680 INFO TX: 1,923,915,470; RX: 1,701,314,663; (Est.) Dropped: 2,517,335; Drop rate: 50.0885%
+ 2020-04-25 12:59:26,687 INFO TX: 1,928,944,879; RX: 1,705,886,869; (Est.) Dropped: 457,203; Drop rate: 9.0906%
+ 2020-04-25 12:59:27,696 INFO TX: 1,933,969,377; RX: 1,710,911,346; (Est.) Dropped: 21; Drop rate: 0.0004%
+ 2020-04-25 12:59:28,702 INFO TX: 1,938,998,536; RX: 1,713,843,740; (Est.) Dropped: 2,096,765; Drop rate: 41.6922%
+ 2020-04-25 12:59:29,710 INFO TX: 1,944,019,920; RX: 1,718,226,356; (Est.) Dropped: 638,768; Drop rate: 12.7210%
+ 2020-04-25 12:59:30,718 INFO TX: 1,949,050,206; RX: 1,723,256,639; (Est.) Dropped: 3; Drop rate: 0.0001%
+ 2020-04-25 12:59:31,725 INFO TX: 1,954,075,270; RX: 1,728,281,726; (Est.) Dropped: -23; Drop rate: -0.0005%
+ 2020-04-25 12:59:32,732 INFO TX: 1,959,094,908; RX: 1,733,301,290; (Est.) Dropped: 74; Drop rate: 0.0015%
+ 2020-04-25 12:59:33,739 INFO TX: 1,964,118,902; RX: 1,738,325,357; (Est.) Dropped: -73; Drop rate: -0.0015%
+ 2020-04-25 12:59:34,746 INFO TX: 1,969,143,790; RX: 1,743,350,230; (Est.) Dropped: 15; Drop rate: 0.0003%
+ 2020-04-25 12:59:35,753 INFO TX: 1,974,165,773; RX: 1,748,372,291; (Est.) Dropped: -78; Drop rate: -0.0016%
+ 2020-04-25 12:59:36,759 INFO TX: 1,979,188,496; RX: 1,753,394,957; (Est.) Dropped: 57; Drop rate: 0.0011%
+ 2020-04-25 12:59:37,767 INFO TX: 1,984,208,956; RX: 1,757,183,844; (Est.) Dropped: 1,231,573; Drop rate: 24.5311%
+ 2020-04-25 12:59:38,773 INFO TX: 1,989,233,595; RX: 1,761,729,705; (Est.) Dropped: 478,778; Drop rate: 9.5286%
+ 2020-04-25 12:59:39,780 INFO TX: 1,994,253,350; RX: 1,766,749,467; (Est.) Dropped: -7; Drop rate: -0.0001%
+ 2020-04-25 12:59:40,787 INFO TX: 1,999,276,622; RX: 1,771,772,738; (Est.) Dropped: 1; Drop rate: 0.0000%
+ 2020-04-25 12:59:41,794 INFO TX: 2,004,299,940; RX: 1,776,796,065; (Est.) Dropped: -9; Drop rate: -0.0002%
+ 2020-04-25 12:59:42,800 INFO TX: 2,009,320,453; RX: 1,781,816,583; (Est.) Dropped: -5; Drop rate: -0.0001%
+ 2020-04-25 12:59:43,807 INFO TX: 2,014,340,581; RX: 1,786,503,172; (Est.) Dropped: 333,539; Drop rate: 6.6440%
+ 2020-04-25 12:59:44,814 INFO TX: 2,019,362,996; RX: 1,789,009,857; (Est.) Dropped: 2,515,730; Drop rate: 50.0900%
+ 2020-04-25 12:59:45,821 INFO TX: 2,024,386,346; RX: 1,791,517,070; (Est.) Dropped: 2,516,137; Drop rate: 50.0888%
+
+
+How to read each line:
+
+.. code-block:: bash
+
+ 2020-04-25 10:46:41,276 INFO TX: 4,004,436; RX: 4,004,381; (Est.) Dropped: 55; Drop rate: 0.0014%
+
+At this poing in time, NFvbench has sent 4,004,436 and received 4,004,381 since the start of the run.
+There is deficit of 55 packets on reception which corresponds to 0.0014% of all packets sent during that reporting window interval (last 1 second)
+A negative value means that the RX count is higher than the tx count in that window – this is possible since the RX and TX reads are not atomic.
-NDR/PDR is the default rate when not specified.
NDR and PDR
-----------
Both **single run** or **NDR/PDR** can be run as multichain. Running multichain is a scenario closer to a real life situation than runs with a single chain.
-External Chain
---------------
-
-NFVbench can measure the performance of 1 or more L3 service chains that are setup externally. Instead of being setup by NFVbench,
-the complete environment (VMs and networks) has to be setup prior to running NFVbench.
-
-Each external chain is made of 1 or more VNFs and has exactly 2 end network interfaces (left and right network interfaces) that are connected to 2 neutron networks (left and right networks).
-The internal composition of a multi-VNF service chain can be arbitrary (usually linear) as far as NFVbench is concerned,
-the only requirement is that the service chain can route L3 packets properly between the left and right networks.
-
-To run NFVbench on such external service chains:
-
-- explicitly tell NFVbench to use external service chain by adding ``-sc EXT`` or ``--service-chain EXT`` to NFVbench CLI options
-- specify the number of external chains using the ``-scc`` option (defaults to 1 chain)
-- specify the 2 end point networks of your environment in ``external_networks`` inside the config file.
- - The two networks specified there have to exist in Neutron and will be used as the end point networks by NFVbench ('napa' and 'marin' in the diagram below)
-- specify the router gateway IPs for the external service chains (1.1.0.2 and 2.2.0.2)
-- specify the traffic generator gateway IPs for the external service chains (1.1.0.102 and 2.2.0.102 in diagram below)
-- specify the packet source and destination IPs for the virtual devices that are simulated (10.0.0.0/8 and 20.0.0.0/8)
-
-
-.. image:: images/extchain-config.svg
-
-L3 routing must be enabled in the VNF and configured to:
-
-- reply to ARP requests to its public IP addresses on both left and right networks
-- route packets from each set of remote devices toward the appropriate dest gateway IP in the traffic generator using 2 static routes (as illustrated in the diagram)
-
-Upon start, NFVbench will:
-- first retrieve the properties of the left and right networks using Neutron APIs,
-- extract the underlying network ID (typically VLAN segmentation ID),
-- generate packets with the proper VLAN ID and measure traffic.
-
-Note that in the case of multiple chains, all chains end interfaces must be connected to the same two left and right networks.
-The traffic will be load balanced across the corresponding gateway IP of these external service chains.
-
-
Multiflow
---------
The corresponding ``step`` is used for ranging the IP addresses from the `ip_addrs``, ``tg_gateway_ip_addrs`` and ``gateway_ip_addrs`` base addresses.
0.0.0.1 is the default step for all IP ranges. In ``ip_addrs``, 'random' can be configured which tells NFVBench to generate random src/dst IP pairs in the traffic stream.
+UDP ports can be controlled with the following NFVbench configuration options:
+
+.. code-block:: bash
+
+ udp_src_port: ['1024', '65000']
+ udp_dst_port: 53
+ udp_port_step: 1
+
+``udp_src_port`` and ``udp_dst_port`` are the UDP port value used by the traffic generators.
+These can be written for unique port or range ports for all flow.
+
+The corresponding ``udp_port_step`` is used for ranging the UDP port.
+1 is the default step for all UDP ranges, 'random' can be configured which tells NFVBench to generate random src/dst UDP pairs in the traffic stream.
+
+NB:
+ Use of UDP range will increase possible values of flows (based on ip src/dst and port src/dst tuple).
+ NFVBench will calculate the least common multiple for this tuple to adapt flows generation to ``flow_count`` parameter.
+
Traffic Configuration via CLI
-----------------------------
-While traffic configuration can be modified using the configuration file, it can be inconvenient to have to change the configuration file everytime you need to change a traffic configuration option. Traffic configuration options can be overridden with a few CLI options.
+While traffic configuration can be modified using the configuration file, it can be inconvenient to have to change the configuration file everytime
+you need to change a traffic configuration option. Traffic configuration options can be overridden with a few CLI options.
Here is an example of configuring traffic via CLI:
NFVbench will dicover the MAC addresses to use for generated frames using:
- either OpenStack discovery (find the MAC of an existing VM) in the case of PVP and PVVP service chains
- or using dynamic ARP discovery (find MAC from IP) in the case of external chains.
+- In case of L3 chain with SDN-GW or router between traffic generator and loop VM ARP is needed to discover SDN-GW mac addresses, use ``--loop-vm-arp`` flag or ``loop_vm_arp: true`` in config file.
-Cleanup Script
---------------
+Status and Cleanup of NFVbench Resources
+----------------------------------------
+
+The --status option will display the status of NFVbench and list any NFVbench resources. You need to pass the OpenStack RC
+file in order to connect to OpenStack.
+
+.. code-block:: none
+
+ # nfvbench --status -r /tmp/nfvbench/openrc
+ 2018-04-09 17:05:48,682 INFO Version: 1.3.2.dev1
+ 2018-04-09 17:05:48,683 INFO Status: idle
+ 2018-04-09 17:05:48,757 INFO Discovering instances nfvbench-loop-vm...
+ 2018-04-09 17:05:49,252 INFO Discovering flavor nfvbench.medium...
+ 2018-04-09 17:05:49,281 INFO Discovering networks...
+ 2018-04-09 17:05:49,365 INFO No matching NFVbench resources found
+ #
+
+The Status can be either "idle" or "busy (run pending)".
-The nfvbench_cleanup script will cleanup resources created by NFVbench. You need to pass the OpenStack RC file in order to connect to
-OpenStack.
+The --cleanup option will first discover resources created by NFVbench and prompt if you want to proceed with cleaning them up.
Example of run:
.. code-block:: none
- # nfvbench_cleanup -r /tmp/nfvbench/openrc
- Discovering Storage resources...
- Discovering Compute resources...
- Discovering Network resources...
- Discovering Keystone resources...
-
- SELECTED RESOURCES:
- +-----------+-------------------+--------------------------------------+
- | Type | Name | UUID |
- |-----------+-------------------+--------------------------------------|
- | flavors | nfvbench.medium | 362b2215-89d1-4f46-8b89-8e58165ff5bc |
- | instances | nfvbench-loop-vm0 | f78dfb74-1b8e-4c5c-8d83-652a7571da95 |
- | networks | nfvbench-net0 | 57d7e6c9-325f-4c13-9b1b-929344cc9c39 |
- | networks | nfvbench-net1 | 2d429bcd-33fa-4aa4-9f2e-299a735177c9 |
- +-----------+-------------------+--------------------------------------+
-
- Warning: You didn't specify a resource list file as the input. The script will delete all resources shown above.
+ # nfvbench --cleanup -r /tmp/nfvbench/openrc
+ 2018-04-09 16:58:00,204 INFO Version: 1.3.2.dev1
+ 2018-04-09 16:58:00,205 INFO Status: idle
+ 2018-04-09 16:58:00,279 INFO Discovering instances nfvbench-loop-vm...
+ 2018-04-09 16:58:00,829 INFO Discovering flavor nfvbench.medium...
+ 2018-04-09 16:58:00,876 INFO Discovering networks...
+ 2018-04-09 16:58:00,960 INFO Discovering ports...
+ 2018-04-09 16:58:01,012 INFO Discovered 6 NFVbench resources:
+ +----------+-------------------+--------------------------------------+
+ | Type | Name | UUID |
+ |----------+-------------------+--------------------------------------|
+ | Instance | nfvbench-loop-vm0 | b039b858-777e-467e-99fb-362f856f4a94 |
+ | Flavor | nfvbench.medium | a027003c-ad86-4f24-b676-2b05bb06adc0 |
+ | Network | nfvbench-net0 | bca8d183-538e-4965-880e-fd92d48bfe0d |
+ | Network | nfvbench-net1 | c582a201-8279-4309-8084-7edd6511092c |
+ | Port | | 67740862-80ac-4371-b04e-58a0b0f05085 |
+ | Port | | b5db95b9-e419-4725-951a-9a8f7841e66a |
+ +----------+-------------------+--------------------------------------+
+ 2018-04-09 16:58:01,013 INFO NFVbench will delete all resources shown...
Are you sure? (y/n) y
- *** STORAGE cleanup
- *** COMPUTE cleanup
- . Waiting for 1 instances to be fully deleted...
- . INSTANCE 1 left to be deleted, retries left=5...
- . INSTANCE 1 left to be deleted, retries left=4...
- + INSTANCE nfvbench-loop-vm0 is successfully deleted
- + FLAVOR nfvbench.medium is successfully deleted
- *** NETWORK cleanup
- + Network port 075d91f3-fa6a-428c-bd3f-ebd40cd935e1 is successfully deleted
- + Network port 3a7ccd8c-53a6-43d0-a823-4b5ca762d06e is successfully deleted
- + NETWORK nfvbench-net0 is successfully deleted
- + Network port 5b5a75bd-e0b5-4f81-91b9-9e216d194f48 is successfully deleted
- + Network port cc2d8f1b-49fe-491e-9e44-6990fc57e891 is successfully deleted
- + NETWORK nfvbench-net1 is successfully deleted
- *** KEYSTONE cleanup
+ 2018-04-09 16:58:01,865 INFO Deleting instance nfvbench-loop-vm0...
+ 2018-04-09 16:58:02,058 INFO Waiting for 1 instances to be fully deleted...
+ 2018-04-09 16:58:02,182 INFO 1 yet to be deleted by Nova, retries left=6...
+ 2018-04-09 16:58:04,506 INFO 1 yet to be deleted by Nova, retries left=5...
+ 2018-04-09 16:58:06,636 INFO 1 yet to be deleted by Nova, retries left=4...
+ 2018-04-09 16:58:08,701 INFO Deleting flavor nfvbench.medium...
+ 2018-04-09 16:58:08,729 INFO Deleting port 67740862-80ac-4371-b04e-58a0b0f05085...
+ 2018-04-09 16:58:09,102 INFO Deleting port b5db95b9-e419-4725-951a-9a8f7841e66a...
+ 2018-04-09 16:58:09,620 INFO Deleting network nfvbench-net0...
+ 2018-04-09 16:58:10,357 INFO Deleting network nfvbench-net1...
#
+
+The --force-cleanup option will do the same but without prompting for confirmation.
+
+Service mode for TRex
+---------------------
+
+The ``--service-mode`` option allows you to capture traffic on a TRex window during the NFVBench test. Thus, you will be
+able to capture packets generated by TRex to observe many information on it.
+
+Example of use :
+
+.. code-block:: bash
+
+ nfvbench ``--service-mode``
+
+.. note:: It is preferable to define the minimum rate (2002 pps) to have a better capture
+
+In another bash window, you should connect to the TRex console doing :
+
+.. code-block:: bash
+
+ cd /opt/trex/vX.XX/ # use completion here to find your corresponding TRex version
+ ./trex-console --python3 -r
+ # capture on port number 0
+ capture monitor start --rx 0 -v
+
+ # to stop capture
+ capture monitor stop
+
+Start this capture once you have started the NFVBench test, and you will observe packets on the TRex console :
+
+.. code-block:: bash
+
+ #26342 Port: 0 ◀── RX
+
+ trex(read-only)>
+
+ Type: UDP, Size: 66 B, TS: 26.30 [sec]
+
+ trex(read-only)>
+ ###[ Ethernet ]###
+ dst = a0:36:9f:7a:58:8e
+ src = fa:16:3e:57:8f:df
+ type = 0x8100
+ ###[ 802.1Q ]###
+ prio = 0
+ id = 0
+ vlan = 1093
+ type = 0x800
+ ###[ IP ]###
+ version = 4
+ ihl = 5
+ tos = 0x1
+ len = 46
+ id = 65535
+ flags =
+ frag = 0
+ ttl = 63
+ proto = udp
+ chksum = 0x8425
+ src = 120.0.0.0
+ dst = 110.0.17.153
+ \options \
+ ###[ UDP ]###
+ sport = 53
+ dport = 53
+ len = 26
+ chksum = 0xfd83
+ ###[ Raw ]###
+ load = "xx\xab'\x01\x00?s\x00\x00\xbci\xf0_{U~\x00"
+ ###[ Padding ]###
+ load = '6\x85'
+
+Check on the NFVBench window that the following log appears just before the testing phase :
+
+.. code-block:: bash
+
+ 2019-10-21 09:38:51,532 INFO Starting to generate traffic...
+ 2019-10-21 09:38:51,532 INFO Running traffic generator
+ 2019-10-21 09:38:51,541 INFO ``Service mode is enabled``
+ 2019-10-21 09:38:52,552 INFO TX: 2004; RX: 2003; Est. Dropped: 1; Est. Drop rate: 0.0499%
+ 2019-10-21 09:38:53,559 INFO TX: 4013; RX: 4011; Est. Dropped: 2; Est. Drop rate: 0.0498%
\ No newline at end of file
Code Review / nfvbench.git / blobdiff