Xavier Simonart [Fri, 10 Jul 2020 14:27:06 +0000 (16:27 +0200)]
Fixed compilation issue on dpdk 19.08 (and more recent)
Change-Id: Iae0a0d95b5a216463ec7511337be4019fdd4359f
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Thu, 2 Jul 2020 08:05:49 +0000 (10:05 +0200)]
Fix minor compilation issue on recent gcc compilers
Change-Id: I7eeaaa3a0dfa1c0a831863e772b50697c5ff5054
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Luc Provoost [Mon, 15 Jun 2020 10:02:46 +0000 (12:02 +0200)]
Adding vdev support and vfio
Using vfio in stead of igb_uio. DPDK 2020.5 is NOT compiling the igb_uio
driver by default any more.
For the l3 mode, we are now using the new PROX vdev feature. This is
achieved by adding the vdev parameter in the port section of the PROX
config files. One of the advantages is that we can now ping a port
managed by PROX: the requests will be forwarded by PROX to the LINUX
stack, who will reply to the ping. A tap device is being created by PROX
to support this feature.
Installing old version of nasm since latest causes some issues.
If not pushgateway is being used, it should be commented out in the
config_file.
vfio is now loaded by specifying the module in /etc/modules-load.d
There is also an after_boot.sh script that will be run by
check_prox_system_setup.sh, which can be used for further configuration
of the instance after boot.
There is now also a file created /opt/rapid/system_ready_for_rapid by
the check_prox_system_setup.sh script. This file is created once all
configuration is done. The runrapid.py script will wait till this file
is created before executing any tests.
Change-Id: Ic5c41af82642066af42134c3323297f5a06f6f72
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Yury Kylulin [Fri, 5 Jun 2020 16:05:28 +0000 (16:05 +0000)]
Clean up for the test pods
Option for Kubernetes environment clean up (createrapidk8s.py -c) from the test pods added.
Signed-off-by: Yury Kylulin <yury.kylulin@intel.com>
Change-Id: I1c6c7090462b55fbadd1d8f1a277470a6eeb2dee
Yury Kylulin [Wed, 3 Jun 2020 14:04:25 +0000 (14:04 +0000)]
Fix for containers to use /opt/rapid
Fix scripts and build procedure to use /opt/rapid as a main directory for
binaries and config files.
Signed-off-by: Yury Kylulin <yury.kylulin@intel.com>
Change-Id: Icb38c37a85a78a6f2de2e751df5fafd624b4f1b3
Luc Provoost [Tue, 2 Jun 2020 14:02:17 +0000 (16:02 +0200)]
Using /opt/rapid to run prox
All config files are now copied into /opt/rapid. We are not using the
home directory any more. The prox executable is now also copied into the
same /opt/rapid.
Change-Id: I838636167f370bf8a855a3cfb0796c659e27a3ad
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Fri, 29 May 2020 11:47:03 +0000 (13:47 +0200)]
Fixing some more copyright dates
New files created in 2020 have now the right date in the copyright
notice.
Change-Id: I8eb244beec3d793e819cce1f63c39671f18286a1
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Xavier Simonart [Fri, 29 May 2020 22:17:29 +0000 (00:17 +0200)]
Updated PROX version to 20.05 to reflect support of DPDK 20.05
Change-Id: Iad6882cc28ac2039984b12f6b7a897b81d975d9b
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Fri, 29 May 2020 21:15:04 +0000 (23:15 +0200)]
Fix compilation issues on old DPDK
Also fixed minor issues such as
- prevent using vdev on DPDK <= 17.05 as not supported
- potential segmentation fault after panic (if panic thrown early
in prox initialization)
Change-Id: I49a1962e95a4a44dddadd1fdd906e1b3b232465c
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Thu, 28 May 2020 22:06:45 +0000 (00:06 +0200)]
VLAN support with vdev devices + few other changes
- vdev devices now support VLAN.
- kernel tap device can be configured with a netmask (/24 was
always used in previous version).
- when sending a (fake) packet to the kernel, this packet will now
not be routed by the kernel (i.e. it will leave through the
interface configured by PROX). This might change in the futture
when PROX supports multiple VLANs per port. But today it prevents
ARP being sent on management interfaces.
- Log error in case kernel unable to send packet.
- Added support for comments (';') in lua sections.
- Prevent duplication of local_ipv4 - should now be configured in port section
local_ipv4 in core section still supported
Change-Id: I8f9a40fe6ad6f3013ff91b58b44627c7f34081e6
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Wed, 27 May 2020 13:17:52 +0000 (15:17 +0200)]
Add support for latency measurement for IMIX packets
Before this patch latency measurements were only
supported if signature was disabled.
Change-Id: I565b00f76818ab42e9cca4ddc32f9d857648929d
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Tue, 26 May 2020 09:56:41 +0000 (11:56 +0200)]
Fix Idle count
Change-Id: Idd72b593504139ac9d42c33767f8a18395b2cc87
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Tue, 26 May 2020 09:46:41 +0000 (11:46 +0200)]
Fix source mac address when applying imix
Change-Id: I8cbbe1b08af9b5472c304f2fba3635f0a084fdd8
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Tue, 26 May 2020 09:20:22 +0000 (11:20 +0200)]
Fix potential core dump when closing ports at exit
Change-Id: I45ea9741be1c552d8f13a7de027994c23995df69
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Mon, 11 May 2020 19:00:33 +0000 (21:00 +0200)]
Added initial support for NDP (IPv6)
Following messages are now handled by PROX
- router_solicitation
- neighbour_solicitation
- router_advertisement
- neighbour_advertisement
The following parameters are supported (through the PROX config file)
- sub mode=ndp
This will enable handling of router and neighbour solicitation
and advertisement.
- local ipv6=xxxx:xxxx:xxxxx:xxxx:xxxx:xxxx:xxxx:xxxx
This will configure the local IPv6 address of the port.
This parameter is optional. If not specified, the local IPv6
will be calculated from the EUI.
- global ipv6=xxxx:xxxx:xxxxx:xxxx:xxxx:xxxx:xxxx:xxxx
This will configure the global IPv6 address of the port.
This parameter is optional. If not specified, the global IPv6
will be calculated from the EUI and the router prefix received
from the router.
- ipv6 router=yes
This will cause the core to behave as an IPv6 router
i.e. it will generate Router Advertisement messages
This is only useful in back to back cases, when no real
IPv6 router is present in the setup.
- router prefix=xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx
The router prefix usedin the router advertisement
The prefix will be used by the node to build an IPv6 global
address in cases none were configured.
"Unsollicited NA" parameter has been added within the core/task section.
If set to yes (Unsollicited NA=yes), then an unsollicited neighbour
Advertisement is sent at startup
A same core/task cannot support both l3 and ndp mode.
Those messages will be generated or handled when submode
is set to "ndp":
- neighbour sollicitation
- neighbour advertisement
- router sollicitation
- router advertisement
An example configuration is provided: config/ipv6.cfg in which
port 0 / core 1 plays the role of the generator and port 1 /
core 2 plays the role of the swap.
Change-Id: Id0ab32d384448b4cf767fb4a1c486fc023f4f395
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Fri, 3 Jan 2020 02:09:19 +0000 (03:09 +0100)]
Added support for IMIX through config and command line
In Prox configuration IMIX can be specified using the following syntax:
imix=pkt_size,pkt_size... Up to 127 different pkt sizes can be specified.
Through command line, the following syntax can be used
imix core_id task_id pkt_size,pkt_size... Up to 127 packet sizes can be specified.
When IMIX is enabled, PROX will loop through the list of packet sizes.
PROX supports two different submodes to handle pcap files.
In the 1st submode (default, no submode specified), timestamps from
the pcap file are ignored. In this submode IMIX is supported as well and each
packets in the pcap file will be generated with all IMIX sizes.
In the second submode, called pcap in gen mode, packets are generated at
the timestamp specified in the pcap file. This submode does not support IMIX.
Change-Id: I53cbf1378a5364254285b81e6848350d98561184
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Mon, 23 Mar 2020 23:31:13 +0000 (00:31 +0100)]
Added support for secondary process & virtual devices
A virtual device can be created adding the following option on the eal field within
the eal options section:
--vdev=net_ring0
This will cause a new (virtual) dpdk port to be created.
Such a device can then be used in a primary process as other DPDK devices i.e.
a [port x] section must be defined in the configuration file
For a secondary process, there is no need to add --vdev, but, of course
--proc-type=secondary (or --proc-type=auto) must be set within the eal field.
There are two ways to use such a virtual device in a secondary process
1) Using the DPDK port number. Note however that the DPDK port number of virtual devices
in the secondary process might not start from 0 and might hence have to be guessed
e.g. if using 2 virtual devices in primary process (--vdev=net_ring0 --vdev=net_ring1)
the port_id will be 0 and 1 in primary process and 2 & 3 in secondary process.
2) Using the port name, as defined in --vdev in the primary process. In this way,
no [port] section must be defined, and the virtual port names are directly used within
the "rx port" and "tx port" configuration e.g.:
rx port=net_ring0
tx port=net_ring1
Limitations
===========
There seems to be a PROX leak causing DPDK port id to increase when restarting the
secondary process (causing the secondary process configuration through port id more difficult).
As the primary process configures the port (including the number of rx and tx queues)
based on its config file, the secondary process can't setup its own queues.
Simple configurations (such as using gen or lat in primary, and swap in secondary) work as PROX
allocates a RX queue even in gen mode and a TX queue even in lat mode. Better configuration/support
for secondary process should be designed.
The use of dpdk port names is only supported by PROX within "rx port" and "tx port". It is not
supported by other configuration fields such as "tx ports from routing table".
The use of dpdk port names in "rx port" and "tx port" is limited to only 1 port.
Change-Id: Iaa606625da471403713a21df79d3ded4bb91b91e
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Mon, 11 May 2020 09:26:49 +0000 (11:26 +0200)]
Fix mbuf leak and stop sending pseudo packet
In addition to a mmbuf leak when receiving netlink routes packets
this commit prevents sending the PROX pseudo packet i.e.
the packet PROX sends to the kernel socket to have it generate
an ARP request.
Change-Id: Iabbdecbe412e4b90ac0df7e30fa36d096c5326f0
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Sun, 10 May 2020 22:20:51 +0000 (00:20 +0200)]
Added initial VLAN support with vdev devices
For kernel supported devices, add for vlan tag support
This can be configured through port parameter:
vlan tag=<vlan tag>
If this parameter is set, a vlan tagged interface is created
on top of the tap device
This is only supported for vdev tap devices
When sending (untagged) packet to the tap device (through socket)
the tap should react in sending tagged packet
Note that receiving in L3 mode (w/o tap support) a tagged packet
is not yet supported.
Change-Id: I363fa2f8d2341ac41ef23620222ece1d944bf336
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Sun, 10 May 2020 19:04:08 +0000 (21:04 +0200)]
Improve performance in l3 submode
Two cases where performance has been improved
- When using a gateway from a routing table (l3 submode),
store the mac within the next hop table, to avoid a hash_lookup.
This gives ~10% improvement.
- Read tsc only once per bulk (of up to 64 packets).
This gives ~10% improvement to swap, 4% to gen.
In addition a small fix has been added, preventig "No route" Error
to be written too aften.
Change-Id: I8a7ab74a32f09c8ff47f751ee91e84afee1b2147
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Sat, 2 May 2020 19:51:24 +0000 (21:51 +0200)]
Added initial support for BGP
Through this commit BGP messages are forwarded to tap device
Netlink messages are enabled to receive route Updates.
In addition, generating tasks can also specify a routing table
which will be used when sending packets
The routes initialized by the routing table can be changed through
the reception of BGP messages
Change-Id: I187ba9a921885cbc9b209aae5fb654309e3388b8
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Tue, 28 Apr 2020 23:51:30 +0000 (01:51 +0200)]
Fix ctrlplane handling vdev
Change-Id: I8f570940d2c4e1fc09ebae03105e436f5caebbd3
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Tue, 28 Apr 2020 23:22:10 +0000 (01:22 +0200)]
Improve ctrlplane performance
The crlplane performance has been improved. This is necessary when
handling many IP in L3 mode (i.e. many ARP Requests/Replies).
Before this change, arp timer was updated as soon as an
arp request was tentativaly sent to the master. This means that,
if the request failed to be sent (e.g. ring full) we had to wait
arp_update_time (default 1sec) before trying again.
Now arp_update_time is only used when an arp reply is received.
If a request has been successfully sent, then the arp timer
is updated by 1 second (i.e. we will not send any arp reuest for this
IP within this second)
If we failed to send the request, then the timer is updated by 100 msec
which means we will wait 100 msec before trying again.
A too high value here would have meant that we have to wait a long time
before trying again. A too short value overload the master ring with
request to transmit ARP requests, so that master is unable to handle
arp replies.
arp_update_time is now also partly randomized to avoid all IPs to request
arp request at the beginning of the test. So if arp_update_time is 10 sec
then the configuration sets it to 5 to 15 sec.
As a final note the following parameter (already implemented before)
can be changed to improve ctrlplane performance (at the cost of dataplane)
ctrl path polling frequency=10000
Change-Id: I9660458a22e7442eaa0c83aaa0c9893a15069f98
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Fri, 24 Apr 2020 19:52:12 +0000 (21:52 +0200)]
Added support for netlink
Through this commit ARP and ICMP messages are forwarded to the kernel
when vdev tap devices are enabled, as well as PROX l3 mode.
ICMP support has also been added to master (i.e. PROX L3 mode) and to
swap (so when L3 submode is not enabled).
Change-Id: Ie6bf52cbae7171bfca041ff18651d4ec866f44cd
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Mon, 20 Apr 2020 21:25:22 +0000 (23:25 +0200)]
Add support for vdev tap devices
This commit adds support for the creation of tap devices.
To create a tap device, and associate it with a DPDK port,
add within the port section:
vdev=<tap_name>
local ipv4=<ip_assigned_to_port>
<tap_name> will appear as a kernel network device, with an IP <ip_assigned_to_port>
ARP packets received from the network on the DPDK interface will be forwarded to the kernel.
1st packet which needs to be generated to a new IP will cause an IP packet
to be sent to the kernel (using socket sendto) forcing the kernel to send an ARP_REQUEST.
In this commit, there is no notification from the kernel about ARP table changes.
This is subject to further commits.
Two config files, gen_tap.cfg and swap_tap.cfg have been created. They work one against each other.
*** Note however that they do not work if running within the same host
(with the port being connected back to back) ***
This is due to the fact that the kernel would bypass the ports and use local interface...
Change-Id: Iadeec0d99e3c693472ea44bdb9163a3bf97df2fa
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Luc Provoost [Fri, 29 May 2020 08:22:45 +0000 (10:22 +0200)]
Fix for container .env file and small updates
Since runrapid.py is now supporting more than 1 dataplane interface, we
need to use a port index in the rapid.env file for the dataplane ports.
In k8sdeployment.py, we only support 1 dataplane port, but we are
writing dp_ip1 and dp_mac1 in the environment file. The copyright dates
are also updated. Some references to VM have also been changed to instance.
Change-Id: Ie1d036c13ef5d6f0aa1f35aa068a9760a1cfc2a9
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Thu, 28 May 2020 10:57:46 +0000 (12:57 +0200)]
IPV6 testing & minor changes and fixes
IPV6 test has been added. The generator will now generate IPV6 packets
using NDP to resolve MAC addresses. The hex IP addresses in the inline
definition of the packets in the cfg files is now calculated by a lua
function called in parameters.lua. The hard-coded path /home/centos has
been replaced by ~. An error in the speed calculations has been fixed.
See the RapidTest class for details. The actual number of generated
flows is now reported. If the requested number of flows is not a power
of 2, the lowest power of 2 that is larger than the requested number of
flows will be used and reported.
Change-Id: I36b8d345b4a03c2cf358c19ec353ca8ad2831293
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Fri, 15 May 2020 09:51:56 +0000 (11:51 +0200)]
Support for imix packet sizes
All packet size in the *.test files have been replaces with imix sizes.
imix is now a list of packet sizes. If you want to run with only a
pakcet size of 64, you need to specify a list containing only 1 size:
[64].
You can also specify multiple imix values by specifying a list of list
of sizes. The keyworkds in the test files are now imix & imixs.
Change-Id: Iea47c0266b022133b7bbfc9a6811e3199ae41521
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Mon, 11 May 2020 09:06:17 +0000 (05:06 -0400)]
createrapid.py is now using heat templates yaml
yaml files have been added to the repo as an example. Please check the
README explaining the output section reqs for this yaml file. There is
also a new file (config_file): it also specifies which yaml files to
use. multiple dataplane interfaces per VM can now be specified and will
appear in the <STACK>.env file. An error in setting the packet size has
been fixed (see set_udp_packet_size for packet size setting details)
Change-Id: Ie89a4940521dac7dd3652acca477739abb9f5497
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Mon, 27 Apr 2020 16:32:43 +0000 (12:32 -0400)]
Code rewrite and Python3 support
The code has been rewritten in multiple classes and files for better
readability and maintainability. You should now also use Python3 to run
the scripts. Some of the variables in the config files have moved to
other sections. The pushgateway parameter moved to the [varia] section
of the rapid environment file.
Change-Id: Ic61a1bc3baa39447d42b21dfcc0f16c28fcdfa6e
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Xavier Simonart [Tue, 28 Apr 2020 23:25:46 +0000 (01:25 +0200)]
Fix configuration of arp_timeout and arp_update_time
Change-Id: I9e90c9bdd4d6e83ce0533fd38e1980dca1841bdb
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Thu, 16 Apr 2020 14:09:14 +0000 (16:09 +0200)]
Fix dump packets command
dump packets (e.g. dump 1 0 1) was in some cases dumping more
packets than expected.
Proper count of packets is now dumped.
Change-Id: I0974465d617eda00e77ee87148bf96991ecd3ed7
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Fri, 31 Jan 2020 17:20:15 +0000 (18:20 +0100)]
Added support for DPDK 19.11
Change-Id: I58fa7a0ab7aaa27e9f9383f7995bb71bf0952618
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Luc Provoost [Mon, 23 Mar 2020 10:53:39 +0000 (06:53 -0400)]
Improved variable init and bug fixes
Variables were initiales based on the file name of the PROX
configuration files. These decisions are now based on the content of
these files and other variables, rather than just the name of a file.
This commit also fixes some bugs: irq test was not executing any more,
some issues with the push gateway reporting.
Change-Id: I39308effad3c80cf52d96e4d85d92d4b5f0be9a5
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Yury Kylulin [Tue, 3 Dec 2019 15:29:00 +0000 (18:29 +0300)]
Add support for kubernetes deployment in rapid scripts.
In case of kubernetes based deployments procedure for test environment
creation is different. Please refer to README.k8s file for more
details.
Dockerfile and dockerimage.sh files can be used to create a docker
image with prox.
There is separate createrapidk8s.py script which is used to create
appropriate number of PODs for testing based on the rapid.pods
configuration.
port_info is a helper application to identify port MAC address.
Change-Id: I73611f066ac54260aa11dd17c173379df06f46a5
Signed-off-by: Yury Kylulin <yury.kylulin@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
Xavier Simonart [Fri, 31 Jan 2020 17:13:30 +0000 (17:13 +0000)]
Merge changes from topic "TST009_benchmark"
* changes:
New PROX commit ID and bug fix
code review comments and PROX commit ID
Code review feedback & back-ground traffic support
Fix run_iteration when not using the sut_sock
Specify latency bucket size & correct calculation
Initial support for percentile latency measurements
Combine code for fixed rate and variable rate tests
Better Latency accuray warning message
Support for ETSI GS NFV-TST 009
Luc Provoost [Fri, 31 Jan 2020 17:06:02 +0000 (12:06 -0500)]
New PROX commit ID and bug fix
Updated deploycentos.sh with latest PROX commit ID and took another code
review comment into account.
Change-Id: Iedfa695be7496459d990f36e2dc00c8890b19fb3
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Xavier Simonart [Fri, 31 Jan 2020 16:54:54 +0000 (16:54 +0000)]
Merge "Update PROX version to (preprod) 1908 to reflect DPDK 19.08 support"
Luc Provoost [Wed, 29 Jan 2020 13:22:32 +0000 (08:22 -0500)]
code review comments and PROX commit ID
Implemented the code review comments I got on previous commits and using
commit ID
2cd03b7 to select the PROX version that will be installed in
the VMs through deploycentostools.sh
Change-Id: Ifc4c18c657a8bd57c68a7fc9d885c75ffdcb7b5d
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Xavier Simonart [Wed, 29 Jan 2020 12:11:00 +0000 (13:11 +0100)]
Update PROX version to (preprod) 1908 to reflect DPDK 19.08 support
Change-Id: I04e0bdc6679a74a8ace26eb30d6f8d2857d6c1c7
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Sun, 12 Jan 2020 15:52:10 +0000 (16:52 +0100)]
PROX: print panic message on screen
When PROX detects an error condition at startup, it calls PROX_PANIC
which prints an message before calling rte_panic to end the application.
If ncurses was already started, this results in the message being printed
in ncurses (and prox.log) then leaving ncurses; hence the message is usually
not visible to the user (until it open prox.log), giving the impression of an
unexpected crash.
With this fix, the error message is repeated after closing ncurses, hence the message
will be printed on stdout.
Note that it might also be printed twice on stdout if ncurses was not already started.
As part of this fix, prox.log is now properly closed.
Change-Id: If41875843f1a39bc715f4264b3992c3fa018394e
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Mon, 30 Dec 2019 22:49:52 +0000 (23:49 +0100)]
Added command lat all stats
lat all stats <core_id> <task_id> combines "lat stats" and "lat packets" within one command.
This command returns:
- One line in case the core_task parser fails. In that case the line will start with: "error: invalid syntax"
- One set of lines for each core task:
- if core/task is invalid, or not running lat mode, the line will contain (start with) "error...". this will be the only line for that core/task.
- Otherwise the set of lines will be:
- One line as returned by "lat stats". This line will be
- One line starting with "error" in case of error, or
- One line containing lat_min_usec, lat_max_usec, lat_avg_usec, tot_lat_min_usec, tot_lat_max_usec, last_tsc, rte_get_tsc_hz(), lcore_id, task_id
- The return from lat packets i.e.
- One line starting with "error" in case of error, or
- 128 lines - one for each bucket - as returned by "lat packets" command
This command is more useful then lat packets, as it contains the tsc - needed to see whether the sample is new or not.
Change-Id: I04231aa7c5bd2d352ed6f94d40e88d3b411ce744
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Sun, 29 Dec 2019 17:41:50 +0000 (18:41 +0100)]
Add Latency Distribution display
- Latency Distribution (i.e. histograms) has been added when latency is enabled.
- Histograms are now compiled by default.
- A bug has also been fixed, which was causing strange behavior in the histograms
(due to bucket size being overwritten).
- The default bucket size has been changed to "11" which means that the size of each bucket
is now (1 cycle << 11) = 2048 cycles = 1 microsecond at 2GHz. It was (1 << (10 - 1)).
As there are 128 latency buckets, it is not possible to show all of them on the display.
Hence the latency buckets are displayed based on on the assumption that the minimum latency is usually relatively
stable and that the maximum latency change more often.
- The first empty buckets are not shown (i.e. buckets empty for all tasks)
- If more than 15 buckets are non empty, then the buckets are combined, so that a maximum of
15 (bigger) buckets are shown
- If less than 15 buckets are non empty, the following (empty) buckets are shown
(this avoid seeing every x seconds some columns being added and removed).
Change-Id: I27fe6ac0e513a5558e42ff2e74255c55ba79516d
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Tue, 6 Aug 2019 11:01:27 +0000 (13:01 +0200)]
Added support for DPDK 19.08
Change-Id: I090bf8f1cd5a5d2dfa4f336bc66c46cc03d6c5bb
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Wed, 23 Oct 2019 10:38:51 +0000 (12:38 +0200)]
Prepare for DPDK 19.08 support
This commit prepares PROX for supporting changes from DPDK 19.08
Mainly, the "sed" listed below were run.
In addition,
- some forward definition have been removed,
- prox_compat.h has been updated
- prox_compat.h has been included where necessary.
Hence such a commit is rather easy to review and to reapply
on other branches if necessary.
sed -i 's/struct ether_hdr/prox_rte_ether_hdr/g' *.h
sed -i 's/struct ether_hdr/prox_rte_ether_hdr/g' *.c
sed -i 's/struct ether_addr/prox_rte_ether_addr/g' *.c
sed -i 's/struct ether_addr/prox_rte_ether_addr/g' *.h
sed -i 's/prox_rte_ether_addr_port/struct ether_addr_port/g' *.c
sed -i 's/prox_rte_ether_addr_port/struct ether_addr_port/g' *.h
sed -i 's/struct vlan_hdr/prox_rte_vlan_hdr/g' *.c
sed -i 's/struct vlan_hdr/prox_rte_vlan_hdr/g' *.h
sed -i 's/struct ipv4_hdr/prox_rte_ipv4_hdr/g' *.h
sed -i 's/struct ipv4_hdr/prox_rte_ipv4_hdr/g' *.c
sed -i 's/struct ipv6_hdr/prox_rte_ipv6_hdr/g' *.c
sed -i 's/struct ipv6_hdr/prox_rte_ipv6_hdr/g' *.h
sed -i 's/struct udp_hdr/prox_rte_udp_hdr/g' *.c
sed -i 's/struct udp_hdr/prox_rte_udp_hdr/g' *.h
sed -i 's/struct tcp_hdr/prox_rte_tcp_hdr/g' *.c
sed -i 's/struct tcp_hdr/prox_rte_tcp_hdr/g' *.h
sed -i 's/struct prox_rte_ether_addr_copy/ether_addr_copy/g' *.c
sed -i 's/struct prox_rte_ether_addr_copy/ether_addr_copy/g' *.h
sed -i 's/struct prox_rte_ether_addr_copy/prox_rte_ether_addr_copy/g' *.c
sed -i 's/struct prox_rte_ether_addr_copy/ether_addr_copy/g' *.h
sed -i 's/prox_rte_ether_addr_copy/ether_addr_copy/g' *.h
sed -i 's/prox_rte_ether_addr_copy/ether_addr_copy/g' *.c
sed -i 's/ether_addr_copy/prox_rte_ether_addr_copy/g' *.c
sed -i 's/ether_addr_copy/prox_rte_ether_addr_copy/g' *.h
sed -i 's/prox_rte_ether_hdr_arp/struct ether_hdr_arp/g' *.h
sed -i 's/prox_rte_ether_hdr_arp/struct ether_hdr_arp/g' *.c
sed -i 's/struct vxlan_gpe_hdr/prox_rte_vxlan_gpe_hdr/g' *.h
sed -i 's/struct vxlan_gpe_hdr/prox_rte_vxlan_gpe_hdr/g' *.c
sed -i 's/eth_random_addr/prox_rte_eth_random_addr/g' *.c
sed -i 's/eth_random_addr/prox_rte_eth_random_addr/g' *.h
sed -i 's/ETHER_CRC_LEN/PROX_RTE_ETHER_CRC_LEN/g' *.c
sed -i 's/ETHER_CRC_LEN/PROX_RTE_ETHER_CRC_LEN/g' *.h
sed -i 's/ETHER_HDR_LEN/PROX_RTE_ETHER_HDR_LEN/g' *.c
sed -i 's/ETHER_HDR_LEN/PROX_RTE_ETHER_HDR_LEN/g' *.h
sed -i 's/ETHER_MAX_LEN/PROX_RTE_ETHER_MAX_LEN/g' *.c
sed -i 's/ETHER_MAX_LEN/PROX_RTE_ETHER_MAX_LEN/g' *.h
sed -i 's/ETHER_MIN_LEN/PROX_RTE_ETHER_MIN_LEN/g' *.c
sed -i 's/ETHER_MIN_LEN/PROX_RTE_ETHER_MIN_LEN/g' *.h
sed -i 's/TCP_SYN_FLAG/PROX_RTE_TCP_SYN_FLAG/g' *.c
sed -i 's/TCP_RST_FLAG/PROX_RTE_TCP_RST_FLAG/g' *.c
sed -i 's/TCP_FIN_FLAG/PROX_RTE_TCP_FIN_FLAG/g' *.c
sed -i 's/TCP_ACK_FLAG/PROX_RTE_TCP_ACK_FLAG/g' *.c
Change-Id: I4dfe8be68e618c94dcaae28754579fbeb571bb00
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Luc Provoost [Mon, 27 Jan 2020 15:14:09 +0000 (10:14 -0500)]
Code review feedback & back-ground traffic support
Various code review comments taken into account. We also support now
background traffic, meaning the scripts will start a test between
machines and monitor packet loss, latency, etc., but at the same time,
the script will also start traffic on background machines. That traffic
will not be monitored. Each generator will generate the same ammoung of
traffic.
Change-Id: I5cd495be230a526f91de965f3d4dd0581681f5ca
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Thu, 23 Jan 2020 11:12:03 +0000 (06:12 -0500)]
Fix run_iteration when not using the sut_sock
When no sut_sock monitoring the swap, the while loop would never come to
an end. Also added heartbeat timeout in the generator config files.
Change-Id: I3c36991c65b9b826faf3d051a1354ca7eb34c24e
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Fri, 17 Jan 2020 18:28:37 +0000 (13:28 -0500)]
Specify latency bucket size & correct calculation
A new parameter bucket_size_exp in the *.test file specifies the bucket
size for latency measurement. The default is 11 which corresponds to a
bucket size of 1 usec for a 2GHz processor. This fix is also calculating
the bucket size based on the reported frequency. Before it was assuming
a 2GHz processor.
Change-Id: I70d15e90e151a7e474cd05c9ac91b71a11fd5c78
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Mon, 13 Jan 2020 08:27:23 +0000 (03:27 -0500)]
Initial support for percentile latency measurements
We are now reporting a percentile number for the round-trip latency. The
numbers are only correct for processors running at 2GHz. Will fix this
next so it runs for any freqency. We only report 1-127 usec. If the
latency goes higher, we are now reporting "VERY HIGH".
Change-Id: If9fda63cea8616febcc0f24dc52590976cf39f3d
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Wed, 8 Jan 2020 10:18:28 +0000 (05:18 -0500)]
Combine code for fixed rate and variable rate tests
Previously, there were 2 functions: run_flow_size_test and
run_fixed_rate. The functionality of run_fixed_rate is now added to
run_flow_size_test.
Change-Id: Ifda70b8088b9131cc6c034d8007825e6d0917682
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Tue, 31 Dec 2019 13:30:07 +0000 (08:30 -0500)]
Better Latency accuray warning message
The lacency accuracy warning is now given only taking into account
packets that were not dropped. Also fixed an error with the value of r
in the run_iteration function
Change-Id: Ic05fd2e4b8c59a4b7bcc131e981c08e0001436a6
Signed-off-by: Luc Provoos <luc.provoost@intel.com>
Luc Provoost [Mon, 30 Dec 2019 15:28:14 +0000 (10:28 -0500)]
Support for ETSI GS NFV-TST 009
New test added TST009_Throughput.test allowing to run the throughput
benchmark as defined in ETSI GS NFV-TST 009. This change adds the binary
search algorithm as defined in the specification document.
Change-Id: I00e81a811a9c42b06136694fc1387bc59e954a28
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Mon, 2 Dec 2019 11:56:38 +0000 (06:56 -0500)]
Remove UUIDs in centos.json
Removed the remaining UUIDs from the centos.json file, making it easier
to reuse this file. This does require a recent version of the packer
tool. Updated the README accordingly and also added the --env option in
the packer alias, so that the OS_CLOUD variable is set when running
packer in the container. Updated rapid.vms with the security group name
used in the centos.json file.
Change-Id: I4f6815a3250811e336c14857e0ec5e7793a77acf
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Mon, 2 Dec 2019 15:38:35 +0000 (10:38 -0500)]
Fix mempool size for swap.cfg
Fixing an error re-introduced in the swap.cfg file. The mempool size
needs to be bigger than the number of rx and tx descriptors (we need
some extra buffers in the mempool for other things than rx and tx
descriptors). The default number of tx and rx descriptors has been
changed in PROX to 2K. This initially went through the testing since
this is just giving a warning in the prox.log file for some PMDs. For
other PMDs, PROX will not start.
Also fixed an inconsistency in rapid.vms: name of the security group.
Change-Id: If95677bd6169a16da1b3b2a56d13936a854c385e
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Wed, 20 Nov 2019 10:56:52 +0000 (11:56 +0100)]
rapid november release combining changes
-Renamed monitorswap.test into corestats.test. This is a more generic
test case that can monitor the core statistics of all PROX instances
-New portstats.test which can monitor the port statistics of all PROX
instances
-Removed the tasks parameter in the [DEFAULT] section of the *.test
files. Runrapid.py is now scaning the PROX configuration files to
automatically detect the PROX tasks that are being used.
-There is now an option to add the PushGateway parameter in the
[Default] section of some *.test files. This will push the measurements
to the Gateway, so they can be used by tools like Grafana. The logging
of the data in a csv file has also been improved. Results can now be
found on the screen, in the RUN*.log files, in the *.csv files and are
being pushed to the PushGateway.
-In the *.test files, replaced the swapcores, irqcores and impaircores
by the cores parameter since they are not treated differently by the
code. For a generator, we are still using the gencores and latcores
parameters since we need to treat them differently in runrapid.py
-devbind.sh is removed from the repository since this file is now
created by runrapid.py
-renamed the rapidVMs.vms file into rapid.vms
-image is now built without ssh access for root. Only using centos user.
runrapid.py is now using centos user for ssh sessions. PROX tools are
strted using sudo
-using new DPDK, mb_lib and PROX versions as defined in
deploycentostools.sh
-installing missing yum-utils package in image
-added flowsize parameter to the impairtest
-added list of flowsizes to the fixed_rate testing (l3framerate.test)
-the irqtest can now loop through multiple Test Machines in stead of
only checking the irq of one. Note that we should have the same cores
parameter for all Test machines
-changes in prox_ctrl.py to support setting the speed of a generator on
a combination of mulitple cores and tasks. Also added the
multi_port_stats function to support the new portstats.test
-Updated version to 19.11.21
-Fixed some power of 2 in the flow table
Change-Id: Ia14543bf3195a2b1bd4184b455da6940f1777fc9
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Illia Polliul [Thu, 24 Oct 2019 14:00:05 +0000 (16:00 +0200)]
Update RAPID readme about packer details
Change-Id: Ieff72073fff06818a5c5bc8fb9116461c95c2901
Signed-off-by: Illia Polliul <ipolliul@juniper.net>
Xavier Simonart [Wed, 23 Oct 2019 10:38:51 +0000 (12:38 +0200)]
Added support for DPDK 19.05 / IPSec
Note that since
6073b359 (April 2018) IPsec PROX support requires
at least DPDK 17.11. Hence README has been updated.
Change-Id: I34b7a968c16db5c3a3fc7bb6af111fdb49ad6b91
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Mon, 21 Oct 2019 14:19:45 +0000 (16:19 +0200)]
Fix two strncpy issues introduced by
fb0c44a8
Change-Id: Ic452b22dc2522a1b49e3091cb25f2784bc505b41
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Sat, 12 Oct 2019 22:31:39 +0000 (00:31 +0200)]
Update PROX version to (preprod) 1905 to reflect DPDK 19.05 support
Change-Id: I7ae7d9f2f5f3058df032062e0b2f6e3207086930
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Thu, 17 Oct 2019 12:02:53 +0000 (12:02 +0000)]
Merge "Increase the default number of rx and tx descriptors to 2K"
Xavier Simonart [Sat, 12 Oct 2019 22:27:47 +0000 (00:27 +0200)]
Fix support for old DPDK (such as 16.04)
When support for DPDK 18.05 was added it prevented supporting some
much older DPDK such as 16.04. This has been fixed.
Change-Id: Idd7ddd7e485c54c41def0540b8c126c097a05941
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Sat, 12 Oct 2019 21:36:41 +0000 (23:36 +0200)]
Increase the default number of rx and tx descriptors to 2K
A high number of RX descriptor can be helpful in avoiding
packet loss due to the core being interrupted.
This commit increases the default number of descriptors from 256 to 2K.
Those values are checked versus the minimum and maximum supported by the pmd.
Number of mbufs (mempool size) have been updated n config files to support this
default number of dsescriptors.
The number of descriptors can still be overwritten by the config file. However
those values must always remain within the limits reported by the pmd.
Change-Id: I5f7999eca886dad68f3c0733da0d796bf4c06a56
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Sat, 12 Oct 2019 22:18:14 +0000 (00:18 +0200)]
Disable TX checksum offload in some virtio cases
Some OVS versions reports that they support UDP offload and no IPv4 offload,
but fails when UDP offload is enabled.
Hence, now, when virtio reports that it supports UDP checksum and no IPV4
checksum offload, we disable UDP checksum offload.
Change-Id: I4f22f6be4d97b0aedf267ebeaa498c0dacef3317
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Fri, 11 Oct 2019 13:18:33 +0000 (13:18 +0000)]
Merge "Fix generation through pcap file"
Xavier Simonart [Wed, 4 Sep 2019 17:20:52 +0000 (19:20 +0200)]
Fix generation through pcap file
- When using "pcap file" argument in PROX config file in the gen mode
(and no submode), packets were truncated to 8 bytes (and a warning printed)
This has been fixed.
- "Set value" command was checking whether offset was within first 1518 bytes
unrespective to the mtu (e.g. jumbo packets). This has been fixed.
- There was a limit of 64K packets loaded from a pcap file (which could be
overwritten through PROX config). This limit has been removed - by default
all packets from pcap are loaded
- When pcap was loaded, memory was allocated to store the packet. However
just enough memory was allocated, so that there was no way to change (increase)
packet size through command line after that. Now memory is allocated to store
up to mtu size
- While "set value" command was acting on all packets, "pkt size" command
was only changing the first packet. This has now been changed and both commands
change all the packets.
- A few other minor cleanups.
Change-Id: I7264cee8d57e3e6a405dfd7e7a52a95b99399d99
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Mon, 1 Jul 2019 15:11:54 +0000 (17:11 +0200)]
Fix port throughput related statistics for i40e virtual functions
When CRC stripping is enabled (default PROX behavior), the number of
rx bytes received by the NIC returned by DPDK usually does not take the
CRC 4 bytes into account.
However, for i40e virtual functions those 4 bytes are counted as part of
the received bytes. This resulted in both DPDK and PROX taking those 4 bytes
into account causing the RX % to be above 100%.
Change-Id: I59883b638ed67a6778f80c4dd6bfbfc4f9f2f528
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Fri, 28 Jun 2019 14:56:26 +0000 (16:56 +0200)]
Update PROX to CRC STRIP default bahavior change in DPDK
Change-Id: I7c3e2981692d823c86787b77d61190327d6448e4
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Wed, 9 Oct 2019 17:03:29 +0000 (19:03 +0200)]
Fix strncpy issue introduced by
fb0c44a8
Change-Id: I833d753bd5c426d50c0c9fdee5668f3b9fff13b9
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Mon, 24 Jun 2019 15:01:54 +0000 (17:01 +0200)]
Add support for igmp and multicast
Multicast can be enabled through configuration or through command line
- Through configuration
Add multicast=mcast_address (e.g. multicast=01:00:5e:01:02:03) in the port
section
- Through command line
run enable multicast port_id mcast_address (e.g. enable multicast 1 01:00:5e:01:02:03)
IGMP join message is sent unsollicited through command line:
join igmp core_id task_id ip (e.g. join igmp 1 0 224.1.1.3)
To enable swap answering IGMP Query (w/ IGMP Join)
- Through configuration
Add igmp ipv4=ip_address within the core/task section
- Through command line
join igmp core_id task_id ip (e.g. join igmp 1 0 224.1.1.3)
(this will 1st initiate an unsollicated join, then answer any subsequent query)
UDP/TCP packets received on a multicast address (224.0.0.0 => 239.255.255.255) are discarded
To stop sending responses to IGMP query:
leave igmp core_id task_id
Change-Id: I3808ccabf3b38b5a1e10e1b044db63aa05bcd7b5
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Tue, 28 May 2019 13:31:43 +0000 (15:31 +0200)]
Added support for DPDK 19.02 and 19.05
This includes supporting following API changes
- RTE_VER_... now in rte_config.h instead of rte_version.h
- <rte_ethdev.h> now seems required by <rte_eth_ctrl.h>
- rte_sched_port_pkt_write takes extra argument
- rte_sched_port_pkt_read_tree_path takes extra argument
- enum rte_meter_color replaced by rte_color
The extra argument, sched_port, is initialized based on qos_conf parameters.
Modes such as qinq_encap4 uses sched_port when doing classification, but does
not set qos_conf params. It inherits the sched_port from QoS
Change-Id: If8312918241e7d482161a7538f53faac6c303e86
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Wed, 4 Sep 2019 12:07:53 +0000 (14:07 +0200)]
Fix parsing parameter errors in start and stop commands
Some wrong parameters in start (such as "start all 4" while there are only
less than 4 tasks available on some cores) and stop command could
have potentially caused a crash in some cases.
Change-Id: I6dc201575b574c53ded242ea795c35de82fb787e
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Tue, 8 Oct 2019 21:54:22 +0000 (17:54 -0400)]
Fix compatibility with GCC 6 and earlier
fb0c44a fixed compilation on GCC 8 by using:
1) #pragma GCC diagnostic ignored "-Wstringop-truncation"
2) __attribute__ ((fallthrough));
However, -Wstringop-truncation has only been introduced in GCC 8 and
__attribute__ ((fallthrough)) has only been introduced in GCC 7.
Hence they caused Warnings/Errors on earlier versions.
Comments about fallthough (see https://developers.redhat.com/blog/2017/03/10/wimplicit-fallthrough-in-gcc-7/)
can be used on GCC 8 to prevent fall through warnings
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Change-Id: I6f22e023191bbf22d8b01f1812061795a6494134
Xavier Simonart [Tue, 24 Sep 2019 16:42:44 +0000 (18:42 +0200)]
Fix setting mac address through [port] section
MAC address can be configured through the [port] section.
However this had only some effect on packets being transmitted.
For instance it was used to set the src mac address by the generator.
It had no effect on packets reception: when not in promiscuous mode
only the packets with the MAC originally set by the HW were received.
Now, when not in promiscuous mode, only packets with MAC as defined
in [port] section will be received.
Change-Id: I064817bf6727d098e89fbbb7a90412ed599f0fb6
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Tue, 24 Sep 2019 16:54:17 +0000 (18:54 +0200)]
Fix setting rx and tx queue stats mapping
rx_queue_stats_mapping and tx_queue_stats_mapping were called
unrespective of the number of active queues, and failing
on all unactive queues. this caused lot of errros in prox.log
Change-Id: I15ecb7adfb562caff8efb0eb24c341a3ba354571
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Mon, 19 Aug 2019 10:09:39 +0000 (06:09 -0400)]
Fix display error with repeated characters (ncurses/xterm related issue)
On recent OSes such as RHEL 8.0, ncurses(6.1) introduced support
for ECMA-48 repeat character control.
Some terminal emulators use TERM=xterm but do not support this feature.
In this case, printing repeated character such as "
22000000 Hz" might
be displayed as 220 Hz.
Other emulators, such as tmux, use TERM=screen, and do not exhibit the issue.
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Change-Id: I79977c7421699b59957d6cbd8aa954f06f3e0227
Xavier Simonart [Tue, 13 Aug 2019 14:14:32 +0000 (10:14 -0400)]
Support for recent GCC compiler
This is required for instance on gcc (GCC) 8.2.1
20180905
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Change-Id: Id86de5d39d77c5cbf168cc51434f436f84376a4c
Xavier Simonart [Wed, 4 Sep 2019 11:54:31 +0000 (13:54 +0200)]
Add heartbeat support (stop all cores in case of TCP socket issues)
"heartbeat timeout" (in second) can be specified as a global parameter
in PROX config file. If set, a timer is started when the first
command is received from the TCP socket. This timer is reset at each
commands received through the TCP socket. If the timer expires, then
- all cores are stopped
- the TCP socket is closed, causing an error at client side.
This feature helps in case a script starts PROX and the traffic generated
through PROX causes issues to the control plane.
Change-Id: I900f22fa091786a564f6b7d846f5abc2c5cbcc58
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Wed, 28 Aug 2019 08:34:43 +0000 (08:34 +0000)]
Merge "Add option to change scheduler policy to SCHED_RR and increase scheduler priority"
Xavier Simonart [Wed, 15 May 2019 09:48:26 +0000 (11:48 +0200)]
Add warning in case of wrong configuration (missing l3 submode)
Add warning if gateway_ip is configured and l3 is not configured.
If l3 sub mode is not configured, then gateway_ip parameter is ignored
(an error is not returned to make it easy to switch from an l3 sub
mode to default submode).
Change-Id: Ica4a522b037a024dd54bf616d32d69e29d5b8b92
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Fri, 28 Jun 2019 13:47:55 +0000 (15:47 +0200)]
Add option to change scheduler policy to SCHED_RR and increase scheduler priority
Set "realtime scheduling=yes" option to change Linux scheduler policy
to SCHED_RR and increase the priority to the maximum possible for the policy.
Change-Id: I3ecef5cbc3816cf2b56364bb4e806ae5ac093c23
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Fri, 14 Jun 2019 17:14:16 +0000 (19:14 +0200)]
Fix reset value
In addition to resetting all values set by the user, reset value was also
resetting internal configuration. It resulted that the src mac address
in the outgoing packet was set to whatever was specified in pkt inline,
which is wrong (by default src mac address should be set the the port mac address).
Note that reset value reset any "set values" but does not reset
the packet size.
Change-Id: I9d1cd92be9f5814bd7b104932dd6ffe5df3b6edd
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Patrice Buriez [Fri, 5 Jul 2019 14:19:07 +0000 (14:19 +0000)]
Merge changes from topic 'june_release'
* changes:
Some fixes after code review
Cosmetic change when printing warnings
Multiple changes for June release
Luc Provoost [Fri, 5 Jul 2019 10:05:20 +0000 (06:05 -0400)]
Some fixes after code review
Taking into account comments from Patrice and Xavier
Change-Id: Ifdabd1945e074c9ee97b059956f107901392c020
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Xavier Simonart [Wed, 3 Jul 2019 22:04:17 +0000 (00:04 +0200)]
Fix keyboard related issues on Ubuntu 18.04
On Ubuntu 18.04, the keyboard was not properly handled in PROX
For instance, many keys pressed were silently discarded.
The issue is related to a change in histedit/libedit version.
Recent libedit have (silently) changed the prototype of the get_char function
passed in el_set(el, EL_GETCFN, get_char), with some input parameters
changing from char to wchar_t.
As PROX used the old prototype (char based), this resulted in some
uninitialized field (= garbage) in the wchar_t, causing libedit to discard
the character.
PROX now uses different get_char prototypes, depending of the libedit version
being used.
If PROX was already compiled and a the OS is updated (e.g. from Ubuntu 16.04
to Ubuntu 18.04), this will require a 'make clean'.
Change-Id: Icb0e555a21e13cdaf98172bad17f2f838fb7bc3a
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Provoost [Mon, 1 Jul 2019 15:21:00 +0000 (11:21 -0400)]
Cosmetic change when printing warnings
Warnings can be printed in the following cases:
- When not enough packets are taken into account for measuring packet
latency accuracy.
- When there us a potential network throughput issue, meaning the generator
is generating more packets than the the NIC can handle
- When the generator cannot generate the requested load
If there are no warnings for a certain measurement, nothing gets printed,
suppressing an empty line
Change-Id: Iee07c12142e28dcc0ac406bfed7626731ab08f98
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Sun, 30 Jun 2019 07:46:10 +0000 (09:46 +0200)]
Multiple changes for June release
- Change inittest in warmuptest. This has now a warmuptime, warmupspeed,
packet size and flow size.
- Change in centos.json. This will now also copy deploycentostools.sh
during image building so that it can be used to re-compile prox in the VM
by typing "./deploycentostools.sh compile"
- runrapid.py parameters needing a file name, need to be complete file
names: no extensions will be added any more by the scripts.
- Changes in createrapid.py to handel the openstack CLI output in a
simpler way.
- The management interface of the VMs can now also be an SRIOV interface.
We have now an extra optional parameter in the VM sections of the
rapiVMs.vms file: SRIOV_mgmt_port
- Changed the name of some sections and keys in the environment file since
runrapid.py will not always communicate to OpenStack VMs. This could be
containers or any other (virtual) machine.
- The previous MachineMap.cfg has been renamed to machine.map
- A new test has been added: monitorswap is just showing the statistics of
a swap (virtual) machine, without generating any packets. This is useful
in situations where an external tester is used.
- Latency and core statistics can now be measured even if there are
multiple PROX cores and tasks running. A new parameter has been added in
the test files with following default value: tasks=[0]. During the
collection of statistics, all tasks in this list will be queried. It is
ok to have a non-existing task in such a query since it will be ignored.
- A --screenlog parameter was added for runrapid.py allowing for more
detailed output on the screen during debugging. No need to check the log
file.
- Previous tests to run multiple packet size with a given flow size and to
run multiple flow size, with a given packet size are now combined by
specifiying 2 lists: packetsizes & flows
- The screen output of this test has also been reworked with more
meaningful column names and the test result is now in the field "core
received". This allows also for faster termination of the test: When the
all packets sent by Gen NIC are received by the cores within the
thresholds for packet loss and latency, the test is now stopped, even we
were requesting more packets to be sent.
Change-Id: I3307e7a972f2140e739f376f146fe875df0303e6
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Luc Provoost [Thu, 27 Jun 2019 10:52:11 +0000 (12:52 +0200)]
Change of openstackrapid directory name
The scripts are orrignally written to support an OpenStack deployment.
We are now making changes to support container environments, hence
renaming the directory to rapid.
Change-Id: Icaa2fad83c6da451eee733934151e98e9cd07de4
Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Xavier Simonart [Thu, 27 Jun 2019 12:38:02 +0000 (14:38 +0200)]
Fix "code nested too deep error" introduced by commit
843ca042
Change-Id: Ifb55091feedb28ffc5947ffe61e6f6a5ce034459
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Fri, 7 Jun 2019 12:54:43 +0000 (14:54 +0200)]
Increase window size for latency accuracy
Accuracy of latency is obtained by writing TX accuracy for packet N
in a later packet i.e. packet N + WINDOW.
If this window is too small, packet N + WINDOW (conveying accuracy
for packet N) might arrive before packet N, resulting in no accuracy
for packet N.
This change increases this window from 64 to 8K packets.
This change should result in a higher number of packets used for
latency measurements when packets are reordered.
This change might have a performance impact as it uses more memory.
Change-Id: I1016fddb66af86605c73a24050238da41cf54152
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
Xavier Simonart [Fri, 14 Jun 2019 15:12:56 +0000 (17:12 +0200)]
Add support for latency task with multiple ports
Change-Id: Iaae603a7558666b5794ec4c76f447eb3260a1df9
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
Xavier Simonart [Fri, 7 Jun 2019 12:42:41 +0000 (14:42 +0200)]
Support multiple tasks in "lat stats" command
Similar support and comments as for "dp core stats" command
Change-Id: I215fc57c292fbc1f12a3fe95d40a0a80c29b714a
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
Xavier Simonart [Wed, 15 May 2019 10:36:21 +0000 (12:36 +0200)]
Support multiple tasks in "dp core stats" command
The "dp core stats" command supported, as other commands, multiple cores
but not multiple tasks. It now supports both (e.g. dp core stats 1-2 0-1).
This is necessary so that users can collect all statistics from PROX in one
command line.
It will return similar information on the screen or through the socket:
- if the syntax is wrong (e.g. missing task, too many cores or tasks, ...)
=> an error message is printed on the screen, and the single error line
"error: invalid syntax" is returned through the socket
- if the syntax is correct
=> one line is printed or returned for each core/task pair
=> if the core/task pair is invalid, the line reports an error
=> otherwise, the line provides dataplane statistics for the core/task pair,
with the core_id and task_id added, either at the end of the line returned
through the socket, or at the beginning of the line printed on the screen.
This means a change in behaviour when using the socket: before, when an error
happened (e.g. bad task), then nothing was returned which made it difficult for
the script to catch the syntax error.
Similar behaviour could/should be implemented for other commands.
However, care must be taken as some scripts like NSB might count the number
of lines within the output, or the number of items within the line.
This should not be an issue for dp core stats as NSB does not use it, and other
PROX scripts do not count the number of items and will be adjusted to count the
number of lines.
Change-Id: I582a671ae3d2f6493f791e80fc28e70f8e3a38d1
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
Patrice Buriez [Fri, 14 Jun 2019 15:03:14 +0000 (17:03 +0200)]
Add parse_cores_tasks()
and rename parse_core_task() to parse_cores_task()
Change-Id: I256dfb8d02520a70d65d6623feddcc9037938501
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
Patrice Buriez [Wed, 12 Jun 2019 16:39:15 +0000 (18:39 +0200)]
Add "total TX fail" count to "dp core stats" command output
Warning:
This change breaks backward-compatibility of PROX socket scripting API,
because the "total TX fail" count is reported before the "last TSC"
field.
As a result, PROX scripts that make use of the "dp core stats" command,
such as the core_stats() method in helper-scripts/openstackrapid/prox_ctrl.py,
MUST be reworked to correctly parse the returned set of fields.
Change-Id: I3fe7f37dccee19f5f0b2719f262c6cfa7bad850f
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
Patrice Buriez [Wed, 12 Jun 2019 15:17:26 +0000 (17:17 +0200)]
Simplify cores_task_are_valid()
by means of calling core_task_is_valid()
Change-Id: Ice64aabc84066633288ff43d4dc9af3f5b378e33
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
Xavier Simonart [Tue, 28 May 2019 18:24:00 +0000 (20:24 +0200)]
Fix soft checksum calculation
Soft CRC was wrong in some cases, such as when compiled with gcc -O2
This might have caused the checksum to be set to 0 in those cases.
Change-Id: Idf01df92876bd2a152ad028293b67a31861a0dfc
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
Deepak S [Fri, 19 Apr 2019 14:01:42 +0000 (14:01 +0000)]
Merge "PROX: update version"
Xavier Simonart [Mon, 15 Apr 2019 09:18:11 +0000 (11:18 +0200)]
PROX: update version
PROX version was not udpated since a long time, resulting in
difficulty to easily understand which version is being run.
Improved solution should be to support sha1.
Change-Id: I9d68bd64f52f32544d31d2b8c6bcd0c55c7c228c
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Xavier Simonart [Mon, 15 Apr 2019 08:48:46 +0000 (10:48 +0200)]
PROX: fix minimum rx buffer size used within generator
Different PMD have different requirements for minimum rx buffer size.
vmxnet3 has (in DPDK 18.08) requires minimum 1518 + 128 (RTE_PKTMBUF_HEADROOM).
Change-Id: I327ae5b8dcf7eef8dd71fb4e2f0603b9bab6e1d3
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Luc Provoost [Fri, 12 Apr 2019 09:20:58 +0000 (11:20 +0200)]
Improved image building & new PROX
Bug fixes for the script for building an image via the packer tool. This
also installs a newer version of PROX with some bug fixes in PROX. The
*.test files have been simplified. Also introduced a new file
MachineMap.cfg to map test machines on actual VMs.
Change-Id: I69445327ef0a907bc8c1566aaa60c733418c541e
Signed-off-by: Luc Provoost <luc.provoost@intel.com>