**Title**: RFC 2889 X% packet loss Max Forwarding Rate Soak Test
- **Prerequisite Test** LTD.Throughput.RFC2544.PacketLossRatio
+ **Prerequisite Tests**:
+
+ LTD.Throughput.RFC2544.PacketLossRatio will determine the offered load and
+ frame size for which the maximum theoretical throughput of the interface
+ has not been achieved. As described in RFC 2544 section 24, the final
+ determination of the benchmark SHOULD be conducted using a full length
+ trial, and for this purpose the duration is 5 minutes with zero loss ratio.
+
+ It is also essential to verify that the Traffic Generator has sufficient
+ stability to conduct Soak tests. Therefore, a prerequisite is to perform
+ this test with the DUT removed and replaced with a cross-over cable (or
+ other equivalent very low overhead method such as a loopback in a HW switch),
+ so that the traffic generator (and any other network involved) can be tested
+ over the Soak period. Note that this test may be challenging for software-
+ based traffic generators.
**Priority**:
The aim of this test is to understand the Max Forwarding Rate stability
over an extended test duration in order to uncover any outliers. To allow
for an extended test duration, the test should ideally run for 24 hours
- or, if this is not possible, for at least 6 hours. For this test, each frame
- size must be sent at the highest Throughput rate with X% packet loss, as
- determined in the prerequisite test. The default loss percentages to be
- tested are: - X = 0% - X = 10^-7%
+ or if this is not possible, for at least 6 hours.
- Note: Other values can be tested if required by the user.
+ For this test, one frame size must be sent at the highest frame rate with
+ X% packet loss ratio, as determined in the prerequisite test (a short trial).
+ The loss ratio shall be measured and recorded every 5 minutes during the test
+ (it may be sufficient to collect lost frame counts and divide by the number
+ of frames sent in 5 minutes to see if a threshold has been crossed,
+ and accept some small inaccuracy in the threshold evaluation, not the result).
+ The default loss ratio is X = 0% and loss ratio > 10^-7% is the default
+ threshold to terminate the test early (or inform the test operator of
+ the failure status).
+
+ Note: Other values of X and loss threshold can be tested if required by the user.
**Expected Result**:
and reporting any time intervals with packet loss. The
`RFC2889 <https://www.rfc-editor.org/rfc/rfc2289.txt>`__
Forwarding Rate shall be measured in each interval.
- An interval of 60s is suggested.
+ An interval of 300s is suggested.
- CPU and memory utilization may also be collected as part of this
test, to determine the vSwitch's performance footprint on the system.
- The `RFC5481 <https://www.rfc-editor.org/rfc/rfc5481.txt>`__
PDV form of delay variation on the traffic flow,
- using the 99th percentile.
+ using the 99th percentile, may also be collected.
.. 3.2.2.1.7
**Title**: RFC 2889 Max Forwarding Rate Soak Test with Frame Modification
**Prerequisite Test**:
+
LTD.Throughput.RFC2544.PacketLossRatioFrameModification (0% Packet Loss)
+ will determine the offered load and
+ frame size for which the maximum theoretical throughput of the interface
+ has not been achieved. As described in RFC 2544 section 24, the final
+ determination of the benchmark SHOULD be conducted using a full length
+ trial, and for this purpose the duration is 5 minutes with zero loss ratio.
+
+ It is also essential to verify that the Traffic Generator has sufficient
+ stability to conduct Soak tests. Therefore, a prerequisite is to perform
+ this test with the DUT removed and replaced with a cross-over cable (or
+ other equivalent very low overhead method such as a loopback in a HW switch),
+ so that the traffic generator (and any other network involved) can be tested
+ over the Soak period. Note that this test may be challenging for software-
+ based traffic generators.
+
**Priority**:
The aim of this test is to understand the Max Forwarding Rate stability over an
extended test duration in order to uncover any outliers. To allow for an
extended test duration, the test should ideally run for 24 hours or, if
- this is not possible, for at least 6 hour. For this test, each frame
- size must be sent at the highest Throughput rate with 0% packet loss, as
- determined in the prerequisite test.
+ this is not possible, for at least 6 hours.
+
+ For this test, one frame size must be sent at the highest frame rate with
+ X% packet loss ratio, as determined in the prerequisite test (a short trial).
+ The loss ratio shall be measured and recorded every 5 minutes during the test
+ (it may be sufficient to collect lost frame counts and divide by the number
+ of frames sent in 5 minutes to see if a threshold has been crossed,
+ and accept some small inaccuracy in the threshold evaluation, not the result).
+ The default loss ratio is X = 0% and loss ratio > 10^-7% is the default
+ threshold to terminate the test early (or inform the test operator of
+ the failure status).
+
+ Note: Other values of X and loss threshold can be tested if required by the user.
During this test, the DUT must perform the following operations on the
traffic flow:
and reporting any time intervals with packet loss. The
`RFC2889 <https://www.rfc-editor.org/rfc/rfc2289.txt>`__
Forwarding Rate shall be measured in each interval.
- An interval of 60s is suggested.
+ An interval of 300s is suggested.
- CPU and memory utilization may also be collected as part of this
test, to determine the vSwitch's performance footprint on the system.
- The `RFC5481 <https://www.rfc-editor.org/rfc/rfc5481.txt>`__
PDV form of delay variation on the traffic flow, using the 99th
- percentile.
+ percentile, may also be collected.
.. 3.2.2.1.8
**Title**: Packet Delay Variation Soak Test
- **Prerequisite Tests**: LTD.Throughput.RFC2544.PacketLossRatio (0% Packet Loss)
+ **Prerequisite Tests**:
+
+ LTD.Throughput.RFC2544.PacketLossRatio will determine the offered load and
+ frame size for which the maximum theoretical throughput of the interface
+ has not been achieved. As described in RFC 2544 section 24, the final
+ determination of the benchmark SHOULD be conducted using a full length
+ trial, and for this purpose the duration is 5 minutes with zero loss ratio.
+
+ It is also essential to verify that the Traffic Generator has sufficient
+ stability to conduct Soak tests. Therefore, a prerequisite is to perform
+ this test with the DUT removed and replaced with a cross-over cable (or
+ other equivalent very low overhead method such as a loopback in a HW switch),
+ so that the traffic generator (and any other network involved) can be tested
+ over the Soak period. Note that this test may be challenging for software-
+ based traffic generators.
+
**Priority**:
variation for different frame sizes over an extended test duration and
to determine if there are any outliers. To allow for an extended test
duration, the test should ideally run for 24 hours or, if this is not
- possible, for at least 6 hour. For this test, each frame size must be
- sent at the highest possible throughput with 0% packet loss, as
- determined in the prerequisite test.
+ possible, for at least 6 hours.
+
+ For this test, one frame size must be sent at the highest frame rate with
+ X% packet loss ratio, as determined in the prerequisite test (a short trial).
+ The loss ratio shall be measured and recorded every 5 minutes during the test
+ (it may be sufficient to collect lost frame counts and divide by the number
+ of frames sent in 5 minutes to see if a threshold has been crossed,
+ and accept some small inaccuracy in the threshold evaluation, not the result).
+ The default loss ratio is X = 0% and loss ratio > 10^-7% is the default
+ threshold to terminate the test early (or inform the test operator of
+ the failure status).
+
+ Note: Other values of X and loss threshold can be tested if required by the user.
+
**Expected Result**:
- The packet delay variation value for traffic passing through the DUT.
- The `RFC5481 <https://www.rfc-editor.org/rfc/rfc5481.txt>`__
PDV form of delay variation on the traffic flow,
- using the 99th percentile, for each 60s interval during the test.
+ using the 99th percentile, for each 300s interval during the test.
- CPU and memory utilization may also be collected as part of this
test, to determine the vSwitch's performance footprint on the system.
# pylint: disable=wrong-import-position, import-error
sys.path.append(settings.getValue('PATHS')['trafficgen']['Trex']['src']['path'])
from trex_stl_lib.api import *
+ from trex_stl_lib import trex_stl_exceptions
except ImportError:
# VSPERF performs detection of T-Rex api during testcase initialization. So if
# T-Rex is requsted and API is not available it will fail before this code
'tx_pps': 0.0,
'tx_util': 0.0,}}
+
class Trex(ITrafficGenerator):
"""Trex Traffic generator wrapper."""
_logger = logging.getLogger(__name__)
self._trex_user = settings.getValue('TRAFFICGEN_TREX_USER')
self._stlclient = None
self._verification_params = None
+ self._show_packet_data = False
+
+ def show_packet_info(self, packet_a, packet_b):
+ """
+ Log packet layers to screen
+ :param packet_a: Scapy.layers packet
+ :param packet_b: Scapy.layers packet
+ :return: None
+ """
+ # we only want to show packet data once per test
+ if self._show_packet_data:
+ self._show_packet_data = False
+ self._logger.info(packet_a.show())
+ self._logger.info(packet_b.show())
def connect(self):
'''Connect to Trex traffic generator
self._stlclient.set_service_mode(ports=my_ports, enabled=False)
ports_info = self._stlclient.get_port_info(my_ports)
+
+ # get max support speed
+ max_speed = 0
+ if settings.getValue('TRAFFICGEN_TREX_FORCE_PORT_SPEED'):
+ max_speed = settings.getValue('TRAFFICGEN_TREX_PORT_SPEED')
+ elif ports_info[0]['supp_speeds']:
+ max_speed_1 = max(ports_info[0]['supp_speeds'])
+ max_speed_2 = max(ports_info[1]['supp_speeds'])
+ else:
+ # if max supported speed not in port info or set manually, just assume 10G
+ max_speed = 10000
+ if not max_speed:
+ # since we can only control both ports at once take the lower of the two
+ max_speed = min(max_speed_1, max_speed_2)
+ gbps_speed = (max_speed / 1000) * (float(traffic['frame_rate']) / 100.0)
+ self._logger.debug('Starting traffic at %s Gpbs speed', gbps_speed)
+
# for SR-IOV
if settings.getValue('TRAFFICGEN_TREX_PROMISCUOUS'):
self._stlclient.set_port_attr(my_ports, promiscuous=True)
packet_1, packet_2 = Trex.create_packets(traffic, ports_info)
+ self.show_packet_info(packet_1, packet_2)
stream_1, stream_2, stream_1_lat, stream_2_lat = Trex.create_streams(packet_1, packet_2, traffic)
self._stlclient.add_streams(stream_1, ports=[0])
self._stlclient.add_streams(stream_2, ports=[1])
pcap_id[pcap_dir] = self._stlclient.start_capture(**capture)
self._stlclient.clear_stats()
- self._stlclient.start(ports=my_ports, force=True, duration=duration)
+ # if the user did not start up T-Rex server with more than default cores, use default mask.
+ # Otherwise use mask to take advantage of multiple cores.
+ try:
+ self._stlclient.start(ports=my_ports, force=True, duration=duration, mult="{}gbps".format(gbps_speed),
+ core_mask=self._stlclient.CORE_MASK_PIN)
+ except STLError:
+ self._stlclient.start(ports=my_ports, force=True, duration=duration, mult="{}gbps".format(gbps_speed))
self._stlclient.wait_on_traffic(ports=my_ports)
stats = self._stlclient.get_stats(sync_now=True)
if test_lossrate <= lossrate:
# save the last passing trial for verification
self._verification_params = copy.deepcopy(new_params)
- self._logger.debug("Iteration: %s, frame rate: %s, throughput_rx_fps: %s, frame_loss_percent: %s",
- iteration, "{:.3f}".format(new_params['frame_rate']), stats['total']['rx_pps'],
- "{:.3f}".format(test_lossrate))
+ packets_lost = stats['total']['opackets'] - stats['total']['ipackets']
+ self._logger.debug("Iteration: %s, frame rate: %s, throughput_rx_fps: %s," +
+ " frames lost %s, frame_loss_percent: %s", iteration,
+ "{:.3f}".format(new_params['frame_rate']), stats['total']['rx_pps'],
+ packets_lost, "{:.3f}".format(test_lossrate))
if test_lossrate == 0.0 and new_params['frame_rate'] == traffic['frame_rate']:
return copy.deepcopy(stats)
elif test_lossrate > lossrate:
self._logger.info("In Trex send_cont_traffic method")
self._params.clear()
+ self._show_packet_data = True
+
self._params['traffic'] = self.traffic_defaults.copy()
if traffic:
self._params['traffic'] = merge_spec(
"""
self._logger.info("In Trex send_rfc2544_throughput method")
self._params.clear()
+ self._show_packet_data = True
self._params['traffic'] = self.traffic_defaults.copy()
if traffic:
self._params['traffic'] = merge_spec(
Code Review / vswitchperf.git / commitdiff