tc_epc_saegw_tput_relocation_landslide
tc_epc_network_service_request_landslide
tc_epc_ue_service_request_landslide
+ tc_vfw_rfc2544.rst
+ tc_vfw_rfc2544_correlated.rst
+ tc_vfw_rfc3511.rst
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+************************************************
+Yardstick Test Case Description: NSB vFW RFC2544
+************************************************
+
++------------------------------------------------------------------------------+
+| NSB vFW test for VNF characterization |
+| |
++---------------+--------------------------------------------------------------+
+| test case id | tc_{context}_rfc2544_ipv4_1rule_1flow_{pkt_size}_{tg_type} |
+| | |
+| | * context = baremetal, heat, heat_external, ovs, sriov |
+| | heat_sriov_external contexts; |
+| | * tg_type = ixia (context != heat,heat_sriov_external), |
+| | trex; |
+| | * pkt_size = 64B - all contexts; |
+| | 128B, 256B, 512B, 1024B, 1280B, 1518B - |
+| | (context = heat, tg_type = ixia) |
+| | |
++---------------+--------------------------------------------------------------+
+| metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * TG Latency; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * VNF Packets Fwd; |
+| | * Dropped packets; |
+| | |
++---------------+--------------------------------------------------------------+
+| test purpose | The VFW RFC2544 tests measure performance characteristics of |
+| | the SUT (multiple ports) and sends UDP bidirectional traffic |
+| | from all TG ports to SampleVNF vFW application. The |
+| | application forwards received traffic based on rules |
+| | provided by the user in the TC configuration and default |
+| | rules created by vFW to send traffic from uplink ports to |
+| | downlink and voice versa. |
+| | |
++---------------+--------------------------------------------------------------+
+| configuration | The 2 ports RFC2544 test cases are listed below: |
+| | |
+| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
+| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1024B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1280B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_128B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1518B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_256B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_512B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
+| | * tc_heat_sriov_external_rfc2544_ipv4_1rule_1flow_64B_trex. |
+| | yaml |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
+| | * tc_ovs_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
+| | * tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
+| | * tc_sriov_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
+| | * tc_sriov_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
+| | |
+| | The 4 ports RFC2544 test cases are listed below: |
+| | |
+| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia_4port.yaml |
+| | * tc_tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex_4port. |
+| | yaml |
+| | * tc_tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_trex_4 |
+| | port.yaml |
+| | * tc_tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_4port.yaml |
+| | |
+| | The scale-up RFC2544 test cases are listed below: |
+| | |
+| | * tc_tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml |
+| | |
+| | The scale-out RFC2544 test cases are listed below: |
+| | |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale_out.yaml |
+| | |
+| | Test duration is set as 30 sec for each test and default |
+| | number of rules are applied. These can be configured |
+| | |
++---------------+--------------------------------------------------------------+
+| test tool | The vFW is a DPDK application that performs basic filtering |
+| | for malformed packets and dynamic packet filtering of |
+| | incoming packets using the connection tracker library. |
+| | |
++---------------+--------------------------------------------------------------+
+| applicability | The vFW RFC2544 test cases can be configured with different: |
+| | |
+| | * packet sizes; |
+| | * test duration; |
+| | * tolerated loss; |
+| | * traffic flows; |
+| | * rules; |
+| | |
+| | Default values exist. |
+| | |
++---------------+--------------------------------------------------------------+
+| pre-test | For OpenStack test case image (yardstick-samplevnf) needs |
+| conditions | to be installed into Glance with vFW and DPDK included in |
+| | it (NSB install). |
+| | |
+| | For Baremetal tests cases vFW and DPDK must be installed on |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information. |
+| | |
+| | For standalone (SA) SRIOV/OvS test cases the |
+| | yardstick-samplevnf image needs to be installed on hosts and |
+| | pod.yaml file must be provided with necessary system, NIC |
+| | information. |
+| | |
++---------------+--------------------------------------------------------------+
+| test sequence | Description and expected result |
+| | |
++---------------+--------------------------------------------------------------+
+| step 1 | For Baremetal test: The TG (except IXIA) and VNF are started |
+| | on the hosts based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(vFW) based on the test flavor. In case of scale-out |
+| | scenario the multiple VNF VMs will be started. |
+| | |
+| | For Heat external test: vFW VM is booted and TG (except IXIA)|
+| | generator is started on the external host based on the pod |
+| | file. In case of scale-out scenario the multiple VNF VMs |
+| | will be deployed. |
+| | |
+| | For Heat SRIOV external test: vFW VM is booted with network |
+| | interfaces of `direct` type which are mapped to VFs that are |
+| | available to OpenStack. TG (except IXIA) is started on the |
+| | external host based on the pod file. In case of scale-out |
+| | scenario the multiple VNF VMs will be deployed. |
+| | |
+| | For SRIOV test: VF ports are created on host's PFs specified |
+| | in the TC file and VM is booed using those ports and image |
+| | provided in the configuration. TG (except IXIA) is started |
+| | on other host connected to VNF machine based on the pod |
+| | file. The vFW is started in the booted VM. In case of |
+| | scale-out scenario the multiple VNF VMs will be created. |
+| | |
+| | For OvS-DPDK test: OvS DPDK switch is started and bridges |
+| | are created with ports specified in the TC file. DPDK vHost |
+| | ports are added to corresponding bridge and VM is booed |
+| | using those ports and image provided in the configuration. |
+| | TG (except IXIA) is started on other host connected to VNF |
+| | machine based on the pod file. The vFW is started in the |
+| | booted VM. In case of scale-out scenario the multiple VNF |
+| | VMs will be deployed. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 2 | Yardstick is connected with the TG and VNF by using ssh (in |
+| | case of IXIA TG is connected via TCL interface). The test |
+| | will resolve the topology and instantiate all VNFs |
+| | and TG and collect the KPI's/metrics. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 3 | The TG will send packets to the VNFs. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for different |
+| | packet size with an accepted minimal packet loss for the |
+| | default configuration. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 4 | In Baremetal test: The test quits the application and unbind |
+| | the DPDK ports. |
+| | |
+| | In Heat test: All VNF VMs and TG are deleted on test |
+| | completion. |
+| | |
+| | In SRIOV test: The deployed VM with vFW is destroyed on the |
+| | host and TG (exclude IXIA) is stopped. |
+| | |
+| | In Heat SRIOV test: The deployed VM with vFW is destroyed, |
+| | VFs are released and TG (exclude IXIA) is stopped. |
+| | |
+| | In OvS test: The deployed VM with vFW is destroyed on the |
+| | host and OvS DPDK switch is stopped and ports are unbinded. |
+| | The TG (exclude IXIA) is stopped. |
+| | |
++---------------+--------------------------------------------------------------+
+| test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++---------------+--------------------------------------------------------------+
+
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+*************************************************************
+Yardstick Test Case Description: NSB vFW RFC2544 (correlated)
+*************************************************************
+
++------------------------------------------------------------------------------+
+| NSB vFW test for VNF characterization using correlated traffic |
+| |
++---------------+--------------------------------------------------------------+
+| test case id | tc_{context}_rfc2544_ipv4_1rule_1flow_64B_trex_corelated |
+| | |
+| | * context = baremetal, heat |
+| | |
++---------------+--------------------------------------------------------------+
+| metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * TG Latency; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * VNF Packets Fwd; |
+| | * Dropped packets; |
+| | |
+| | NOTE: For correlated TCs the TG metrics are available on |
+| | uplink ports. |
+| | |
++---------------+--------------------------------------------------------------+
+| test purpose | The VFW RFC2544 correlated tests measure performance |
+| | characteristics of the SUT (multiple ports) and sends UDP |
+| | traffic from uplink TG ports to SampleVNF vFW application. |
+| | The application forwards received traffic from uplink ports |
+| | to downlink ports based on rules provided by the user in the |
+| | TC configuration and default rules created by vFW. The VNF |
+| | downlink traffic is received by another UDPReplay VNF and it |
+| | is mirrored back to the VNF on the same port. Finally, the |
+| | traffic is received back to the TG uplink port. |
+| | |
++---------------+--------------------------------------------------------------+
+| configuration | The 2 ports RFC2544 correlated test cases are listed below: |
+| | |
+| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex_corelated |
+| | _traffic.yaml |
+| | |
+| | Multiple VNF (2, 4, 10) RFC2544 correlated test cases are |
+| | listed below: |
+| | |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated |
+| | _scale_10.yaml |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale |
+| | _2.yaml |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale |
+| | _4.yaml |
+| | |
+| | The scale-out RFC2544 test cases are listed below: |
+| | |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale |
+| | _out.yaml |
+| | |
+| | Test duration is set as 30 sec for each test and default |
+| | number of rules are applied. These can be configured |
+| | |
++---------------+--------------------------------------------------------------+
+| test tool | The vFW is a DPDK application that performs basic filtering |
+| | for malformed packets and dynamic packet filtering of |
+| | incoming packets using the connection tracker library. |
+| | |
++---------------+--------------------------------------------------------------+
+| applicability | The vFW RFC2544 test cases can be configured with different: |
+| | |
+| | * packet sizes; |
+| | * test duration; |
+| | * tolerated loss; |
+| | * traffic flows; |
+| | * rules; |
+| | |
+| | Default values exist. |
+| | |
++---------------+--------------------------------------------------------------+
+| pre-test | For OpenStack test case image (yardstick-samplevnf) needs |
+| conditions | to be installed into Glance with vFW and DPDK included in |
+| | it (NSB install). |
+| | |
+| | For Baremetal tests cases vFW and DPDK must be installed on |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information. |
+| | |
++---------------+--------------------------------------------------------------+
+| test sequence | Description and expected result |
+| | |
++---------------+--------------------------------------------------------------+
+| step 1 | For Baremetal test: The TG (except IXIA), vFW and UDPReplay |
+| | VNFs are started on the hosts based on the pod file. |
+| | |
+| | For Heat test: Three host VMs are booted, as Traffic |
+| | generator, vFW and UDPReplay VNF(vFW) based on the test |
+| | flavor. In case of scale-out scenario the multiple vFW VNF |
+| | VMs will be started. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 2 | Yardstick is connected with the TG, vFW and UDPReplay VNF by |
+| | using ssh (in case of IXIA TG is connected via TCL |
+| | interface). The test will resolve the topology and |
+| | instantiate all VNFs and TG and collect the KPI's/metrics. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 3 | The TG will send packets to the VNFs. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 64B packet |
+| | size with an accepted minimal packet loss for the default |
+| | configuration. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 4 | In Baremetal test: The test quits the application and unbind |
+| | the DPDK ports. |
+| | |
+| | In Heat test: All VNF VMs and TG are deleted on test |
+| | completion. |
+| | |
++---------------+--------------------------------------------------------------+
+| test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++---------------+--------------------------------------------------------------+
+
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+*******************************************************
+Yardstick Test Case Description: NSB vFW RFC3511 (HTTP)
+*******************************************************
+
++------------------------------------------------------------------------------+
+| NSB vFW test for VNF characterization based on RFC3511 and IXIA |
+| |
++---------------+--------------------------------------------------------------+
+| test case id | tc_{context}_http_ixload_{http_size}_Requests-65000_{type} |
+| | |
+| | * context = baremetal, heat_external |
+| | * http_size = 1b, 4k, 64k, 256k, 512k, 1024k payload size |
+| | * type = Concurrency, Connections, Throughput |
+| | |
++---------------+--------------------------------------------------------------+
+| metric | * HTTP Total Throughput (Kbps); |
+| | * HTTP Simulated Users; |
+| | * HTTP Concurrent Connections; |
+| | * HTTP Connection Rate; |
+| | * HTTP Transaction Rate |
+| | |
++---------------+--------------------------------------------------------------+
+| test purpose | The vFW RFC3511 tests measure performance characteristics of |
+| | the SUT by sending the HTTP traffic from uplink to downlink |
+| | TG ports through vFW VNF. The application forwards received |
+| | traffic based on rules provided by the user in the TC |
+| | configuration and default rules created by vFW to send |
+| | traffic from uplink ports to downlink and voice versa. |
+| | |
++---------------+--------------------------------------------------------------+
+| configuration | The 2 ports RFC3511 test cases are listed below: |
+| | |
+| | * tc_baremetal_http_ixload_1024k_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_baremetal_http_ixload_1b_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_baremetal_http_ixload_256k_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_baremetal_http_ixload_4k_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_baremetal_http_ixload_512k_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_baremetal_http_ixload_64k_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_heat_external_http_ixload_1b_Requests-10Gbps |
+| | _Throughput.yaml |
+| | * tc_heat_external_http_ixload_1b_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_heat_external_http_ixload_1b_Requests-65000 |
+| | _Connections.yaml |
+| | |
+| | The 4 ports RFC3511 test cases are listed below: |
+| | |
+| | * tc_baremetal_http_ixload_1b_Requests-65000 |
+| | _Concurrency_4port.yaml |
+| | |
++---------------+--------------------------------------------------------------+
+| test tool | The vFW is a DPDK application that performs basic filtering |
+| | for malformed packets and dynamic packet filtering of |
+| | incoming packets using the connection tracker library. |
+| | |
++---------------+--------------------------------------------------------------+
+| applicability | The vFW RFC3511 test cases can be configured with different: |
+| | |
+| | * http payload sizes; |
+| | * traffic flows; |
+| | * rules; |
+| | |
+| | Default values exist. |
+| | |
++---------------+--------------------------------------------------------------+
+| pre-test | For OpenStack test case image (yardstick-samplevnf) needs |
+| conditions | to be installed into Glance with vFW and DPDK included in |
+| | it (NSB install). |
+| | |
+| | For Baremetal tests cases vFW and DPDK must be installed on |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information. |
+| | |
++---------------+--------------------------------------------------------------+
+| test sequence | Description and expected result |
+| | |
++---------------+--------------------------------------------------------------+
+| step 1 | For Baremetal test: The vFW VNF is started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat external test: The vFW VM are deployed and booted. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 2 | Yardstick is connected with the TG (IxLoad) via IxLoad API |
+| | and VNF by using ssh. The test will resolve the topology and |
+| | instantiate all VNFs and TG and collect the KPI's/metrics. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 3 | The TG simulates HTTP traffic based on selected type of TC. |
+| | |
+| | Concurrency: |
+| | The TC attempts to simulate some number of human users. |
+| | The simulated users are gradually brought online until 64K |
+| | users is met (the Ramp-Up phase), then taken offline (the |
+| | Ramp Down phase). |
+| | |
+| | Connections: |
+| | The TC creates some number of HTTP connections per second. |
+| | It will attempt to generate the 64K of HTTP connections |
+| | per second. |
+| | |
+| | Throughput: |
+| | TC simultaneously transmits and receives TCP payload |
+| | (bytes) at a certain rate measured in Megabits per second |
+| | (Mbps), Kilobits per second (Kbps), or Gigabits per |
+| | second. The 10 Gbits is default throughput. |
+| | |
+| | At the end of the TC, the KPIs are collected and stored |
+| | (depends on the selected dispatcher). |
+| | |
++---------------+--------------------------------------------------------------+
+| step 4 | In Baremetal test: The test quits the application and |
+| | unbinds the DPDK ports. |
+| | |
+| | In Heat test: All VNF VMs are deleted and connections to TG |
+| | are terminated. |
+| | |
++---------------+--------------------------------------------------------------+
+| test verdict | The test case will try to achieve the configured HTTP |
+| | Concurrency/Throughput/Connections. |
++---------------+--------------------------------------------------------------+
+
Code Review / yardstick.git / commitdiff