12-nsb_installation
13-nsb_operation
15-list-of-tcs
+ nsb/nsb-list-of-tcs
glossary
references
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+=====================
+NSB Sample Test Cases
+=====================
+
+Abstract
+========
+
+This chapter lists available NSB test cases.
+
+NSB PROX Test Case Descriptions
+===============================
+
+.. toctree::
+ :maxdepth: 1
+
+ tc_prox_context_acl_port
+ tc_prox_context_bng_port
+ tc_prox_context_bng_qos_port
+ tc_prox_context_l2fwd_port
+ tc_prox_context_l3fwd_port
+ tc_prox_context_mpls_tagging_port
+ tc_prox_context_buffering_port
+ tc_prox_context_load_balancer_port
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+*********************************************
+Yardstick Test Case Description: NSB PROX ACL
+*********************************************
+
++-----------------------------------------------------------------------------+
+|NSB PROX test for NFVI characterization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_prox_{context}_acl-{port_num} |
+| | |
+| | * context = baremetal or heat_context; |
+| | * port_num = 2 or 4; |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * Dropped packets; |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | This test allows to measure how well the SUT can exploit |
+| | structures in the list of ACL rules. |
+| | The ACL rules are matched against a 7-tuple of the input |
+| | packet: the regular 5-tuple and two VLAN tags. |
+| | The rules in the rule set allow the packet to be forwarded |
+| | and the rule set contains a default "match all" rule. |
+| | |
+| | The KPI is measured with the rule set that has a moderate |
+| | number of rules with moderate similarity between the rules & |
+| | the fraction of rules that were used. |
+| | |
+| | The ACL test cases are implemented to run in baremetal |
+| | and heat context for 2 port and 4 port configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The ACL test cases are listed below: |
+| | |
+| | * tc_prox_baremetal_acl-2.yaml |
+| | * tc_prox_baremetal_acl-4.yaml |
+| | * tc_prox_heat_context_acl-2.yaml |
+| | * tc_prox_heat_context_acl-4.yaml |
+| | |
+| | Test duration is set as 300sec for each test. |
+| | Packet size set as 64 bytes in traffic profile. |
+| | These can be configured |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | PROX |
+| | PROX is a DPDK application that can simulate VNF workloads |
+| | and can generate traffic and used for NFVI characterization |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | This PROX ACL test cases can be configured with different: |
+| | |
+| | * packet sizes; |
+| | * test durations; |
+| | * tolerated loss; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | For Openstack test case image (yardstick-samplevnfs) needs |
+|conditions | to be installed into Glance with Prox and Dpdk included in |
+| | it. The test need multi-queue enabled in Glance image. |
+| | |
+| | For Baremetal tests cases Prox and Dpdk must be installed in |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | For Baremetal test: The TG and VNF are started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(ACL workload) based on the test flavor. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the TG and VNF by using ssh. |
+| | The test will resolve the topology and instantiate the VNF |
+| | and TG and collect the KPI's/metrics. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The TG will send packets to the VNF. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 64 bytes |
+| | packet size with an accepted minimal packet loss for the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | In Baremetal test: The test quits the application and unbind |
+| | the dpdk ports. |
+| | |
+| | In Heat test: Two host VMs are deleted on test completion. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++--------------+--------------------------------------------------------------+
+
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+*********************************************
+Yardstick Test Case Description: NSB PROX BNG
+*********************************************
+
++-----------------------------------------------------------------------------+
+|NSB PROX test for NFVI characterization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_prox_{context}_bng-{port_num} |
+| | |
+| | * context = baremetal or heat_context; |
+| | * port_num = 4; |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * Dropped packets; |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The BNG workload converts packets from QinQ to GRE tunnels, |
+| | handles routing and adds/removes MPLS tags. |
+| | This use case simulates a realistic and complex application. |
+| | The number of users is 32K per port and the number of routes |
+| | is 8K. |
+| | |
+| | The BNG test cases are implemented to run in baremetal |
+| | and heat context an require 4 port topology to run the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The BNG test cases are listed below: |
+| | |
+| | * tc_prox_baremetal_bng-2.yaml |
+| | * tc_prox_baremetal_bng-4.yaml |
+| | * tc_prox_heat_context_bng-2.yaml |
+| | * tc_prox_heat_context_bng-4.yaml |
+| | |
+| | Test duration is set as 300sec for each test. |
+| | The minimum packet size for BNG test is 78 bytes. This is |
+| | set in the BNG traffic profile and can be configured to use |
+| | a higher packet size for the test. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | PROX |
+| | PROX is a DPDK application that can simulate VNF workloads |
+| | and can generate traffic and used for NFVI characterization |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | The PROX BNG test cases can be configured with different: |
+| | |
+| | * packet sizes; |
+| | * test durations; |
+| | * tolerated loss; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | For Openstack test case image (yardstick-samplevnfs) needs |
+|conditions | to be installed into Glance with Prox and Dpdk included in |
+| | it. The test need multi-queue enabled in Glance image. |
+| | |
+| | For Baremetal tests cases Prox and Dpdk must be installed in |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | For Baremetal test: The TG and VNF are started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(BNG workload) based on the test flavor. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the TG and VNF by using ssh. |
+| | The test will resolve the topology and instantiate the VNF |
+| | and TG and collect the KPI's/metrics. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The TG will send packets to the VNF. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 78 bytes |
+| | packet size with an accepted minimal packet loss for the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | In Baremetal test: The test quits the application and unbind |
+| | the dpdk ports. |
+| | |
+| | In Heat test: Two host VMs are deleted on test completion. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++--------------+--------------------------------------------------------------+
+
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+*************************************************
+Yardstick Test Case Description: NSB PROX BNG_QoS
+*************************************************
+
++-----------------------------------------------------------------------------+
+|NSB PROX test for NFVI characterization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_prox_{context}_bng_qos-{port_num} |
+| | |
+| | * context = baremetal or heat_context; |
+| | * port_num = 4; |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * Dropped packets; |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The BNG+QoS workload converts packets from QinQ to GRE |
+| | tunnels, handles routing and adds/removes MPLS tags and |
+| | performs a QoS. |
+| | This use case simulates a realistic and complex application. |
+| | The number of users is 32K per port and the number of routes |
+| | is 8K. |
+| | |
+| | The BNG_QoS test cases are implemented to run in baremetal |
+| | and heat context an require 4 port topology to run the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The BNG_QoS test cases are listed below: |
+| | |
+| | * tc_prox_baremetal_bng_qos-2.yaml |
+| | * tc_prox_baremetal_bng_qos-4.yaml |
+| | * tc_prox_heat_context_bng_qos-2.yaml |
+| | * tc_prox_heat_context_bng_qos-4.yaml |
+| | |
+| | Test duration is set as 300sec for each test. |
+| | The minumum packet size for BNG_QoS test is 78 bytes. This |
+| | is set in the bng_qos traffic profile and can be configured |
+| | to use a higher packet size for the test. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | PROX |
+| | PROX is a DPDK application that can simulate VNF workloads |
+| | and can generate traffic and used for NFVI characterization |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | This PROX BNG_QoS test cases can be configured with |
+| | different: |
+| | |
+| | * packet sizes; |
+| | * test durations; |
+| | * tolerated loss; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | For Openstack test case image (yardstick-samplevnfs) needs |
+|conditions | to be installed into Glance with Prox and Dpdk included in |
+| | it. The test need multi-queue enabled in Glance image. |
+| | |
+| | For Baremetal tests cases Prox and Dpdk must be installed in |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | For Baremetal test: The TG and VNF are started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(BNG_QoS workload) based on the test flavor. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the TG and VNF by using ssh. |
+| | The test will resolve the topology and instantiate the VNF |
+| | and TG and collect the KPI's/metrics. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The TG will send packets to the VNF. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 78 bytes |
+| | packet size with an accepted minimal packet loss for the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | In Baremetal test: The test quits the application and unbind |
+| | the dpdk ports. |
+| | |
+| | In Heat test: Two host VMs are deleted on test completion. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++--------------+--------------------------------------------------------------+
+
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+**********************************************************
+Yardstick Test Case Description: NSB PROX Packet Buffering
+**********************************************************
+
++-----------------------------------------------------------------------------+
+|NSB PROX test for NFVI characterization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_prox_{context}_buffering-{port_num} |
+| | |
+| | * context = baremetal or heat_context |
+| | * port_num = 1 |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * Dropped packets; |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | This test measures the impact of the condition when packets |
+| | get buffered, thus they stay in memory for the extended |
+| | period of time, 125ms in this case. |
+| | |
+| | The Packet Buffering test cases are implemented to run in |
+| | baremetal and heat context. |
+| | |
+| | The test runs only on the first port of the SUT. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The Packet Buffering test cases are listed below: |
+| | |
+| | * tc_prox_baremetal_buffering-1.yaml |
+| | * tc_prox_heat_context_buffering-1.yaml |
+| | |
+| | Test duration is set as 300sec for each test. |
+| | The minimum packet size for Buffering test is 64 bytes. This |
+| | is set in the traffic profile and can be configured to use |
+| | a higher packet size for the test. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | PROX |
+| | PROX is a DPDK application that can simulate VNF workloads |
+| | and can generate traffic and used for NFVI characterization |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | The PROX Packet Buffering test cases can be configured with |
+| | different: |
+| | |
+| | * packet sizes; |
+| | * test durations; |
+| | * tolerated loss; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | For Openstack test case image (yardstick-samplevnfs) needs |
+|conditions | to be installed into Glance with Prox and Dpdk included in |
+| | it. The test need multi-queue enabled in Glance image. |
+| | |
+| | For Baremetal tests cases Prox and Dpdk must be installed in |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | For Baremetal test: The TG and VNF are started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(Packet Buffering workload) based on the test flavor. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the TG and VNF by using ssh. |
+| | The test will resolve the topology and instantiate the VNF |
+| | and TG and collect the KPI's/metrics. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The TG will send packets to the VNF. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI in this test is the maximum number of packets that |
+| | can be forwarded given the requirement that the latency of |
+| | each packet is at least 125 millisecond. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | In Baremetal test: The test quits the application and unbind |
+| | the dpdk ports. |
+| | |
+| | In Heat test: Two host VMs are deleted on test completion. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++--------------+--------------------------------------------------------------+
+
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+***********************************************
+Yardstick Test Case Description: NSB PROX L2FWD
+***********************************************
+
++-----------------------------------------------------------------------------+
+|NSB PROX test for NFVI characterization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_prox_{context}_l2fwd-{port_num} |
+| | |
+| | * context = baremetal or heat_context; |
+| | * port_num = 2 or 4; |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * Dropped packets; |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The PROX L2FWD test has 3 types of test cases: |
+| | L2FWD: The application will take packets in from one port |
+| | and forward them unmodified to another port |
+| | L2FWD_Packet_Touch: The application will take packets in |
+| | from one port, update src and dst MACs and forward them to |
+| | another port. |
+| | L2FWD_Multi_Flow: The application will take packets in |
+| | from one port, update src and dst MACs and forward them to |
+| | another port. This test case exercises the softswitch |
+| | with 200k flows. |
+| | |
+| | The above test cases are implemented for baremetal and heat |
+| | context for 2 port and 4 port configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The L2FWD test cases are listed below: |
+| | |
+| | * tc_prox_baremetal_l2fwd-2.yaml |
+| | * tc_prox_baremetal_l2fwd-4.yaml |
+| | * tc_prox_baremetal_l2fwd_pktTouch-2.yaml |
+| | * tc_prox_baremetal_l2fwd_pktTouch-4.yaml |
+| | * tc_prox_baremetal_l2fwd_multiflow-2.yaml |
+| | * tc_prox_baremetal_l2fwd_multiflow-4.yaml |
+| | * tc_prox_heat_context_l2fwd-2.yaml |
+| | * tc_prox_heat_context_l2fwd-4.yaml |
+| | * tc_prox_heat_context_l2fwd_pktTouch-2.yaml |
+| | * tc_prox_heat_context_l2fwd_pktTouch-4.yaml |
+| | * tc_prox_heat_context_l2fwd_multiflow-2.yaml |
+| | * tc_prox_heat_context_l2fwd_multiflow-4.yaml |
+| | |
+| | Test duration is set as 300sec for each test. |
+| | Packet size set as 64 bytes in traffic profile |
+| | These can be configured |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | PROX |
+| | PROX is a DPDK application that can simulate VNF workloads |
+| | and can generate traffic and used for NFVI characterization |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | The PROX L2FWD test cases can be configured with different: |
+| | |
+| | * packet sizes; |
+| | * test durations; |
+| | * tolerated loss; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | For Openstack test case image (yardstick-samplevnfs) needs |
+|conditions | to be installed into Glance with Prox and Dpdk included in |
+| | it. |
+| | |
+| | For Baremetal tests cases Prox and Dpdk must be installed in |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | For Baremetal test: The TG and VNF are started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(L2FWD workload) based on the test flavor. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the TG and VNF by using ssh. |
+| | The test will resolve the topology and instantiate the VNF |
+| | and TG and collect the KPI's/metrics. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The TG will send packets to the VNF. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 64 bytes |
+| | packet size with an accepted minimal packet loss for the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | In Baremetal test: The test quits the application and unbind |
+| | the dpdk ports. |
+| | |
+| | In Heat test: Two host VMs are deleted on test completion. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++--------------+--------------------------------------------------------------+
+
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+***********************************************
+Yardstick Test Case Description: NSB PROX L3FWD
+***********************************************
+
++-----------------------------------------------------------------------------+
+|NSB PROX test for NFVI characterization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_prox_{context}_l3fwd-{port_num} |
+| | |
+| | * context = baremetal or heat_context; |
+| | * port_num = 2 or 4; |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * Dropped packets; |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The PROX L3FWD application performs basic routing of packets |
+| | with LPM based look-up method. |
+| | |
+| | The L3FWD test cases are implemented for baremetal and heat |
+| | context for 2 port and 4 port configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The L3FWD test cases are listed below: |
+| | |
+| | * tc_prox_baremetal_l3fwd-2.yaml |
+| | * tc_prox_baremetal_l3fwd-4.yaml |
+| | * tc_prox_heat_context_l3fwd-2.yaml |
+| | * tc_prox_heat_context_l3fwd-4.yaml |
+| | |
+| | Test duration is set as 300sec for each test. |
+| | The minimum packet size for L3FWD test is 64 bytes. This is |
+| | set in the traffic profile and can be configured to use |
+| | a higher packet size for the test. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | PROX |
+| | PROX is a DPDK application that can simulate VNF workloads |
+| | and can generate traffic and used for NFVI characterization |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | This PROX L3FWD test cases can be configured with different: |
+| | |
+| | * packet sizes; |
+| | * test durations; |
+| | * tolerated loss; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | For Openstack test case image (yardstick-samplevnfs) needs |
+|conditions | to be installed into Glance with Prox and Dpdk included in |
+| | it. The test need multi-queue enabled in Glance image. |
+| | |
+| | For Baremetal tests cases Prox and Dpdk must be installed in |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | For Baremetal test: The TG and VNF are started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(L3FWD workload) based on the test flavor. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the TG and VNF by using ssh. |
+| | The test will resolve the topology and instantiate the VNF |
+| | and TG and collect the KPI's/metrics. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The TG will send packet to the VNF. If the number of dropped |
+| | packets is more than the tolerated loss the line rate |
+| | or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 64 byte |
+| | packets with an accepted minimal packet loss for the default |
+| | configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | In Baremetal test: The test quits the application and unbind |
+| | the dpdk ports. |
+| | |
+| | In Heat test: Two host VMs are deleted on test completion. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++--------------+--------------------------------------------------------------+
+
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+*******************************************************
+Yardstick Test Case Description: NSB PROX Load Balancer
+*******************************************************
+
++-----------------------------------------------------------------------------+
+|NSB PROX test for NFVI characterization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_prox_{context}_lb-{port_num} |
+| | |
+| | * context = baremetal or heat_context |
+| | * port_num = 4 |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * Dropped packets; |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The applciation transmits packets on one port and revieves |
+| | them on 4 ports. |
+| | The conventional 5-tuple is used in this test as it requires |
+| | some extraction steps and allows defining enough distinct |
+| | values to find the performance limits. |
+| | |
+| | The load is increased (adding more ports if needed) while |
+| | packets are load balanced using a hash table of 8M entries |
+| | |
+| | The number of packets per second that can be forwarded |
+| | determines the KPI. The default packet size is 64 bytes. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The Load Balancer test cases are listed below: |
+| | |
+| | * tc_prox_baremetal_lb-4.yaml |
+| | * tc_prox_heat_context_lb-4.yaml |
+| | |
+| | Test duration is set as 300sec for each test. |
+| | Packet size set as 64 bytes in traffic profile. |
+| | These can be configured |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | PROX |
+| | PROX is a DPDK application that can simulate VNF workloads |
+| | and can generate traffic and used for NFVI characterization |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | The PROX Load Balancer test cases can be configured with |
+| | different: |
+| | |
+| | * packet sizes; |
+| | * test durations; |
+| | * tolerated loss; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | For Openstack test case image (yardstick-samplevnfs) needs |
+|conditions | to be installed into Glance with Prox and Dpdk included in |
+| | it. The test need multi-queue enabled in Glance image. |
+| | |
+| | For Baremetal tests cases Prox and Dpdk must be installed in |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | For Baremetal test: The TG and VNF are started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(Load Balancer workload) based on the test flavor. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the TG and VNF by using ssh. |
+| | The test will resolve the topology and instantiate the VNF |
+| | and TG and collect the KPI's/metrics. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The TG will send packets to the VNF. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 78 bytes |
+| | packet size with an accepted minimal packet loss for the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | In Baremetal test: The test quits the application and unbind |
+| | the dpdk ports. |
+| | |
+| | In Heat test: Two host VMs are deleted on test completion. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++--------------+--------------------------------------------------------------+
+
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+******************************************************
+Yardstick Test Case Description: NSB PROX MPLS Tagging
+******************************************************
+
++-----------------------------------------------------------------------------+
+|NSB PROX test for NFVI characterization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_prox_{context}_mpls_tagging-{port_num} |
+| | |
+| | * context = baremetal or heat_context; |
+| | * port_num = 2 or 4; |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * Dropped packets; |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The PROX MPLS Tagging test will take packets in from one |
+| | port add an MPLS tag and forward them to another port. |
+| | While forwarding packets in other direction MPLS tags will |
+| | be removed. |
+| | |
+| | The MPLS test cases are implemented to run in baremetal |
+| | and heat context an require 4 port topology to run the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The MPLS Tagging test cases are listed below: |
+| | |
+| | * tc_prox_baremetal_mpls_tagging-2.yaml |
+| | * tc_prox_baremetal_mpls_tagging-4.yaml |
+| | * tc_prox_heat_context_mpls_tagging-2.yaml |
+| | * tc_prox_heat_context_mpls_tagging-4.yaml |
+| | |
+| | Test duration is set as 300sec for each test. |
+| | The minimum packet size for MPLS test is 68 bytes. This is |
+| | set in the traffic profile and can be configured to use |
+| | higher packet sizes. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | PROX |
+| | PROX is a DPDK application that can simulate VNF workloads |
+| | and can generate traffic and used for NFVI characterization |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | The PROX MPLS Tagging test cases can be configured with |
+| | different: |
+| | |
+| | * packet sizes; |
+| | * test durations; |
+| | * tolerated loss; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | For Openstack test case image (yardstick-samplevnfs) needs |
+|conditions | to be installed into Glance with Prox and Dpdk included in |
+| | it. |
+| | |
+| | For Baremetal tests cases Prox and Dpdk must be installed in |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | For Baremetal test: The TG and VNF are started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(MPLS workload) based on the test flavor. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the TG and VNF by using ssh. |
+| | The test will resolve the topology and instantiate the VNF |
+| | and TG and collect the KPI's/metrics. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The TG will send packets to the VNF. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 68 bytes |
+| | packet size with an accepted minimal packet loss for the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | In Baremetal test: The test quits the application and unbind |
+| | the dpdk ports. |
+| | |
+| | In Heat test: Two host VMs are deleted on test completion. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++--------------+--------------------------------------------------------------+
+
Code Review / yardstick.git / commitdiff