1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, 2018 Intel Corporation.
6 ************************************************
7 Yardstick Test Case Description: NSB vFW RFC2544
8 ************************************************
10 +------------------------------------------------------------------------------+
11 | NSB vFW test for VNF characterization |
13 +---------------+--------------------------------------------------------------+
14 | test case id | tc_{context}_rfc2544_ipv4_1rule_1flow_{pkt_size}_{tg_type} |
16 | | * context = baremetal, heat, heat_external, ovs, sriov |
17 | | heat_sriov_external contexts; |
18 | | * tg_type = ixia (context != heat,heat_sriov_external), |
20 | | * pkt_size = 64B - all contexts; |
21 | | 128B, 256B, 512B, 1024B, 1280B, 1518B - |
22 | | (context = heat, tg_type = ixia) |
24 +---------------+--------------------------------------------------------------+
25 | metric | * Network Throughput; |
26 | | * TG Packets Out; |
27 | | * TG Packets In; |
29 | | * VNF Packets Out; |
30 | | * VNF Packets In; |
31 | | * VNF Packets Fwd; |
32 | | * Dropped packets; |
34 +---------------+--------------------------------------------------------------+
35 | test purpose | The VFW RFC2544 tests measure performance characteristics of |
36 | | the SUT (multiple ports) and sends UDP bidirectional traffic |
37 | | from all TG ports to SampleVNF vFW application. The |
38 | | application forwards received traffic based on rules |
39 | | provided by the user in the TC configuration and default |
40 | | rules created by vFW to send traffic from uplink ports to |
41 | | downlink and voice versa. |
43 +---------------+--------------------------------------------------------------+
44 | configuration | The 2 ports RFC2544 test cases are listed below: |
46 | | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
47 | | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
48 | | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1024B_ixia.yaml |
49 | | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1280B_ixia.yaml |
50 | | * tc_heat_external_rfc2544_ipv4_1rule_1flow_128B_ixia.yaml |
51 | | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1518B_ixia.yaml |
52 | | * tc_heat_external_rfc2544_ipv4_1rule_1flow_256B_ixia.yaml |
53 | | * tc_heat_external_rfc2544_ipv4_1rule_1flow_512B_ixia.yaml |
54 | | * tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
55 | | * tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
56 | | * tc_heat_sriov_external_rfc2544_ipv4_1rule_1flow_64B_trex. |
58 | | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
59 | | * tc_ovs_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
60 | | * tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
61 | | * tc_sriov_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
62 | | * tc_sriov_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
64 | | The 4 ports RFC2544 test cases are listed below: |
66 | | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia_4port.yaml |
67 | | * tc_tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex_4port. |
69 | | * tc_tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_trex_4 |
71 | | * tc_tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_4port.yaml |
73 | | The scale-up RFC2544 test cases are listed below: |
75 | | * tc_tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml |
77 | | The scale-out RFC2544 test cases are listed below: |
79 | | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale_out.yaml |
81 | | Test duration is set as 30 sec for each test and default |
82 | | number of rules are applied. These can be configured |
84 +---------------+--------------------------------------------------------------+
85 | test tool | The vFW is a DPDK application that performs basic filtering |
86 | | for malformed packets and dynamic packet filtering of |
87 | | incoming packets using the connection tracker library. |
89 +---------------+--------------------------------------------------------------+
90 | applicability | The vFW RFC2544 test cases can be configured with different: |
93 | | * test duration; |
94 | | * tolerated loss; |
95 | | * traffic flows; |
98 | | Default values exist. |
100 +---------------+--------------------------------------------------------------+
101 | pre-test | For OpenStack test case image (yardstick-samplevnf) needs |
102 | conditions | to be installed into Glance with vFW and DPDK included in |
103 | | it (NSB install). |
105 | | For Baremetal tests cases vFW and DPDK must be installed on |
106 | | the hosts where the test is executed. The pod.yaml file must |
107 | | have the necessary system and NIC information. |
109 | | For standalone (SA) SRIOV/OvS test cases the |
110 | | yardstick-samplevnf image needs to be installed on hosts and |
111 | | pod.yaml file must be provided with necessary system, NIC |
114 +---------------+--------------------------------------------------------------+
115 | test sequence | Description and expected result |
117 +---------------+--------------------------------------------------------------+
118 | step 1 | For Baremetal test: The TG (except IXIA) and VNF are started |
119 | | on the hosts based on the pod file. |
121 | | For Heat test: Two host VMs are booted, as Traffic generator |
122 | | and VNF(vFW) based on the test flavor. In case of scale-out |
123 | | scenario the multiple VNF VMs will be started. |
125 | | For Heat external test: vFW VM is booted and TG (except IXIA)|
126 | | generator is started on the external host based on the pod |
127 | | file. In case of scale-out scenario the multiple VNF VMs |
128 | | will be deployed. |
130 | | For Heat SRIOV external test: vFW VM is booted with network |
131 | | interfaces of `direct` type which are mapped to VFs that are |
132 | | available to OpenStack. TG (except IXIA) is started on the |
133 | | external host based on the pod file. In case of scale-out |
134 | | scenario the multiple VNF VMs will be deployed. |
136 | | For SRIOV test: VF ports are created on host's PFs specified |
137 | | in the TC file and VM is booed using those ports and image |
138 | | provided in the configuration. TG (except IXIA) is started |
139 | | on other host connected to VNF machine based on the pod |
140 | | file. The vFW is started in the booted VM. In case of |
141 | | scale-out scenario the multiple VNF VMs will be created. |
143 | | For OvS-DPDK test: OvS DPDK switch is started and bridges |
144 | | are created with ports specified in the TC file. DPDK vHost |
145 | | ports are added to corresponding bridge and VM is booed |
146 | | using those ports and image provided in the configuration. |
147 | | TG (except IXIA) is started on other host connected to VNF |
148 | | machine based on the pod file. The vFW is started in the |
149 | | booted VM. In case of scale-out scenario the multiple VNF |
150 | | VMs will be deployed. |
152 +---------------+--------------------------------------------------------------+
153 | step 2 | Yardstick is connected with the TG and VNF by using ssh (in |
154 | | case of IXIA TG is connected via TCL interface). The test |
155 | | will resolve the topology and instantiate all VNFs |
156 | | and TG and collect the KPI's/metrics. |
158 +---------------+--------------------------------------------------------------+
159 | step 3 | The TG will send packets to the VNFs. If the number of |
160 | | dropped packets is more than the tolerated loss the line |
161 | | rate or throughput is halved. This is done until the dropped |
162 | | packets are within an acceptable tolerated loss. |
164 | | The KPI is the number of packets per second for different |
165 | | packet size with an accepted minimal packet loss for the |
166 | | default configuration. |
168 +---------------+--------------------------------------------------------------+
169 | step 4 | In Baremetal test: The test quits the application and unbind |
170 | | the DPDK ports. |
172 | | In Heat test: All VNF VMs and TG are deleted on test |
175 | | In SRIOV test: The deployed VM with vFW is destroyed on the |
176 | | host and TG (exclude IXIA) is stopped. |
178 | | In Heat SRIOV test: The deployed VM with vFW is destroyed, |
179 | | VFs are released and TG (exclude IXIA) is stopped. |
181 | | In OvS test: The deployed VM with vFW is destroyed on the |
182 | | host and OvS DPDK switch is stopped and ports are unbinded. |
183 | | The TG (exclude IXIA) is stopped. |
185 +---------------+--------------------------------------------------------------+
186 | test verdict | The test case will achieve a Throughput with an accepted |
187 | | minimal tolerated packet loss. |
188 +---------------+--------------------------------------------------------------+