docs/testing/user/userguide/opnfv_yardstick_tc087.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International
   2 .. License.
   3 .. http://creativecommons.org/licenses/by/4.0
   4 .. (c) OPNFV, Ericsson and others.
   5
   6 *************************************
   7 Yardstick Test Case Description TC087
   8 *************************************
   9
  10 +-----------------------------------------------------------------------------+
  11 |SDN Controller resilience in non-HA configuration                            |
  12 |                                                                             |
  13 +--------------+--------------------------------------------------------------+
  14 |test case id  | OPNFV_YARDSTICK_TC087: SDN controller resilience in          |
  15 |              | non-HA configuration                                         |
  16 |              |                                                              |
  17 +--------------+--------------------------------------------------------------+
  18 |test purpose  | This test validates that network data plane services are     |
  19 |              | highly available in the event of an SDN Controller failure,  |
  20 |              | even if the SDN controller is deployed in a non-HA           |
  21 |              | configuration. Specifically, the test verifies that          |
  22 |              | existing data plane connectivity is not impacted, i.e. all   |
  23 |              | configured network services such as DHCP, ARP, L2,           |
  24 |              | L3 Security Groups should continue to operate                |
  25 |              | between the existing VMs while the SDN controller is         |
  26 |              | offline or rebooting.                                        |
  27 |              |                                                              |
  28 |              | The test also validates that new network service operations  |
  29 |              | (creating a new VM in the existing L2/L3 network or in a new |
  30 |              | network, etc.) are operational after the SDN controller      |
  31 |              | has recovered from a failure.                                |
  32 |              |                                                              |
  33 +--------------+--------------------------------------------------------------+
  34 |test method   | This test case fails the SDN controller service running      |
  35 |              | on the OpenStack controller node, then checks if already     |
  36 |              | configured DHCP/ARP/L2/L3/SNAT connectivity is not           |
  37 |              | impacted between VMs and the system is able to execute       |
  38 |              | new virtual network operations once the SDN controller       |
  39 |              | is restarted and has fully recovered                         |
  40 |              |                                                              |
  41 +--------------+--------------------------------------------------------------+
  42 |attackers     | In this test case, an attacker called “kill-process” is      |
  43 |              | needed. This attacker includes three parameters:             |
  44 |              |                                                              |
  45 |              |  1. fault_type: which is used for finding the attacker's     |
  46 |              |     scripts. It should be set to 'kill-process' in this test |
  47 |              |                                                              |
  48 |              |  2. process_name: should be set to the name of the SDN       |
  49 |              |     controller process                                       |
  50 |              |                                                              |
  51 |              |  3. host: which is the name of a control node where the      |
  52 |              |     SDN controller process is running                        |
  53 |              |                                                              |
  54 |              | e.g. -fault_type: "kill-process"                             |
  55 |              |      -process_name: "opendaylight"                           |
  56 |              |      -host: node1                                            |
  57 |              |                                                              |
  58 +--------------+--------------------------------------------------------------+
  59 |monitors      | This test case utilizes two monitors of type "ip-status"     |
  60 |              | and one monitor of type "process" to track the following     |
  61 |              | conditions:                                                  |
  62 |              |                                                              |
  63 |              |  1. "ping_same_network_l2": monitor ICMP traffic between     |
  64 |              |     VMs in the same Neutron network                          |
  65 |              |                                                              |
  66 |              |  2. "ping_external_snat": monitor ICMP traffic from VMs to   |
  67 |              |     an external host on the Internet to verify SNAT          |
  68 |              |     functionality.                                           |
  69 |              |                                                              |
  70 |              |  3. "SDN controller process monitor": a monitor checking the |
  71 |              |     state of a specified SDN controller process. It measures |
  72 |              |     the recovery time of the given process.                  |
  73 |              |                                                              |
  74 |              | Monitors of type "ip-status" use the "ping" utility to       |
  75 |              | verify reachability of a given target IP.                    |
  76 |              |                                                              |
  77 +--------------+--------------------------------------------------------------+
  78 |operations    | In this test case, the following operations are needed:      |
  79 |              |                                                              |
  80 |              |  1. "nova-create-instance-in_network": create a VM instance  |
  81 |              |     in one of the existing Neutron network.                  |
  82 |              |                                                              |
  83 +--------------+--------------------------------------------------------------+
  84 |metrics       | In this test case, there are two metrics:                    |
  85 |              |                                                              |
  86 |              |  1. process_recover_time: which indicates the maximun        |
  87 |              |     time (seconds) from the process being killed to          |
  88 |              |     recovered                                                |
  89 |              |                                                              |
  90 |              |  2. packet_drop: measure the packets that have been dropped  |
  91 |              |     by the monitors using pktgen.                            |
  92 |              |                                                              |
  93 +--------------+--------------------------------------------------------------+
  94 |test tool     | Developed by the project. Please see folder:                 |
  95 |              | "yardstick/benchmark/scenarios/availability/ha_tools"        |
  96 |              |                                                              |
  97 +--------------+--------------------------------------------------------------+
  98 |references    | none                                                         |
  99 |              |                                                              |
 100 +--------------+--------------------------------------------------------------+
 101 |configuration | This test case needs two configuration files:                |
 102 |              |                                                              |
 103 |              |  1. test case file: opnfv_yardstick_tc087.yaml               |
 104 |              |                                                              |
 105 |              |     - Attackers: see above “attackers” discription           |
 106 |              |     - waiting_time: which is the time (seconds) from the     |
 107 |              |       process being killed to stoping monitors the monitors  |
 108 |              |     - Monitors: see above “monitors” discription             |
 109 |              |     - SLA: see above “metrics” discription                   |
 110 |              |                                                              |
 111 |              |  2. POD file: pod.yaml The POD configuration should record   |
 112 |              |     on pod.yaml first. the “host” item in this test case     |
 113 |              |     will use the node name in the pod.yaml.                  |
 114 |              |                                                              |
 115 +--------------+--------------------------------------------------------------+
 116 |test sequence | Description and expected result                              |
 117 |              |                                                              |
 118 +--------------+--------------------------------------------------------------+
 119 |pre-action    |  1. The OpenStack cluster is set up with a single SDN        |
 120 |              |     controller in a non-HA configuration.                    |
 121 |              |                                                              |
 122 |              |  2. One or more Neutron networks are created with two or     |
 123 |              |     more VMs attached to each of the Neutron networks.       |
 124 |              |                                                              |
 125 |              |  3. The Neutron networks are attached to a Neutron router    |
 126 |              |     which is attached to an external network towards the     |
 127 |              |     DCGW.                                                    |
 128 |              |                                                              |
 129 +--------------+--------------------------------------------------------------+
 130 |step 1        | Start IP connectivity monitors:                              |
 131 |              |  1. Check the L2 connectivity between the VMs in the same    |
 132 |              |     Neutron network.                                         |
 133 |              |                                                              |
 134 |              |  2. Check connectivity from one VM to an external host on    |
 135 |              |     the Internet to verify SNAT functionality.               |
 136 |              |                                                              |
 137 |              | Result: The monitor info will be collected.                  |
 138 |              |                                                              |
 139 +--------------+--------------------------------------------------------------+
 140 |step 2        | Start attacker:                                              |
 141 |              | SSH connect to the VIM node and kill the SDN controller      |
 142 |              | process                                                      |
 143 |              |                                                              |
 144 |              | Result: the SDN controller service will be shutdown          |
 145 |              |                                                              |
 146 +--------------+--------------------------------------------------------------+
 147 |step 3        | Verify the results of the IP connectivity monitors.          |
 148 |              |                                                              |
 149 |              | Result: The outage_time metric reported by the monitors      |
 150 |              | is zero.                                                     |
 151 |              |                                                              |
 152 +--------------+--------------------------------------------------------------+
 153 |step 4        | Restart the SDN controller.                                  |
 154 |              |                                                              |
 155 +--------------+--------------------------------------------------------------+
 156 |step 5        | Create a new VM in the existing Neutron network              |
 157 |              |                                                              |
 158 +--------------+--------------------------------------------------------------+
 159 |step 6        | Verify connectivity between VMs as follows:                  |
 160 |              |  1. Check the L2 connectivity between the previously         |
 161 |              |     existing VM and the newly created VM on the same         |
 162 |              |     Neutron network by sending ICMP messages                 |
 163 |              |                                                              |
 164 +--------------+--------------------------------------------------------------+
 165 |step 7        | Stop IP connectivity monitors after a period of time         |
 166 |              | specified by “waiting_time”                                  |
 167 |              |                                                              |
 168 |              | Result: The monitor info will be aggregated                  |
 169 |              |                                                              |
 170 +--------------+--------------------------------------------------------------+
 171 |step 8        | Verify the IP connectivity monitor results                   |
 172 |              |                                                              |
 173 |              | Result: IP connectivity monitor should not have any packet   |
 174 |              | drop failures reported                                       |
 175 |              |                                                              |
 176 +--------------+--------------------------------------------------------------+
 177 |test verdict  | This test fails if the SLAs are not met or if there is a     |
 178 |              | test case execution problem. The SLAs are define as follows  |
 179 |              | for this test:                                               |
 180 |              |                                                              |
 181 |              |  * SDN Controller recovery                                   |
 182 |              |                                                              |
 183 |              |    * process_recover_time <= 30 sec                          |
 184 |              |                                                              |
 185 |              |  * no impact on data plane connectivity during SDN           |
 186 |              |    controller failure and recovery.                          |
 187 |              |                                                              |
 188 |              |    * packet_drop == 0                                        |
 189 |              |                                                              |
 190 +--------------+--------------------------------------------------------------+
 191