docs/how-to-use/IntegrationTests.rst

   1 SNAPS OpenStack Integration Testing
   2 ===================================
   3
   4 These tests are ones designed to be run within their own dynamically created project along with a newly generated user
   5 account and generally require other OpenStack object creators.
   6
   7 The Test Classes
   8 ================
   9
  10 create_security_group_tests.py - CreateSecurityGroupTests
  11 ---------------------------------------------------------
  12
  13 +---------------------------------------+---------------+-----------------------------------------------------------+
  14 | Test Name                             | API Versions  | Description                                               |
  15 +=======================================+===============+===========================================================+
  16 | test_create_group_without_rules       | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  17 |                                       | Neutron 2     | security group without any rules                          |
  18 +---------------------------------------+---------------+-----------------------------------------------------------+
  19 | test_create_delete_group              | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class clean() method   |
  20 |                                       | Neutron 2     | will not raise an exception should the group be deleted by|
  21 |                                       |               | some other process                                        |
  22 +---------------------------------------+---------------+-----------------------------------------------------------+
  23 | test_create_group_with_one_simple_rule| Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  24 |                                       | Neutron 2     | security group with a single rule                         |
  25 +---------------------------------------+---------------+-----------------------------------------------------------+
  26 | test_create_group_with_several_rules  | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  27 |                                       | Neutron 2     | security group with several rules                         |
  28 +---------------------------------------+---------------+-----------------------------------------------------------+
  29 | test_add_rule                         | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#add_rule() method      |
  30 |                                       | Neutron 2     | properly creates and associates the new rule              |
  31 +---------------------------------------+---------------+-----------------------------------------------------------+
  32 | test_remove_rule_by_id                | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#remove_rule() method   |
  33 |                                       | Neutron 2     | properly deletes and disassociates the old rule via its ID|
  34 +---------------------------------------+---------------+-----------------------------------------------------------+
  35 | test_remove_rule_by_setting           | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#remove_rule() method   |
  36 |                                       | Neutron 2     | properly deletes and disassociates the old rule via its   |
  37 |                                       |               | setting object                                            |
  38 +---------------------------------------+---------------+-----------------------------------------------------------+
  39
  40 create_image_tests.py - CreateImageSuccessTests
  41 -----------------------------------------------
  42
  43 +---------------------------------------+---------------+-----------------------------------------------------------+
  44 | Test Name                             | Glance API    | Description                                               |
  45 +=======================================+===============+===========================================================+
  46 | test_create_image_clean_url           | 1             | Ensures the OpenStackImage class can create an image from |
  47 |                                       |               | a download URL location                                   |
  48 +---------------------------------------+---------------+-----------------------------------------------------------+
  49 | test_create_image_clean_file          | 1             | Ensures the OpenStackImage class can create an image from |
  50 |                                       |               | a locally sourced image file                              |
  51 +---------------------------------------+---------------+-----------------------------------------------------------+
  52 | test_create_delete_image              | 1             | Ensures the OpenStackImage.clean() method deletes an image|
  53 |                                       |               | and does not raise an exception on subsequent calls to the|
  54 |                                       |               | clean() method                                            |
  55 +---------------------------------------+---------------+-----------------------------------------------------------+
  56 | test_create_same_image                | 1             | Ensures the OpenStackImage.create() method does not create|
  57 |                                       |               | another image when one already exists with the same name  |
  58 +---------------------------------------+---------------+-----------------------------------------------------------+
  59
  60 create_image_tests.py - CreateImageNegativeTests
  61 ------------------------------------------------
  62
  63 +---------------------------------------+---------------+-----------------------------------------------------------+
  64 | Test Name                             | Glance API    | Description                                               |
  65 +=======================================+===============+===========================================================+
  66 | test_none_image_name                  | 1             | Ensures OpenStackImage.create() results in an Exception   |
  67 |                                       |               | being raised when the ImageSettings.name attribute has    |
  68 |                                       |               | not been set                                              |
  69 +---------------------------------------+---------------+-----------------------------------------------------------+
  70 | test_bad_image_url                    | 1             | Ensures OpenStackImage.create() results in an Exception   |
  71 |                                       |               | being raised when the download URL is invalid             |
  72 +---------------------------------------+---------------+-----------------------------------------------------------+
  73 | test_bad_image_file                   | 1             | Ensures OpenStackImage.create() results in an Exception   |
  74 |                                       |               | being raised when the image file does not exist           |
  75 +---------------------------------------+---------------+-----------------------------------------------------------+
  76 | test_none_proj_name                   | 1             | Ensures OpenStackImage.create() results in an Exception   |
  77 |                                       |               | being raised when the credentials project name is None    |
  78 +---------------------------------------+---------------+-----------------------------------------------------------+
  79 | test_none_auth_url                    | 1             | Ensures OpenStackImage.create() results in an Exception   |
  80 |                                       |               | being raised when the credentials URL is None             |
  81 +---------------------------------------+---------------+-----------------------------------------------------------+
  82 | test_none_password                    | 1             | Ensures OpenStackImage.create() results in an Exception   |
  83 |                                       |               | being raised when the credentials password is None        |
  84 +---------------------------------------+---------------+-----------------------------------------------------------+
  85 | test_none_user                        | 1             | Ensures OpenStackImage.create() results in an Exception   |
  86 |                                       |               | being raised when the credentials user is None            |
  87 +---------------------------------------+---------------+-----------------------------------------------------------+
  88
  89 create_keypairs_tests.py - CreateKeypairsTests
  90 ----------------------------------------------
  91
  92 +---------------------------------------+---------------+-----------------------------------------------------------+
  93 | Test Name                             | Nova API      | Description                                               |
  94 +=======================================+===============+===========================================================+
  95 | test_create_keypair_only              | 2             | Ensures that a keypair object can be created simply by    |
  96 |                                       |               | only configuring a name                                   |
  97 +---------------------------------------+---------------+-----------------------------------------------------------+
  98 | test_create_delete_keypair            | 2             | Ensures that a keypair object is deleted via              |
  99 |                                       |               | OpenStackKeypair.clean() and subsequent calls do not      |
 100 |                                       |               | result in exceptions                                      |
 101 +---------------------------------------+---------------+-----------------------------------------------------------+
 102 | test_create_keypair_save_pub_only     | 2             | Ensures that a keypair object can be created when the only|
 103 |                                       |               | the public key is cached to disk                          |
 104 +---------------------------------------+---------------+-----------------------------------------------------------+
 105 | test_create_keypair_save_both         | 2             | Ensures that a keypair object can be created when both the|
 106 |                                       |               | public and private keys are cached to disk                |
 107 +---------------------------------------+---------------+-----------------------------------------------------------+
 108 | test_create_keypair_from_file         | 2             | Ensures that a keypair object can be created with an      |
 109 |                                       |               | existing public key file                                  |
 110 +---------------------------------------+---------------+-----------------------------------------------------------+
 111
 112 create_network_tests.py - CreateNetworkSuccessTests
 113 ---------------------------------------------------
 114
 115 +---------------------------------------+---------------+-----------------------------------------------------------+
 116 | Test Name                             | Neutron API   | Description                                               |
 117 +=======================================+===============+===========================================================+
 118 | test_create_network_without_router    | 2             | Ensures that a network can be created via the             |
 119 |                                       |               | OpenStackNetwork class without any routers                |
 120 +---------------------------------------+---------------+-----------------------------------------------------------+
 121 | test_create_delete_network            | 2             | Ensures that a router can be deleted via the              |
 122 |                                       |               | OpenStackNetwork.clean() method                           |
 123 +---------------------------------------+---------------+-----------------------------------------------------------+
 124 | test_create_network_with_router       | 2             | Ensures that a network can be created via the             |
 125 |                                       |               | OpenStackNetwork class with a router                      |
 126 +---------------------------------------+---------------+-----------------------------------------------------------+
 127 | test_create_networks_same_name        | 2             | Ensures that the OpenStackNetwork.create() method will not|
 128 |                                       |               | create a network with the same name                       |
 129 +---------------------------------------+---------------+-----------------------------------------------------------+
 130
 131 create_router_tests.py - CreateRouterSuccessTests
 132 -------------------------------------------------
 133
 134 +---------------------------------------+---------------+-----------------------------------------------------------+
 135 | Test Name                             | Neutron API   | Description                                               |
 136 +=======================================+===============+===========================================================+
 137 | test_create_router_vanilla            | 2             | Ensures that a router can be created via the              |
 138 |                                       |               | OpenStackRouter class with minimal settings               |
 139 +---------------------------------------+---------------+-----------------------------------------------------------+
 140 | test_create_delete_router             | 2             | Ensures that a router can be deleted via the              |
 141 |                                       |               | OpenStackRouter.clean() method                            |
 142 +---------------------------------------+---------------+-----------------------------------------------------------+
 143 | test_create_router_admin_state_false  | 2             | Ensures that a router can created with                    |
 144 |                                       |               | admin_state_up = False                                    |
 145 +---------------------------------------+---------------+-----------------------------------------------------------+
 146 | test_create_router_admin_state_True   | 2             | Ensures that a router can created with                    |
 147 |                                       |               | admin_state_up = True                                     |
 148 +---------------------------------------+---------------+-----------------------------------------------------------+
 149 | test_create_router_private_network    | 2             | Ensures that a router port can be created against a       |
 150 |                                       |               | private network                                           |
 151 +---------------------------------------+---------------+-----------------------------------------------------------+
 152 | test_create_router_external_network   | 2             | Ensures that a router can be created that is connected to |
 153 |                                       |               | both external and private internal networks               |
 154 +---------------------------------------+---------------+-----------------------------------------------------------+
 155
 156 create_router_tests.py - CreateRouterNegativeTests
 157 --------------------------------------------------
 158
 159 +----------------------------------------+---------------+-----------------------------------------------------------+
 160 | Test Name                              | Neutron API   | Description                                               |
 161 +========================================+===============+===========================================================+
 162 | test_create_router_noname              | 2             | Ensures that an exception is raised when attempting to    |
 163 |                                        |               | create a router without a name                            |
 164 +----------------------------------------+---------------+-----------------------------------------------------------+
 165 | test_create_router_invalid_gateway_name| 2             | Ensures that an exception is raised when attempting to    |
 166 |                                        |               | create a router to an external network that does not exist|
 167 +----------------------------------------+---------------+-----------------------------------------------------------+
 168
 169 create_instance_tests.py - CreateInstanceSimpleTests
 170 ----------------------------------------------------
 171
 172 +---------------------------------------+---------------+-----------------------------------------------------------+
 173 | Test Name                             | API Versions  | Description                                               |
 174 +=======================================+===============+===========================================================+
 175 | test_create_delete_instance           | Nova 2        | Ensures that the OpenStackVmInstance.clean() method       |
 176 |                                       | Neutron 2     | deletes the instance                                      |
 177 +---------------------------------------+---------------+-----------------------------------------------------------+
 178
 179 create_instance_tests.py - SimpleHealthCheck
 180 --------------------------------------------
 181
 182 +---------------------------------------+---------------+-----------------------------------------------------------+
 183 | Test Name                             | API Versions  | Description                                               |
 184 +=======================================+===============+===========================================================+
 185 | test_check_vm_ip_dhcp                 | Nova 2        | Tests the creation of an OpenStack instance with a single |
 186 |                                       | Neutron 2     | port and it's assigned IP address                         |
 187 +---------------------------------------+---------------+-----------------------------------------------------------+
 188
 189 create_instance_tests.py - CreateInstanceSingleNetworkTests
 190 -----------------------------------------------------------
 191
 192 +---------------------------------------+---------------+-----------------------------------------------------------+
 193 | Test Name                             | API Versions  | Description                                               |
 194 +=======================================+===============+===========================================================+
 195 | test_single_port_static               | Nova 2        | Ensures that an instance with a single port/NIC with a    |
 196 |                                       | Neutron 2     | static IP can be created                                  |
 197 +---------------------------------------+---------------+-----------------------------------------------------------+
 198 | test_ssh_client_fip_before_active     | Nova 2        | Ensures that an instance can be reached over SSH when the |
 199 |                                       | Neutron 2     | floating IP is assigned prior to the VM becoming ACTIVE   |
 200 +---------------------------------------+---------------+-----------------------------------------------------------+
 201 | test_ssh_client_fip_after_active      | Nova 2        | Ensures that an instance can be reached over SSH when the |
 202 |                                       | Neutron 2     | floating IP is assigned after to the VM becoming ACTIVE   |
 203 +---------------------------------------+---------------+-----------------------------------------------------------+
 204
 205 create_instance_tests.py - CreateInstancePortManipulationTests
 206 --------------------------------------------------------------
 207
 208 +---------------------------------------+---------------+-----------------------------------------------------------+
 209 | Test Name                             | API Versions  | Description                                               |
 210 +=======================================+===============+===========================================================+
 211 | test_set_custom_valid_ip_one_subnet   | Nova 2        | Ensures that an instance's can have a valid static IP is  |
 212 |                                       | Neutron 2     | properly assigned                                         |
 213 +---------------------------------------+---------------+-----------------------------------------------------------+
 214 | test_set_custom_invalid_ip_one_subnet | Nova 2        | Ensures that an instance's port with an invalid static IP |
 215 |                                       | Neutron 2     | raises an exception                                       |
 216 +---------------------------------------+---------------+-----------------------------------------------------------+
 217 | test_set_custom_valid_mac             | Nova 2        | Ensures that an instance's port can have a valid MAC      |
 218 |                                       | Neutron 2     | address properly assigned                                 |
 219 +---------------------------------------+---------------+-----------------------------------------------------------+
 220 | test_set_custom_invalid_mac           | Nova 2        | Ensures that an instance's port with an invalid MAC       |
 221 |                                       | Neutron 2     | address raises and exception                              |
 222 +---------------------------------------+---------------+-----------------------------------------------------------+
 223 | test_set_custom_mac_and_ip            | Nova 2        | Ensures that an instance's port with a valid static IP and|
 224 |                                       | Neutron 2     | MAC are properly assigned                                 |
 225 +---------------------------------------+---------------+-----------------------------------------------------------+
 226 | test_set_allowed_address_pairs        | Nova 2        | Ensures the configured allowed_address_pairs is properly  |
 227 |                                       | Neutron 2     | set on a VMs port                                         |
 228 +---------------------------------------+---------------+-----------------------------------------------------------+
 229 | test_set_allowed_address_pairs_bad_mac| Nova 2        | Ensures the port cannot be created when a bad MAC address |
 230 |                                       | Neutron 2     | format is used in the allowed_address_pairs port attribute|
 231 +---------------------------------------+---------------+-----------------------------------------------------------+
 232 | test_set_allowed_address_pairs_bad_ip | Nova 2        | Ensures the port cannot be created when a bad IP address  |
 233 |                                       | Neutron 2     | format is used in the allowed_address_pairs port attribute|
 234 +---------------------------------------+---------------+-----------------------------------------------------------+
 235
 236 create_instance_tests.py - CreateInstanceOnComputeHost
 237 ------------------------------------------------------
 238
 239 +---------------------------------------+---------------+-----------------------------------------------------------+
 240 | Test Name                             | API Versions  | Description                                               |
 241 +=======================================+===============+===========================================================+
 242 | test_deploy_vm_to_each_compute_node   | Nova 2        | Tests to ensure that one can fire up an instance on each  |
 243 |                                       | Neutron 2     | active compute node                                       |
 244 +---------------------------------------+---------------+-----------------------------------------------------------+
 245
 246 create_instance_tests.py - CreateInstancePubPrivNetTests
 247 --------------------------------------------------------
 248
 249 +---------------------------------------+---------------+-----------------------------------------------------------+
 250 | Test Name                             | API Versions  | Description                                               |
 251 +=======================================+===============+===========================================================+
 252 | test_dual_ports_dhcp                  | Nova 2        | Ensures that a VM with two ports/NICs can have its second |
 253 |                                       | Neutron 2     | NIC configured via SSH/Ansible after startup              |
 254 +---------------------------------------+---------------+-----------------------------------------------------------+
 255
 256 create_instance_tests.py - InstanceSecurityGroupTests
 257 -----------------------------------------------------
 258
 259 +---------------------------------------+---------------+-----------------------------------------------------------+
 260 | Test Name                             | API Versions  | Description                                               |
 261 +=======================================+===============+===========================================================+
 262 | test_add_security_group               | Nova 2        | Ensures that a VM instance can have security group added  |
 263 |                                       | Neutron 2     | to it while its running                                   |
 264 +---------------------------------------+---------------+-----------------------------------------------------------+
 265 | test_add_invalid_security_group       | Nova 2        | Ensures that a VM instance does not accept the addition of|
 266 |                                       | Neutron 2     | a security group that no longer exists                    |
 267 +---------------------------------------+---------------+-----------------------------------------------------------+
 268 | test_remove_security_group            | Nova 2        | Ensures that a VM instance accepts the removal of a       |
 269 |                                       | Neutron 2     | security group                                            |
 270 +---------------------------------------+---------------+-----------------------------------------------------------+
 271 | test_remove_security_group_never_added| Nova 2        | Ensures that a VM instance does not accept the removal of |
 272 |                                       | Neutron 2     | a security group that was never added in the first place  |
 273 +---------------------------------------+---------------+-----------------------------------------------------------+
 274 | test_add_same_security_group          | Nova 2        | Ensures that a VM instance does not add a security group  |
 275 |                                       | Neutron 2     | that has already been added to the instance               |
 276 +---------------------------------------+---------------+-----------------------------------------------------------+
 277
 278 ansible_utils_tests.py - AnsibleProvisioningTests
 279 -------------------------------------------------
 280
 281 +---------------------------------------+---------------+-----------------------------------------------------------+
 282 | Test Name                             | API Versions  | Description                                               |
 283 +=======================================+===============+===========================================================+
 284 | test_apply_simple_playbook            | Nova 2        | Ensures that an instance assigned with a floating IP will |
 285 |                                       | Neutron 2     | apply a simple Ansible playbook                           |
 286 +---------------------------------------+---------------+-----------------------------------------------------------+
 287 | test_apply_template_playbook          | Nova 2        | Ensures that an instance assigned with a floating IP will |
 288 |                                       | Neutron 2     | apply a Ansible playbook containing Jinga2 substitution   |
 289 |                                       |               | values                                                    |
 290 +---------------------------------------+---------------+-----------------------------------------------------------+