docs/how-to-use/IntegrationTests.rst

   1 SNAPS OpenStack Integration Testing
   2 ===================================
   3
   4 These tests are ones designed to be run within their own dynamically created project along with a newly generated user
   5 account and generally require other OpenStack object creators.
   6
   7 The Test Classes
   8 ================
   9
  10 create_security_group_tests.py - CreateSecurityGroupTests
  11 ---------------------------------------------------------
  12
  13 +---------------------------------------+---------------+-----------------------------------------------------------+
  14 | Test Name                             | API Versions  | Description                                               |
  15 +=======================================+===============+===========================================================+
  16 | test_create_group_without_rules       | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  17 |                                       | Neutron 2     | security group without any rules                          |
  18 +---------------------------------------+---------------+-----------------------------------------------------------+
  19 | test_create_delete_group              | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class clean() method   |
  20 |                                       | Neutron 2     | will not raise an exception should the group be deleted by|
  21 |                                       |               | some other process                                        |
  22 +---------------------------------------+---------------+-----------------------------------------------------------+
  23 | test_create_group_with_one_simple_rule| Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  24 |                                       | Neutron 2     | security group with a single rule                         |
  25 +---------------------------------------+---------------+-----------------------------------------------------------+
  26 | test_create_group_with_several_rules  | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  27 |                                       | Neutron 2     | security group with several rules                         |
  28 +---------------------------------------+---------------+-----------------------------------------------------------+
  29 | test_add_rule                         | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#add_rule() method      |
  30 |                                       | Neutron 2     | properly creates and associates the new rule              |
  31 +---------------------------------------+---------------+-----------------------------------------------------------+
  32 | test_remove_rule_by_id                | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#remove_rule() method   |
  33 |                                       | Neutron 2     | properly deletes and disassociates the old rule via its ID|
  34 +---------------------------------------+---------------+-----------------------------------------------------------+
  35 | test_remove_rule_by_setting           | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#remove_rule() method   |
  36 |                                       | Neutron 2     | properly deletes and disassociates the old rule via its   |
  37 |                                       |               | setting object                                            |
  38 +---------------------------------------+---------------+-----------------------------------------------------------+
  39
  40 create_image_tests.py - CreateImageSuccessTests
  41 -----------------------------------------------
  42
  43 +---------------------------------------+---------------+-----------------------------------------------------------+
  44 | Test Name                             | Glance API    | Description                                               |
  45 +=======================================+===============+===========================================================+
  46 | test_create_image_clean_url           | 1             | Ensures the OpenStackImage class can create an image from |
  47 |                                       |               | a download URL location                                   |
  48 +---------------------------------------+---------------+-----------------------------------------------------------+
  49 | test_create_image_clean_file          | 1             | Ensures the OpenStackImage class can create an image from |
  50 |                                       |               | a locally sourced image file                              |
  51 +---------------------------------------+---------------+-----------------------------------------------------------+
  52 | test_create_delete_image              | 1             | Ensures the OpenStackImage.clean() method deletes an image|
  53 |                                       |               | and does not raise an exception on subsequent calls to the|
  54 |                                       |               | clean() method                                            |
  55 +---------------------------------------+---------------+-----------------------------------------------------------+
  56 | test_create_same_image                | 1             | Ensures the OpenStackImage.create() method does not create|
  57 |                                       |               | another image when one already exists with the same name  |
  58 +---------------------------------------+---------------+-----------------------------------------------------------+
  59 | test_create_same_image_new_settings   | 1             | Tests the creation of an OpenStack image when the image   |
  60 |                                       |               | already exists and the configuration only contains the    |
  61 |                                       |               | the name.                                                 |
  62 +---------------------------------------+---------------+-----------------------------------------------------------+
  63
  64 create_image_tests.py - CreateImageNegativeTests
  65 ------------------------------------------------
  66
  67 +---------------------------------------+---------------+-----------------------------------------------------------+
  68 | Test Name                             | Glance API    | Description                                               |
  69 +=======================================+===============+===========================================================+
  70 | test_bad_image_name                   | 1             | Ensures OpenStackImage.create() results in an Exception   |
  71 |                                       |               | being raised when the ImageSettings.name attribute has    |
  72 |                                       |               | not been set                                              |
  73 +---------------------------------------+---------------+-----------------------------------------------------------+
  74 | test_bad_image_url                    | 1             | Ensures OpenStackImage.create() results in an Exception   |
  75 |                                       |               | being raised when the download URL is invalid             |
  76 +---------------------------------------+---------------+-----------------------------------------------------------+
  77 | test_bad_image_file                   | 1             | Ensures OpenStackImage.create() results in an Exception   |
  78 |                                       |               | being raised when the image file does not exist           |
  79 +---------------------------------------+---------------+-----------------------------------------------------------+
  80
  81 create_image_tests.py - CreateMultiPartImageTests
  82 -------------------------------------------------
  83
  84 +----------------------------------------+---------------+-----------------------------------------------------------+
  85 | Test Name                              | Glance API    | Description                                               |
  86 +========================================+===============+===========================================================+
  87 | test_create_three_part_image_from_url  | 1             | Ensures that a 3-part image can be created when each part |
  88 |                                        |               | is being sourced from URLs                                |
  89 +----------------------------------------+---------------+-----------------------------------------------------------+
  90 | test_create_three_part_image_from_file | 1             | Ensures that a 3-part image can be created when each part |
  91 |                                        |               | is being sourced from local files                         |
  92 +----------------------------------------+---------------+-----------------------------------------------------------+
  93
  94 create_keypairs_tests.py - CreateKeypairsTests
  95 ----------------------------------------------
  96
  97 +---------------------------------------+---------------+-----------------------------------------------------------+
  98 | Test Name                             | Nova API      | Description                                               |
  99 +=======================================+===============+===========================================================+
 100 | test_create_keypair_only              | 2             | Ensures that a keypair object can be created simply by    |
 101 |                                       |               | only configuring a name                                   |
 102 +---------------------------------------+---------------+-----------------------------------------------------------+
 103 | test_create_delete_keypair            | 2             | Ensures that a keypair object is deleted via              |
 104 |                                       |               | OpenStackKeypair.clean() and subsequent calls do not      |
 105 |                                       |               | result in exceptions                                      |
 106 +---------------------------------------+---------------+-----------------------------------------------------------+
 107 | test_create_keypair_save_pub_only     | 2             | Ensures that a keypair object can be created when the only|
 108 |                                       |               | the public key is cached to disk                          |
 109 +---------------------------------------+---------------+-----------------------------------------------------------+
 110 | test_create_keypair_save_both         | 2             | Ensures that a keypair object can be created when both the|
 111 |                                       |               | public and private keys are cached to disk                |
 112 +---------------------------------------+---------------+-----------------------------------------------------------+
 113 | test_create_keypair_from_file         | 2             | Ensures that a keypair object can be created with an      |
 114 |                                       |               | existing public key file                                  |
 115 +---------------------------------------+---------------+-----------------------------------------------------------+
 116
 117 create_keypairs_tests.py - CreateKeypairsCleanupTests
 118 -----------------------------------------------------
 119
 120 +---------------------------------------+---------------+-----------------------------------------------------------+
 121 | Test Name                             | Nova API      | Description                                               |
 122 +=======================================+===============+===========================================================+
 123 | test_create_keypair_gen_files_delete_1| 2             | Ensures that new keypair files are deleted by default     |
 124 |                                       |               | by OpenStackKeypair#clean()                               |
 125 +---------------------------------------+---------------+-----------------------------------------------------------+
 126 | test_create_keypair_gen_files_delete_2| 2             | Ensures that new keypair files are deleted by         |
 127 |                                       |               | OpenStackKeypair#clean() when the settings delete_on_clean|
 128 |                                       |               | attribute is set to True                                  |
 129 +---------------------------------------+---------------+-----------------------------------------------------------+
 130 | test_create_keypair_gen_files_keep    | 2             | Ensures that new keypair files are not deleted by         |
 131 |                                       |               | OpenStackKeypair#clean()                                  |
 132 +---------------------------------------+---------------+-----------------------------------------------------------+
 133 | test_create_keypair_exist_files_keep  | 2             | Ensures that existing keypair files are not deleted by    |
 134 |                                       |               | OpenStackKeypair#clean()                                  |
 135 +---------------------------------------+---------------+-----------------------------------------------------------+
 136 | test_create_keypair_exist_files_delete| 2             | Ensures that existing keypair files are deleted by        |
 137 |                                       |               | OpenStackKeypair#clean()                                  |
 138 +---------------------------------------+---------------+-----------------------------------------------------------+
 139
 140 create_network_tests.py - CreateNetworkSuccessTests
 141 ---------------------------------------------------
 142
 143 +---------------------------------------+---------------+-----------------------------------------------------------+
 144 | Test Name                             | Neutron API   | Description                                               |
 145 +=======================================+===============+===========================================================+
 146 | test_create_network_without_router    | 2             | Ensures that a network can be created via the             |
 147 |                                       |               | OpenStackNetwork class without any routers                |
 148 +---------------------------------------+---------------+-----------------------------------------------------------+
 149 | test_create_delete_network            | 2             | Ensures that a router can be deleted via the              |
 150 |                                       |               | OpenStackNetwork.clean() method                           |
 151 +---------------------------------------+---------------+-----------------------------------------------------------+
 152 | test_create_network_with_router       | 2             | Ensures that a network can be created via the             |
 153 |                                       |               | OpenStackNetwork class with a router                      |
 154 +---------------------------------------+---------------+-----------------------------------------------------------+
 155 | test_create_networks_same_name        | 2             | Ensures that the OpenStackNetwork.create() method will not|
 156 |                                       |               | create a network with the same name                       |
 157 +---------------------------------------+---------------+-----------------------------------------------------------+
 158
 159 create_router_tests.py - CreateRouterSuccessTests
 160 -------------------------------------------------
 161
 162 +---------------------------------------+---------------+-----------------------------------------------------------+
 163 | Test Name                             | Neutron API   | Description                                               |
 164 +=======================================+===============+===========================================================+
 165 | test_create_router_vanilla            | 2             | Ensures that a router can be created via the              |
 166 |                                       |               | OpenStackRouter class with minimal settings               |
 167 +---------------------------------------+---------------+-----------------------------------------------------------+
 168 | test_create_delete_router             | 2             | Ensures that a router can be deleted via the              |
 169 |                                       |               | OpenStackRouter.clean() method                            |
 170 +---------------------------------------+---------------+-----------------------------------------------------------+
 171 | test_create_router_admin_state_false  | 2             | Ensures that a router can created with                    |
 172 |                                       |               | admin_state_up = False                                    |
 173 +---------------------------------------+---------------+-----------------------------------------------------------+
 174 | test_create_router_admin_state_True   | 2             | Ensures that a router can created with                    |
 175 |                                       |               | admin_state_up = True                                     |
 176 +---------------------------------------+---------------+-----------------------------------------------------------+
 177 | test_create_router_private_network    | 2             | Ensures that a router port can be created against a       |
 178 |                                       |               | private network                                           |
 179 +---------------------------------------+---------------+-----------------------------------------------------------+
 180 | test_create_router_external_network   | 2             | Ensures that a router can be created that is connected to |
 181 |                                       |               | both external and private internal networks               |
 182 +---------------------------------------+---------------+-----------------------------------------------------------+
 183
 184 create_router_tests.py - CreateRouterNegativeTests
 185 --------------------------------------------------
 186
 187 +----------------------------------------+---------------+-----------------------------------------------------------+
 188 | Test Name                              | Neutron API   | Description                                               |
 189 +========================================+===============+===========================================================+
 190 | test_create_router_noname              | 2             | Ensures that an exception is raised when attempting to    |
 191 |                                        |               | create a router without a name                            |
 192 +----------------------------------------+---------------+-----------------------------------------------------------+
 193 | test_create_router_invalid_gateway_name| 2             | Ensures that an exception is raised when attempting to    |
 194 |                                        |               | create a router to an external network that does not exist|
 195 +----------------------------------------+---------------+-----------------------------------------------------------+
 196
 197 create_instance_tests.py - CreateInstanceSimpleTests
 198 ----------------------------------------------------
 199
 200 +---------------------------------------+---------------+-----------------------------------------------------------+
 201 | Test Name                             | API Versions  | Description                                               |
 202 +=======================================+===============+===========================================================+
 203 | test_create_delete_instance           | Nova 2        | Ensures that the OpenStackVmInstance.clean() method       |
 204 |                                       | Neutron 2     | deletes the instance                                      |
 205 +---------------------------------------+---------------+-----------------------------------------------------------+
 206
 207 create_instance_tests.py - SimpleHealthCheck
 208 --------------------------------------------
 209
 210 +---------------------------------------+---------------+-----------------------------------------------------------+
 211 | Test Name                             | API Versions  | Description                                               |
 212 +=======================================+===============+===========================================================+
 213 | test_check_vm_ip_dhcp                 | Nova 2        | Tests the creation of an OpenStack instance with a single |
 214 |                                       | Neutron 2     | port and it's assigned IP address                         |
 215 +---------------------------------------+---------------+-----------------------------------------------------------+
 216
 217 create_instance_tests.py - CreateInstanceSingleNetworkTests
 218 -----------------------------------------------------------
 219
 220 +---------------------------------------+---------------+-----------------------------------------------------------+
 221 | Test Name                             | API Versions  | Description                                               |
 222 +=======================================+===============+===========================================================+
 223 | test_single_port_static               | Nova 2        | Ensures that an instance with a single port/NIC with a    |
 224 |                                       | Neutron 2     | static IP can be created                                  |
 225 +---------------------------------------+---------------+-----------------------------------------------------------+
 226 | test_ssh_client_fip_before_active     | Nova 2        | Ensures that an instance can be reached over SSH when the |
 227 |                                       | Neutron 2     | floating IP is assigned prior to the VM becoming ACTIVE   |
 228 +---------------------------------------+---------------+-----------------------------------------------------------+
 229 | test_ssh_client_fip_after_active      | Nova 2        | Ensures that an instance can be reached over SSH when the |
 230 |                                       | Neutron 2     | floating IP is assigned after to the VM becoming ACTIVE   |
 231 +---------------------------------------+---------------+-----------------------------------------------------------+
 232
 233 create_instance_tests.py - CreateInstancePortManipulationTests
 234 --------------------------------------------------------------
 235
 236 +---------------------------------------+---------------+-----------------------------------------------------------+
 237 | Test Name                             | API Versions  | Description                                               |
 238 +=======================================+===============+===========================================================+
 239 | test_set_custom_valid_ip_one_subnet   | Nova 2        | Ensures that an instance's can have a valid static IP is  |
 240 |                                       | Neutron 2     | properly assigned                                         |
 241 +---------------------------------------+---------------+-----------------------------------------------------------+
 242 | test_set_custom_invalid_ip_one_subnet | Nova 2        | Ensures that an instance's port with an invalid static IP |
 243 |                                       | Neutron 2     | raises an exception                                       |
 244 +---------------------------------------+---------------+-----------------------------------------------------------+
 245 | test_set_custom_valid_mac             | Nova 2        | Ensures that an instance's port can have a valid MAC      |
 246 |                                       | Neutron 2     | address properly assigned                                 |
 247 +---------------------------------------+---------------+-----------------------------------------------------------+
 248 | test_set_custom_invalid_mac           | Nova 2        | Ensures that an instance's port with an invalid MAC       |
 249 |                                       | Neutron 2     | address raises and exception                              |
 250 +---------------------------------------+---------------+-----------------------------------------------------------+
 251 | test_set_custom_mac_and_ip            | Nova 2        | Ensures that an instance's port with a valid static IP and|
 252 |                                       | Neutron 2     | MAC are properly assigned                                 |
 253 +---------------------------------------+---------------+-----------------------------------------------------------+
 254 | test_set_allowed_address_pairs        | Nova 2        | Ensures the configured allowed_address_pairs is properly  |
 255 |                                       | Neutron 2     | set on a VMs port                                         |
 256 +---------------------------------------+---------------+-----------------------------------------------------------+
 257 | test_set_allowed_address_pairs_bad_mac| Nova 2        | Ensures the port cannot be created when a bad MAC address |
 258 |                                       | Neutron 2     | format is used in the allowed_address_pairs port attribute|
 259 +---------------------------------------+---------------+-----------------------------------------------------------+
 260 | test_set_allowed_address_pairs_bad_ip | Nova 2        | Ensures the port cannot be created when a bad IP address  |
 261 |                                       | Neutron 2     | format is used in the allowed_address_pairs port attribute|
 262 +---------------------------------------+---------------+-----------------------------------------------------------+
 263
 264 create_instance_tests.py - CreateInstanceOnComputeHost
 265 ------------------------------------------------------
 266
 267 +---------------------------------------+---------------+-----------------------------------------------------------+
 268 | Test Name                             | API Versions  | Description                                               |
 269 +=======================================+===============+===========================================================+
 270 | test_deploy_vm_to_each_compute_node   | Nova 2        | Tests to ensure that one can fire up an instance on each  |
 271 |                                       | Neutron 2     | active compute node                                       |
 272 +---------------------------------------+---------------+-----------------------------------------------------------+
 273
 274 create_instance_tests.py - CreateInstanceFromThreePartImage
 275 -----------------------------------------------------------
 276
 277 +-----------------------------------------------------+---------------+-----------------------------------------------------------+
 278 | Test Name                                           | API Versions  | Description                                               |
 279 +=====================================================+===============+===========================================================+
 280 | test_create_delete_instance_from_three_part_image   | Nova 2        | Tests to ensure that one can fire up an instance then     |
 281 |                                                     | Neutron 2     | delete it when using a 3-part image                       |
 282 +-----------------------------------------------------+---------------+-----------------------------------------------------------+
 283
 284 create_instance_tests.py - CreateInstancePubPrivNetTests
 285 --------------------------------------------------------
 286
 287 +---------------------------------------+---------------+-----------------------------------------------------------+
 288 | Test Name                             | API Versions  | Description                                               |
 289 +=======================================+===============+===========================================================+
 290 | test_dual_ports_dhcp                  | Nova 2        | Ensures that a VM with two ports/NICs can have its second |
 291 |                                       | Neutron 2     | NIC configured via SSH/Ansible after startup              |
 292 +---------------------------------------+---------------+-----------------------------------------------------------+
 293
 294 create_instance_tests.py - InstanceSecurityGroupTests
 295 -----------------------------------------------------
 296
 297 +---------------------------------------+---------------+-----------------------------------------------------------+
 298 | Test Name                             | API Versions  | Description                                               |
 299 +=======================================+===============+===========================================================+
 300 | test_add_security_group               | Nova 2        | Ensures that a VM instance can have security group added  |
 301 |                                       | Neutron 2     | to it while its running                                   |
 302 +---------------------------------------+---------------+-----------------------------------------------------------+
 303 | test_add_invalid_security_group       | Nova 2        | Ensures that a VM instance does not accept the addition of|
 304 |                                       | Neutron 2     | a security group that no longer exists                    |
 305 +---------------------------------------+---------------+-----------------------------------------------------------+
 306 | test_remove_security_group            | Nova 2        | Ensures that a VM instance accepts the removal of a       |
 307 |                                       | Neutron 2     | security group                                            |
 308 +---------------------------------------+---------------+-----------------------------------------------------------+
 309 | test_remove_security_group_never_added| Nova 2        | Ensures that a VM instance does not accept the removal of |
 310 |                                       | Neutron 2     | a security group that was never added in the first place  |
 311 +---------------------------------------+---------------+-----------------------------------------------------------+
 312 | test_add_same_security_group          | Nova 2        | Ensures that a VM instance does not add a security group  |
 313 |                                       | Neutron 2     | that has already been added to the instance               |
 314 +---------------------------------------+---------------+-----------------------------------------------------------+
 315
 316 ansible_utils_tests.py - AnsibleProvisioningTests
 317 -------------------------------------------------
 318
 319 +---------------------------------------+---------------+-----------------------------------------------------------+
 320 | Test Name                             | API Versions  | Description                                               |
 321 +=======================================+===============+===========================================================+
 322 | test_apply_simple_playbook            | Nova 2        | Ensures that an instance assigned with a floating IP will |
 323 |                                       | Neutron 2     | apply a simple Ansible playbook                           |
 324 +---------------------------------------+---------------+-----------------------------------------------------------+
 325 | test_apply_template_playbook          | Nova 2        | Ensures that an instance assigned with a floating IP will |
 326 |                                       | Neutron 2     | apply a Ansible playbook containing Jinga2 substitution   |
 327 |                                       |               | values                                                    |
 328 +---------------------------------------+---------------+-----------------------------------------------------------+