docs/how-to-use/IntegrationTests.rst

   1 SNAPS OpenStack Integration Testing
   2 ===================================
   3
   4 These tests are ones designed to be run within their own dynamically created project along with a newly generated user
   5 account and generally require other OpenStack object creators.
   6
   7 The Test Classes
   8 ================
   9
  10 create_security_group_tests.py - CreateSecurityGroupTests
  11 ---------------------------------------------------------
  12
  13 +---------------------------------------+---------------+-----------------------------------------------------------+
  14 | Test Name                             | API Versions  | Description                                               |
  15 +=======================================+===============+===========================================================+
  16 | test_create_group_without_rules       | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  17 |                                       | Neutron 2     | security group without any rules                          |
  18 +---------------------------------------+---------------+-----------------------------------------------------------+
  19 | test_create_group_admin_user_to_new   | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can be created   |
  20 | _project                              | Neutron 2     | by the admin user and associated with a new project       |
  21 +---------------------------------------+---------------+-----------------------------------------------------------+
  22 | test_create_group_new_user_to_admin   | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can be created   |
  23 | _project                              | Neutron 2     | by the new user and associated with the admin project     |
  24 +---------------------------------------+---------------+-----------------------------------------------------------+
  25 | test_create_delete_group              | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class clean() method   |
  26 |                                       | Neutron 2     | will not raise an exception should the group be deleted by|
  27 |                                       |               | some other process                                        |
  28 +---------------------------------------+---------------+-----------------------------------------------------------+
  29 | test_create_group_with_one_simple_rule| Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  30 |                                       | Neutron 2     | security group with a single rule                         |
  31 +---------------------------------------+---------------+-----------------------------------------------------------+
  32 | test_create_group_with_several_rules  | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  33 |                                       | Neutron 2     | security group with several rules                         |
  34 +---------------------------------------+---------------+-----------------------------------------------------------+
  35 | test_add_rule                         | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#add_rule() method      |
  36 |                                       | Neutron 2     | properly creates and associates the new rule              |
  37 +---------------------------------------+---------------+-----------------------------------------------------------+
  38 | test_remove_rule_by_id                | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#remove_rule() method   |
  39 |                                       | Neutron 2     | properly deletes and disassociates the old rule via its ID|
  40 +---------------------------------------+---------------+-----------------------------------------------------------+
  41 | test_remove_rule_by_setting           | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#remove_rule() method   |
  42 |                                       | Neutron 2     | properly deletes and disassociates the old rule via its   |
  43 |                                       |               | setting object                                            |
  44 +---------------------------------------+---------------+-----------------------------------------------------------+
  45
  46 create_image_tests.py - CreateImageSuccessTests
  47 -----------------------------------------------
  48
  49 +---------------------------------------+---------------+-----------------------------------------------------------+
  50 | Test Name                             | Glance API    | Description                                               |
  51 +=======================================+===============+===========================================================+
  52 | test_create_image_clean_url           | 1 & 2         | Ensures the OpenStackImage class can create an image from |
  53 |                                       |               | a download URL location                                   |
  54 +---------------------------------------+---------------+-----------------------------------------------------------+
  55 | test_create_image_clean_url_properties| 1 & 2         | Ensures the OpenStackImage class can create an image from |
  56 |                                       |               | a download URL location with custom properties            |
  57 +---------------------------------------+---------------+-----------------------------------------------------------+
  58 | test_create_image_clean_file          | 1 & 2         | Ensures the OpenStackImage class can create an image from |
  59 |                                       |               | a locally sourced image file                              |
  60 +---------------------------------------+---------------+-----------------------------------------------------------+
  61 | test_create_delete_image              | 1 & 2         | Ensures the OpenStackImage.clean() method deletes an image|
  62 |                                       |               | and does not raise an exception on subsequent calls to the|
  63 |                                       |               | clean() method                                            |
  64 +---------------------------------------+---------------+-----------------------------------------------------------+
  65 | test_create_same_image                | 1 & 2         | Ensures the OpenStackImage.create() method does not create|
  66 |                                       |               | another image when one already exists with the same name  |
  67 +---------------------------------------+---------------+-----------------------------------------------------------+
  68 | test_create_same_image_new_settings   | 1 & 2         | Tests the creation of an OpenStack image when the image   |
  69 |                                       |               | already exists and the configuration only contains the    |
  70 |                                       |               | the name.                                                 |
  71 +---------------------------------------+---------------+-----------------------------------------------------------+
  72
  73 create_image_tests.py - CreateImageNegativeTests
  74 ------------------------------------------------
  75
  76 +---------------------------------------+---------------+-----------------------------------------------------------+
  77 | Test Name                             | Glance API    | Description                                               |
  78 +=======================================+===============+===========================================================+
  79 | test_bad_image_name                   | 1 & 2         | Ensures OpenStackImage.create() results in an Exception   |
  80 |                                       |               | being raised when the ImageSettings.name attribute has    |
  81 |                                       |               | not been set                                              |
  82 +---------------------------------------+---------------+-----------------------------------------------------------+
  83 | test_bad_image_url                    | 1 & 2         | Ensures OpenStackImage.create() results in an Exception   |
  84 |                                       |               | being raised when the download URL is invalid             |
  85 +---------------------------------------+---------------+-----------------------------------------------------------+
  86 | test_bad_image_type                   | 1 & 2         | Ensures OpenStackImage.create() results in an Exception   |
  87 |                                       |               | being raised when the image format is 'foo'               |
  88 +---------------------------------------+---------------+-----------------------------------------------------------+
  89 | test_bad_image_file                   | 1 & 2         | Ensures OpenStackImage.create() results in an Exception   |
  90 |                                       |               | being raised when the image file does not exist           |
  91 +---------------------------------------+---------------+-----------------------------------------------------------+
  92
  93 create_image_tests.py - CreateMultiPartImageTests
  94 -------------------------------------------------
  95
  96 +----------------------------------------+---------------+-----------------------------------------------------------+
  97 | Test Name                              | Glance API    | Description                                               |
  98 +========================================+===============+===========================================================+
  99 | test_create_three_part_image_from_url  | 1 & 2         | Ensures that a 3-part image can be created when each part |
 100 |                                        |               | is being sourced from URLs                                |
 101 +----------------------------------------+---------------+-----------------------------------------------------------+
 102 | test_create_three_part_image_from_file | 1 & 2         | Ensures that a 3-part image can be created when each part |
 103 | _3_creators                            |               | is being sourced from local files and 3 creators are used |
 104 +----------------------------------------+---------------+-----------------------------------------------------------+
 105 | test_create_three_part_image_from_url  | 1 & 2         | Ensures that a 3-part image can be created when each part |
 106 | _3_creators                            |               | is being sourced from a URL and 3 creators are used       |
 107 +----------------------------------------+---------------+-----------------------------------------------------------+
 108
 109 create_keypairs_tests.py - CreateKeypairsTests
 110 ----------------------------------------------
 111
 112 +---------------------------------------+---------------+-----------------------------------------------------------+
 113 | Test Name                             | Nova API      | Description                                               |
 114 +=======================================+===============+===========================================================+
 115 | test_create_keypair_only              | 2             | Ensures that a keypair object can be created simply by    |
 116 |                                       |               | only configuring a name                                   |
 117 +---------------------------------------+---------------+-----------------------------------------------------------+
 118 | test_create_delete_keypair            | 2             | Ensures that a keypair object is deleted via              |
 119 |                                       |               | OpenStackKeypair.clean() and subsequent calls do not      |
 120 |                                       |               | result in exceptions                                      |
 121 +---------------------------------------+---------------+-----------------------------------------------------------+
 122 | test_create_keypair_save_pub_only     | 2             | Ensures that a keypair object can be created when the only|
 123 |                                       |               | the public key is cached to disk                          |
 124 +---------------------------------------+---------------+-----------------------------------------------------------+
 125 | test_create_keypair_save_both         | 2             | Ensures that a keypair object can be created when both the|
 126 |                                       |               | public and private keys are cached to disk                |
 127 +---------------------------------------+---------------+-----------------------------------------------------------+
 128 | test_create_keypair_from_file         | 2             | Ensures that a keypair object can be created with an      |
 129 |                                       |               | existing public key file                                  |
 130 +---------------------------------------+---------------+-----------------------------------------------------------+
 131
 132 create_network_tests.py - CreateNetworkSuccessTests
 133 ---------------------------------------------------
 134
 135 +---------------------------------------+---------------+-----------------------------------------------------------+
 136 | Test Name                             | Neutron API   | Description                                               |
 137 +=======================================+===============+===========================================================+
 138 | test_create_network_without_router    | 2             | Ensures that a network can be created via the             |
 139 |                                       |               | OpenStackNetwork class without any routers                |
 140 +---------------------------------------+---------------+-----------------------------------------------------------+
 141 | test_create_delete_network            | 2             | Ensures that a router can be deleted via the              |
 142 |                                       |               | OpenStackNetwork.clean() method                           |
 143 +---------------------------------------+---------------+-----------------------------------------------------------+
 144 | test_create_network_with_router       | 2             | Ensures that a network can be created via the             |
 145 |                                       |               | OpenStackNetwork class with a router                      |
 146 +---------------------------------------+---------------+-----------------------------------------------------------+
 147 | test_create_networks_same_name        | 2             | Ensures that the OpenStackNetwork.create() method will not|
 148 |                                       |               | create a network with the same name                       |
 149 +---------------------------------------+---------------+-----------------------------------------------------------+
 150 | test_create_networks_router_admin_user| 2             | Ensures that the networks, subnets, and routers can be    |
 151 | _to_new_project                       |               | create created by an admin user and assigned to a new     |
 152 |                                       |               | project ID                                                |
 153 +---------------------------------------+---------------+-----------------------------------------------------------+
 154 | test_create_networks_router_new_user  | 2             | Ensures that the networks, subnets, and routers can be    |
 155 | _to_admin_project                     |               | create created by a new admin user and assigned to the    |
 156 |                                       |               | 'admin' project ID                                        |
 157 +---------------------------------------+---------------+-----------------------------------------------------------+
 158
 159 create_router_tests.py - CreateRouterSuccessTests
 160 -------------------------------------------------
 161
 162 +---------------------------------------+---------------+-----------------------------------------------------------+
 163 | Test Name                             | Neutron API   | Description                                               |
 164 +=======================================+===============+===========================================================+
 165 | test_create_router_vanilla            | 2             | Ensures that a router can be created via the              |
 166 |                                       |               | OpenStackRouter class with minimal settings               |
 167 +---------------------------------------+---------------+-----------------------------------------------------------+
 168 | test_create_router_admin_user_to_new  | 2             | Ensures that a router can be created by an admin user and |
 169 | _project                              |               | assigned to a new project                                 |
 170 +---------------------------------------+---------------+-----------------------------------------------------------+
 171 | test_create_router_new_user_to_admin  | 2             | Ensures that a router can be created by a new user and    |
 172 | _project                              |               | assigned to the admin project                             |
 173 +---------------------------------------+---------------+-----------------------------------------------------------+
 174 | test_create_delete_router             | 2             | Ensures that a router can be deleted via the              |
 175 |                                       |               | OpenStackRouter.clean() method                            |
 176 +---------------------------------------+---------------+-----------------------------------------------------------+
 177 | test_create_router_admin_state_false  | 2             | Ensures that a router can created with                    |
 178 |                                       |               | admin_state_up = False                                    |
 179 +---------------------------------------+---------------+-----------------------------------------------------------+
 180 | test_create_router_admin_state_True   | 2             | Ensures that a router can created with                    |
 181 |                                       |               | admin_state_up = True                                     |
 182 +---------------------------------------+---------------+-----------------------------------------------------------+
 183 | test_create_router_private_network    | 2             | Ensures that a router port can be created against a       |
 184 |                                       |               | private network                                           |
 185 +---------------------------------------+---------------+-----------------------------------------------------------+
 186 | test_create_router_external_network   | 2             | Ensures that a router can be created that is connected to |
 187 |                                       |               | both external and private internal networks               |
 188 +---------------------------------------+---------------+-----------------------------------------------------------+
 189
 190 create_router_tests.py - CreateRouterNegativeTests
 191 --------------------------------------------------
 192
 193 +----------------------------------------+---------------+-----------------------------------------------------------+
 194 | Test Name                              | Neutron API   | Description                                               |
 195 +========================================+===============+===========================================================+
 196 | test_create_router_noname              | 2             | Ensures that an exception is raised when attempting to    |
 197 |                                        |               | create a router without a name                            |
 198 +----------------------------------------+---------------+-----------------------------------------------------------+
 199 | test_create_router_invalid_gateway_name| 2             | Ensures that an exception is raised when attempting to    |
 200 |                                        |               | create a router to an external network that does not exist|
 201 +----------------------------------------+---------------+-----------------------------------------------------------+
 202
 203 create_stack_tests.py - CreateStackSuccessTests
 204 -----------------------------------------------
 205
 206 +---------------------------------------+---------------+-----------------------------------------------------------+
 207 | Test Name                             | Neutron API   | Description                                               |
 208 +=======================================+===============+===========================================================+
 209 | test_create_stack_template_file       | 2             | Ensures that a Heat stack can be created with a file-based|
 210 |                                       |               | Heat template file                                        |
 211 +---------------------------------------+---------------+-----------------------------------------------------------+
 212 | test_create_stack_template_dict       | 2             | Ensures that a Heat stack can be created with a dictionary|
 213 |                                       |               | Heat template                                             |
 214 +---------------------------------------+---------------+-----------------------------------------------------------+
 215 | test_create_delete_stack              | 2             | Ensures that a Heat stack can be created and deleted      |
 216 |                                       |               | while having clean() called 2x without an exception       |
 217 +---------------------------------------+---------------+-----------------------------------------------------------+
 218 | test_create_same_stack                | 2             | Ensures that a Heat stack with the same name cannot be    |
 219 |                                       |               | created 2x                                                |
 220 +---------------------------------------+---------------+-----------------------------------------------------------+
 221
 222 create_stack_tests.py - CreateStackNegativeTests
 223 --------------------------------------------------
 224
 225 +----------------------------------------+---------------+-----------------------------------------------------------+
 226 | Test Name                              | Neutron API   | Description                                               |
 227 +========================================+===============+===========================================================+
 228 | test_missing_dependencies              | 2             | Ensures that a Heat template fails to deploy when expected|
 229 |                                        |               | dependencies are missing                                  |
 230 +----------------------------------------+---------------+-----------------------------------------------------------+
 231 | test_bad_stack_file                    | 2             | Ensures that a Heat template fails to deploy when the Heat|
 232 |                                        |               | template file does not exist                              |
 233 +----------------------------------------+---------------+-----------------------------------------------------------+
 234
 235 create_instance_tests.py - CreateInstanceSimpleTests
 236 ----------------------------------------------------
 237
 238 +---------------------------------------+---------------+-----------------------------------------------------------+
 239 | Test Name                             | API Versions  | Description                                               |
 240 +=======================================+===============+===========================================================+
 241 | test_create_delete_instance           | Nova 2        | Ensures that the OpenStackVmInstance.clean() method       |
 242 |                                       | Neutron 2     | deletes the instance                                      |
 243 +---------------------------------------+---------------+-----------------------------------------------------------+
 244
 245 create_instance_tests.py - SimpleHealthCheck
 246 --------------------------------------------
 247
 248 +---------------------------------------+---------------+-----------------------------------------------------------+
 249 | Test Name                             | API Versions  | Description                                               |
 250 +=======================================+===============+===========================================================+
 251 | test_check_vm_ip_dhcp                 | Nova 2        | Tests the creation of an OpenStack instance with a single |
 252 |                                       | Neutron 2     | port and it's assigned IP address                         |
 253 +---------------------------------------+---------------+-----------------------------------------------------------+
 254
 255 create_instance_tests.py - CreateInstanceSingleNetworkTests
 256 -----------------------------------------------------------
 257
 258 +---------------------------------------+---------------+-----------------------------------------------------------+
 259 | Test Name                             | API Versions  | Description                                               |
 260 +=======================================+===============+===========================================================+
 261 | test_single_port_static               | Nova 2        | Ensures that an instance with a single port/NIC with a    |
 262 |                                       | Neutron 2     | static IP can be created                                  |
 263 +---------------------------------------+---------------+-----------------------------------------------------------+
 264 | test_ssh_client_fip_before_active     | Nova 2        | Ensures that an instance can be reached over SSH when the |
 265 |                                       | Neutron 2     | floating IP is assigned prior to the VM becoming ACTIVE   |
 266 +---------------------------------------+---------------+-----------------------------------------------------------+
 267 | test_ssh_client_fip_after_active      | Nova 2        | Ensures that an instance can be reached over SSH when the |
 268 |                                       | Neutron 2     | floating IP is assigned after to the VM becoming ACTIVE   |
 269 +---------------------------------------+---------------+-----------------------------------------------------------+
 270
 271 create_instance_tests.py - CreateInstancePortManipulationTests
 272 --------------------------------------------------------------
 273
 274 +---------------------------------------+---------------+-----------------------------------------------------------+
 275 | Test Name                             | API Versions  | Description                                               |
 276 +=======================================+===============+===========================================================+
 277 | test_set_custom_valid_ip_one_subnet   | Nova 2        | Ensures that an instance's can have a valid static IP is  |
 278 |                                       | Neutron 2     | properly assigned                                         |
 279 +---------------------------------------+---------------+-----------------------------------------------------------+
 280 | test_set_custom_invalid_ip_one_subnet | Nova 2        | Ensures that an instance's port with an invalid static IP |
 281 |                                       | Neutron 2     | raises an exception                                       |
 282 +---------------------------------------+---------------+-----------------------------------------------------------+
 283 | test_set_custom_valid_mac             | Nova 2        | Ensures that an instance's port can have a valid MAC      |
 284 |                                       | Neutron 2     | address properly assigned                                 |
 285 +---------------------------------------+---------------+-----------------------------------------------------------+
 286 | test_set_custom_invalid_mac           | Nova 2        | Ensures that an instance's port with an invalid MAC       |
 287 |                                       | Neutron 2     | address raises and exception                              |
 288 +---------------------------------------+---------------+-----------------------------------------------------------+
 289 | test_set_custom_mac_and_ip            | Nova 2        | Ensures that an instance's port with a valid static IP and|
 290 |                                       | Neutron 2     | MAC are properly assigned                                 |
 291 +---------------------------------------+---------------+-----------------------------------------------------------+
 292 | test_set_allowed_address_pairs        | Nova 2        | Ensures the configured allowed_address_pairs is properly  |
 293 |                                       | Neutron 2     | set on a VMs port                                         |
 294 +---------------------------------------+---------------+-----------------------------------------------------------+
 295 | test_set_allowed_address_pairs_bad_mac| Nova 2        | Ensures the port cannot be created when a bad MAC address |
 296 |                                       | Neutron 2     | format is used in the allowed_address_pairs port attribute|
 297 +---------------------------------------+---------------+-----------------------------------------------------------+
 298 | test_set_allowed_address_pairs_bad_ip | Nova 2        | Ensures the port cannot be created when a bad IP address  |
 299 |                                       | Neutron 2     | format is used in the allowed_address_pairs port attribute|
 300 +---------------------------------------+---------------+-----------------------------------------------------------+
 301
 302 create_instance_tests.py - CreateInstanceOnComputeHost
 303 ------------------------------------------------------
 304
 305 +---------------------------------------+---------------+-----------------------------------------------------------+
 306 | Test Name                             | API Versions  | Description                                               |
 307 +=======================================+===============+===========================================================+
 308 | test_deploy_vm_to_each_compute_node   | Nova 2        | Tests to ensure that one can fire up an instance on each  |
 309 |                                       | Neutron 2     | active compute node                                       |
 310 +---------------------------------------+---------------+-----------------------------------------------------------+
 311
 312 create_instance_tests.py - CreateInstanceFromThreePartImage
 313 -----------------------------------------------------------
 314
 315 +-----------------------------------------------------+---------------+-----------------------------------------------------------+
 316 | Test Name                                           | API Versions  | Description                                               |
 317 +=====================================================+===============+===========================================================+
 318 | test_create_delete_instance_from_three_part_image   | Nova 2        | Tests to ensure that one can fire up an instance then     |
 319 |                                                     | Neutron 2     | delete it when using a 3-part image                       |
 320 +-----------------------------------------------------+---------------+-----------------------------------------------------------+
 321
 322 create_instance_tests.py - CreateInstancePubPrivNetTests
 323 --------------------------------------------------------
 324
 325 +---------------------------------------+---------------+-----------------------------------------------------------+
 326 | Test Name                             | API Versions  | Description                                               |
 327 +=======================================+===============+===========================================================+
 328 | test_dual_ports_dhcp                  | Nova 2        | Ensures that a VM with two ports/NICs can have its second |
 329 |                                       | Neutron 2     | NIC configured via SSH/Ansible after startup              |
 330 +---------------------------------------+---------------+-----------------------------------------------------------+
 331
 332 create_instance_tests.py - InstanceSecurityGroupTests
 333 -----------------------------------------------------
 334
 335 +---------------------------------------+---------------+-----------------------------------------------------------+
 336 | Test Name                             | API Versions  | Description                                               |
 337 +=======================================+===============+===========================================================+
 338 | test_add_security_group               | Nova 2        | Ensures that a VM instance can have security group added  |
 339 |                                       | Neutron 2     | to it while its running                                   |
 340 +---------------------------------------+---------------+-----------------------------------------------------------+
 341 | test_add_invalid_security_group       | Nova 2        | Ensures that a VM instance does not accept the addition of|
 342 |                                       | Neutron 2     | a security group that no longer exists                    |
 343 +---------------------------------------+---------------+-----------------------------------------------------------+
 344 | test_remove_security_group            | Nova 2        | Ensures that a VM instance accepts the removal of a       |
 345 |                                       | Neutron 2     | security group                                            |
 346 +---------------------------------------+---------------+-----------------------------------------------------------+
 347 | test_remove_security_group_never_added| Nova 2        | Ensures that a VM instance does not accept the removal of |
 348 |                                       | Neutron 2     | a security group that was never added in the first place  |
 349 +---------------------------------------+---------------+-----------------------------------------------------------+
 350 | test_add_same_security_group          | Nova 2        | Ensures that a VM instance does not add a security group  |
 351 |                                       | Neutron 2     | that has already been added to the instance               |
 352 +---------------------------------------+---------------+-----------------------------------------------------------+
 353
 354 ansible_utils_tests.py - AnsibleProvisioningTests
 355 -------------------------------------------------
 356
 357 +---------------------------------------+---------------+-----------------------------------------------------------+
 358 | Test Name                             | API Versions  | Description                                               |
 359 +=======================================+===============+===========================================================+
 360 | test_apply_simple_playbook            | Nova 2        | Ensures that an instance assigned with a floating IP will |
 361 |                                       | Neutron 2     | apply a simple Ansible playbook                           |
 362 +---------------------------------------+---------------+-----------------------------------------------------------+
 363 | test_apply_template_playbook          | Nova 2        | Ensures that an instance assigned with a floating IP will |
 364 |                                       | Neutron 2     | apply a Ansible playbook containing Jinga2 substitution   |
 365 |                                       |               | values                                                    |
 366 +---------------------------------------+---------------+-----------------------------------------------------------+