docs/how-to-use/IntegrationTests.rst

   1 SNAPS OpenStack Integration Testing
   2 ===================================
   3
   4 These tests are ones designed to be run within their own dynamically created project along with a newly generated user
   5 account and generally require other OpenStack object creators.
   6
   7 The Test Classes
   8 ================
   9
  10 create_security_group_tests.py - CreateSecurityGroupTests
  11 ---------------------------------------------------------
  12
  13 +---------------------------------------+---------------+-----------------------------------------------------------+
  14 | Test Name                             | API Versions  | Description                                               |
  15 +=======================================+===============+===========================================================+
  16 | test_create_group_without_rules       | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  17 |                                       | Neutron 2     | security group without any rules                          |
  18 +---------------------------------------+---------------+-----------------------------------------------------------+
  19 | test_create_delete_group              | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class clean() method   |
  20 |                                       | Neutron 2     | will not raise an exception should the group be deleted by|
  21 |                                       |               | some other process                                        |
  22 +---------------------------------------+---------------+-----------------------------------------------------------+
  23 | test_create_group_with_one_simple_rule| Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  24 |                                       | Neutron 2     | security group with a single rule                         |
  25 +---------------------------------------+---------------+-----------------------------------------------------------+
  26 | test_create_group_with_several_rules  | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
  27 |                                       | Neutron 2     | security group with several rules                         |
  28 +---------------------------------------+---------------+-----------------------------------------------------------+
  29 | test_add_rule                         | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#add_rule() method      |
  30 |                                       | Neutron 2     | properly creates and associates the new rule              |
  31 +---------------------------------------+---------------+-----------------------------------------------------------+
  32 | test_remove_rule_by_id                | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#remove_rule() method   |
  33 |                                       | Neutron 2     | properly deletes and disassociates the old rule via its ID|
  34 +---------------------------------------+---------------+-----------------------------------------------------------+
  35 | test_remove_rule_by_setting           | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#remove_rule() method   |
  36 |                                       | Neutron 2     | properly deletes and disassociates the old rule via its   |
  37 |                                       |               | setting object                                            |
  38 +---------------------------------------+---------------+-----------------------------------------------------------+
  39
  40 create_image_tests.py - CreateImageSuccessTests
  41 -----------------------------------------------
  42
  43 +---------------------------------------+---------------+-----------------------------------------------------------+
  44 | Test Name                             | Glance API    | Description                                               |
  45 +=======================================+===============+===========================================================+
  46 | test_create_image_clean_url           | 1             | Ensures the OpenStackImage class can create an image from |
  47 |                                       |               | a download URL location                                   |
  48 +---------------------------------------+---------------+-----------------------------------------------------------+
  49 | test_create_image_clean_file          | 1             | Ensures the OpenStackImage class can create an image from |
  50 |                                       |               | a locally sourced image file                              |
  51 +---------------------------------------+---------------+-----------------------------------------------------------+
  52 | test_create_delete_image              | 1             | Ensures the OpenStackImage.clean() method deletes an image|
  53 |                                       |               | and does not raise an exception on subsequent calls to the|
  54 |                                       |               | clean() method                                            |
  55 +---------------------------------------+---------------+-----------------------------------------------------------+
  56 | test_create_same_image                | 1             | Ensures the OpenStackImage.create() method does not create|
  57 |                                       |               | another image when one already exists with the same name  |
  58 +---------------------------------------+---------------+-----------------------------------------------------------+
  59
  60 create_image_tests.py - CreateImageNegativeTests
  61 ------------------------------------------------
  62
  63 +---------------------------------------+---------------+-----------------------------------------------------------+
  64 | Test Name                             | Glance API    | Description                                               |
  65 +=======================================+===============+===========================================================+
  66 | test_none_image_name                  | 1             | Ensures OpenStackImage.create() results in an Exception   |
  67 |                                       |               | being raised when the ImageSettings.name attribute has    |
  68 |                                       |               | not been set                                              |
  69 +---------------------------------------+---------------+-----------------------------------------------------------+
  70 | test_bad_image_url                    | 1             | Ensures OpenStackImage.create() results in an Exception   |
  71 |                                       |               | being raised when the download URL is invalid             |
  72 +---------------------------------------+---------------+-----------------------------------------------------------+
  73 | test_bad_image_file                   | 1             | Ensures OpenStackImage.create() results in an Exception   |
  74 |                                       |               | being raised when the image file does not exist           |
  75 +---------------------------------------+---------------+-----------------------------------------------------------+
  76 | test_none_proj_name                   | 1             | Ensures OpenStackImage.create() results in an Exception   |
  77 |                                       |               | being raised when the credentials project name is None    |
  78 +---------------------------------------+---------------+-----------------------------------------------------------+
  79 | test_none_auth_url                    | 1             | Ensures OpenStackImage.create() results in an Exception   |
  80 |                                       |               | being raised when the credentials URL is None             |
  81 +---------------------------------------+---------------+-----------------------------------------------------------+
  82 | test_none_password                    | 1             | Ensures OpenStackImage.create() results in an Exception   |
  83 |                                       |               | being raised when the credentials password is None        |
  84 +---------------------------------------+---------------+-----------------------------------------------------------+
  85 | test_none_user                        | 1             | Ensures OpenStackImage.create() results in an Exception   |
  86 |                                       |               | being raised when the credentials user is None            |
  87 +---------------------------------------+---------------+-----------------------------------------------------------+
  88
  89 create_image_tests.py - CreateMultiPartImageTests
  90 -------------------------------------------------
  91
  92 +----------------------------------------+---------------+-----------------------------------------------------------+
  93 | Test Name                              | Glance API    | Description                                               |
  94 +========================================+===============+===========================================================+
  95 | test_create_three_part_image_from_url  | 1             | Ensures that a 3-part image can be created when each part |
  96 |                                        |               | is being sourced from URLs                                |
  97 +----------------------------------------+---------------+-----------------------------------------------------------+
  98 | test_create_three_part_image_from_file | 1             | Ensures that a 3-part image can be created when each part |
  99 |                                        |               | is being sourced from local files                         |
 100 +----------------------------------------+---------------+-----------------------------------------------------------+
 101
 102 create_keypairs_tests.py - CreateKeypairsTests
 103 ----------------------------------------------
 104
 105 +---------------------------------------+---------------+-----------------------------------------------------------+
 106 | Test Name                             | Nova API      | Description                                               |
 107 +=======================================+===============+===========================================================+
 108 | test_create_keypair_only              | 2             | Ensures that a keypair object can be created simply by    |
 109 |                                       |               | only configuring a name                                   |
 110 +---------------------------------------+---------------+-----------------------------------------------------------+
 111 | test_create_delete_keypair            | 2             | Ensures that a keypair object is deleted via              |
 112 |                                       |               | OpenStackKeypair.clean() and subsequent calls do not      |
 113 |                                       |               | result in exceptions                                      |
 114 +---------------------------------------+---------------+-----------------------------------------------------------+
 115 | test_create_keypair_save_pub_only     | 2             | Ensures that a keypair object can be created when the only|
 116 |                                       |               | the public key is cached to disk                          |
 117 +---------------------------------------+---------------+-----------------------------------------------------------+
 118 | test_create_keypair_save_both         | 2             | Ensures that a keypair object can be created when both the|
 119 |                                       |               | public and private keys are cached to disk                |
 120 +---------------------------------------+---------------+-----------------------------------------------------------+
 121 | test_create_keypair_from_file         | 2             | Ensures that a keypair object can be created with an      |
 122 |                                       |               | existing public key file                                  |
 123 +---------------------------------------+---------------+-----------------------------------------------------------+
 124
 125 create_network_tests.py - CreateNetworkSuccessTests
 126 ---------------------------------------------------
 127
 128 +---------------------------------------+---------------+-----------------------------------------------------------+
 129 | Test Name                             | Neutron API   | Description                                               |
 130 +=======================================+===============+===========================================================+
 131 | test_create_network_without_router    | 2             | Ensures that a network can be created via the             |
 132 |                                       |               | OpenStackNetwork class without any routers                |
 133 +---------------------------------------+---------------+-----------------------------------------------------------+
 134 | test_create_delete_network            | 2             | Ensures that a router can be deleted via the              |
 135 |                                       |               | OpenStackNetwork.clean() method                           |
 136 +---------------------------------------+---------------+-----------------------------------------------------------+
 137 | test_create_network_with_router       | 2             | Ensures that a network can be created via the             |
 138 |                                       |               | OpenStackNetwork class with a router                      |
 139 +---------------------------------------+---------------+-----------------------------------------------------------+
 140 | test_create_networks_same_name        | 2             | Ensures that the OpenStackNetwork.create() method will not|
 141 |                                       |               | create a network with the same name                       |
 142 +---------------------------------------+---------------+-----------------------------------------------------------+
 143
 144 create_router_tests.py - CreateRouterSuccessTests
 145 -------------------------------------------------
 146
 147 +---------------------------------------+---------------+-----------------------------------------------------------+
 148 | Test Name                             | Neutron API   | Description                                               |
 149 +=======================================+===============+===========================================================+
 150 | test_create_router_vanilla            | 2             | Ensures that a router can be created via the              |
 151 |                                       |               | OpenStackRouter class with minimal settings               |
 152 +---------------------------------------+---------------+-----------------------------------------------------------+
 153 | test_create_delete_router             | 2             | Ensures that a router can be deleted via the              |
 154 |                                       |               | OpenStackRouter.clean() method                            |
 155 +---------------------------------------+---------------+-----------------------------------------------------------+
 156 | test_create_router_admin_state_false  | 2             | Ensures that a router can created with                    |
 157 |                                       |               | admin_state_up = False                                    |
 158 +---------------------------------------+---------------+-----------------------------------------------------------+
 159 | test_create_router_admin_state_True   | 2             | Ensures that a router can created with                    |
 160 |                                       |               | admin_state_up = True                                     |
 161 +---------------------------------------+---------------+-----------------------------------------------------------+
 162 | test_create_router_private_network    | 2             | Ensures that a router port can be created against a       |
 163 |                                       |               | private network                                           |
 164 +---------------------------------------+---------------+-----------------------------------------------------------+
 165 | test_create_router_external_network   | 2             | Ensures that a router can be created that is connected to |
 166 |                                       |               | both external and private internal networks               |
 167 +---------------------------------------+---------------+-----------------------------------------------------------+
 168
 169 create_router_tests.py - CreateRouterNegativeTests
 170 --------------------------------------------------
 171
 172 +----------------------------------------+---------------+-----------------------------------------------------------+
 173 | Test Name                              | Neutron API   | Description                                               |
 174 +========================================+===============+===========================================================+
 175 | test_create_router_noname              | 2             | Ensures that an exception is raised when attempting to    |
 176 |                                        |               | create a router without a name                            |
 177 +----------------------------------------+---------------+-----------------------------------------------------------+
 178 | test_create_router_invalid_gateway_name| 2             | Ensures that an exception is raised when attempting to    |
 179 |                                        |               | create a router to an external network that does not exist|
 180 +----------------------------------------+---------------+-----------------------------------------------------------+
 181
 182 create_instance_tests.py - CreateInstanceSimpleTests
 183 ----------------------------------------------------
 184
 185 +---------------------------------------+---------------+-----------------------------------------------------------+
 186 | Test Name                             | API Versions  | Description                                               |
 187 +=======================================+===============+===========================================================+
 188 | test_create_delete_instance           | Nova 2        | Ensures that the OpenStackVmInstance.clean() method       |
 189 |                                       | Neutron 2     | deletes the instance                                      |
 190 +---------------------------------------+---------------+-----------------------------------------------------------+
 191
 192 create_instance_tests.py - SimpleHealthCheck
 193 --------------------------------------------
 194
 195 +---------------------------------------+---------------+-----------------------------------------------------------+
 196 | Test Name                             | API Versions  | Description                                               |
 197 +=======================================+===============+===========================================================+
 198 | test_check_vm_ip_dhcp                 | Nova 2        | Tests the creation of an OpenStack instance with a single |
 199 |                                       | Neutron 2     | port and it's assigned IP address                         |
 200 +---------------------------------------+---------------+-----------------------------------------------------------+
 201
 202 create_instance_tests.py - CreateInstanceSingleNetworkTests
 203 -----------------------------------------------------------
 204
 205 +---------------------------------------+---------------+-----------------------------------------------------------+
 206 | Test Name                             | API Versions  | Description                                               |
 207 +=======================================+===============+===========================================================+
 208 | test_single_port_static               | Nova 2        | Ensures that an instance with a single port/NIC with a    |
 209 |                                       | Neutron 2     | static IP can be created                                  |
 210 +---------------------------------------+---------------+-----------------------------------------------------------+
 211 | test_ssh_client_fip_before_active     | Nova 2        | Ensures that an instance can be reached over SSH when the |
 212 |                                       | Neutron 2     | floating IP is assigned prior to the VM becoming ACTIVE   |
 213 +---------------------------------------+---------------+-----------------------------------------------------------+
 214 | test_ssh_client_fip_after_active      | Nova 2        | Ensures that an instance can be reached over SSH when the |
 215 |                                       | Neutron 2     | floating IP is assigned after to the VM becoming ACTIVE   |
 216 +---------------------------------------+---------------+-----------------------------------------------------------+
 217
 218 create_instance_tests.py - CreateInstancePortManipulationTests
 219 --------------------------------------------------------------
 220
 221 +---------------------------------------+---------------+-----------------------------------------------------------+
 222 | Test Name                             | API Versions  | Description                                               |
 223 +=======================================+===============+===========================================================+
 224 | test_set_custom_valid_ip_one_subnet   | Nova 2        | Ensures that an instance's can have a valid static IP is  |
 225 |                                       | Neutron 2     | properly assigned                                         |
 226 +---------------------------------------+---------------+-----------------------------------------------------------+
 227 | test_set_custom_invalid_ip_one_subnet | Nova 2        | Ensures that an instance's port with an invalid static IP |
 228 |                                       | Neutron 2     | raises an exception                                       |
 229 +---------------------------------------+---------------+-----------------------------------------------------------+
 230 | test_set_custom_valid_mac             | Nova 2        | Ensures that an instance's port can have a valid MAC      |
 231 |                                       | Neutron 2     | address properly assigned                                 |
 232 +---------------------------------------+---------------+-----------------------------------------------------------+
 233 | test_set_custom_invalid_mac           | Nova 2        | Ensures that an instance's port with an invalid MAC       |
 234 |                                       | Neutron 2     | address raises and exception                              |
 235 +---------------------------------------+---------------+-----------------------------------------------------------+
 236 | test_set_custom_mac_and_ip            | Nova 2        | Ensures that an instance's port with a valid static IP and|
 237 |                                       | Neutron 2     | MAC are properly assigned                                 |
 238 +---------------------------------------+---------------+-----------------------------------------------------------+
 239 | test_set_allowed_address_pairs        | Nova 2        | Ensures the configured allowed_address_pairs is properly  |
 240 |                                       | Neutron 2     | set on a VMs port                                         |
 241 +---------------------------------------+---------------+-----------------------------------------------------------+
 242 | test_set_allowed_address_pairs_bad_mac| Nova 2        | Ensures the port cannot be created when a bad MAC address |
 243 |                                       | Neutron 2     | format is used in the allowed_address_pairs port attribute|
 244 +---------------------------------------+---------------+-----------------------------------------------------------+
 245 | test_set_allowed_address_pairs_bad_ip | Nova 2        | Ensures the port cannot be created when a bad IP address  |
 246 |                                       | Neutron 2     | format is used in the allowed_address_pairs port attribute|
 247 +---------------------------------------+---------------+-----------------------------------------------------------+
 248
 249 create_instance_tests.py - CreateInstanceOnComputeHost
 250 ------------------------------------------------------
 251
 252 +---------------------------------------+---------------+-----------------------------------------------------------+
 253 | Test Name                             | API Versions  | Description                                               |
 254 +=======================================+===============+===========================================================+
 255 | test_deploy_vm_to_each_compute_node   | Nova 2        | Tests to ensure that one can fire up an instance on each  |
 256 |                                       | Neutron 2     | active compute node                                       |
 257 +---------------------------------------+---------------+-----------------------------------------------------------+
 258
 259 create_instance_tests.py - CreateInstanceFromThreePartImage
 260 -----------------------------------------------------------
 261
 262 +-----------------------------------------------------+---------------+-----------------------------------------------------------+
 263 | Test Name                                           | API Versions  | Description                                               |
 264 +=====================================================+===============+===========================================================+
 265 | test_create_delete_instance_from_three_part_image   | Nova 2        | Tests to ensure that one can fire up an instance then     |
 266 |                                                     | Neutron 2     | delete it when using a 3-part image                       |
 267 +-----------------------------------------------------+---------------+-----------------------------------------------------------+
 268
 269 create_instance_tests.py - CreateInstancePubPrivNetTests
 270 --------------------------------------------------------
 271
 272 +---------------------------------------+---------------+-----------------------------------------------------------+
 273 | Test Name                             | API Versions  | Description                                               |
 274 +=======================================+===============+===========================================================+
 275 | test_dual_ports_dhcp                  | Nova 2        | Ensures that a VM with two ports/NICs can have its second |
 276 |                                       | Neutron 2     | NIC configured via SSH/Ansible after startup              |
 277 +---------------------------------------+---------------+-----------------------------------------------------------+
 278
 279 create_instance_tests.py - InstanceSecurityGroupTests
 280 -----------------------------------------------------
 281
 282 +---------------------------------------+---------------+-----------------------------------------------------------+
 283 | Test Name                             | API Versions  | Description                                               |
 284 +=======================================+===============+===========================================================+
 285 | test_add_security_group               | Nova 2        | Ensures that a VM instance can have security group added  |
 286 |                                       | Neutron 2     | to it while its running                                   |
 287 +---------------------------------------+---------------+-----------------------------------------------------------+
 288 | test_add_invalid_security_group       | Nova 2        | Ensures that a VM instance does not accept the addition of|
 289 |                                       | Neutron 2     | a security group that no longer exists                    |
 290 +---------------------------------------+---------------+-----------------------------------------------------------+
 291 | test_remove_security_group            | Nova 2        | Ensures that a VM instance accepts the removal of a       |
 292 |                                       | Neutron 2     | security group                                            |
 293 +---------------------------------------+---------------+-----------------------------------------------------------+
 294 | test_remove_security_group_never_added| Nova 2        | Ensures that a VM instance does not accept the removal of |
 295 |                                       | Neutron 2     | a security group that was never added in the first place  |
 296 +---------------------------------------+---------------+-----------------------------------------------------------+
 297 | test_add_same_security_group          | Nova 2        | Ensures that a VM instance does not add a security group  |
 298 |                                       | Neutron 2     | that has already been added to the instance               |
 299 +---------------------------------------+---------------+-----------------------------------------------------------+
 300
 301 ansible_utils_tests.py - AnsibleProvisioningTests
 302 -------------------------------------------------
 303
 304 +---------------------------------------+---------------+-----------------------------------------------------------+
 305 | Test Name                             | API Versions  | Description                                               |
 306 +=======================================+===============+===========================================================+
 307 | test_apply_simple_playbook            | Nova 2        | Ensures that an instance assigned with a floating IP will |
 308 |                                       | Neutron 2     | apply a simple Ansible playbook                           |
 309 +---------------------------------------+---------------+-----------------------------------------------------------+
 310 | test_apply_template_playbook          | Nova 2        | Ensures that an instance assigned with a floating IP will |
 311 |                                       | Neutron 2     | apply a Ansible playbook containing Jinga2 substitution   |
 312 |                                       |               | values                                                    |
 313 +---------------------------------------+---------------+-----------------------------------------------------------+