Merge "Fix Malformed Table in Integration Tests doc"
[snaps.git] / docs / how-to-use / IntegrationTests.rst
1 SNAPS OpenStack Integration Testing
2 ===================================
3
4 These tests are ones designed to be run within their own dynamically created project along with a newly generated user
5 account and generally require other OpenStack object creators.
6
7 The Test Classes
8 ================
9
10 create_security_group_tests.py - CreateSecurityGroupTests
11 ---------------------------------------------------------
12
13 +---------------------------------------+---------------+-----------------------------------------------------------+
14 | Test Name                             | API Versions  | Description                                               |
15 +=======================================+===============+===========================================================+
16 | test_create_group_without_rules       | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
17 |                                       | Neutron 2     | security group without any rules                          |
18 +---------------------------------------+---------------+-----------------------------------------------------------+
19 | test_create_group_admin_user_to_new   | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can be created   |
20 | _project                              | Neutron 2     | by the admin user and associated with a new project       |
21 +---------------------------------------+---------------+-----------------------------------------------------------+
22 | test_create_group_new_user_to_admin   | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can be created   |
23 | _project                              | Neutron 2     | by the new user and associated with the admin project     |
24 +---------------------------------------+---------------+-----------------------------------------------------------+
25 | test_create_delete_group              | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class clean() method   |
26 |                                       | Neutron 2     | will not raise an exception should the group be deleted by|
27 |                                       |               | some other process                                        |
28 +---------------------------------------+---------------+-----------------------------------------------------------+
29 | test_create_group_with_one_simple_rule| Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
30 |                                       | Neutron 2     | security group with a single simple rule                  |
31 +---------------------------------------+---------------+-----------------------------------------------------------+
32 | test_create_group_with_one_complex    | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
33 | _rule                                 | Neutron 2     | security group with a single complex rule                 |
34 +---------------------------------------+---------------+-----------------------------------------------------------+
35 | test_create_group_with_several_rules  | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup class can create a     |
36 |                                       | Neutron 2     | security group with several rules                         |
37 +---------------------------------------+---------------+-----------------------------------------------------------+
38 | test_add_rule                         | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#add_rule() method      |
39 |                                       | Neutron 2     | properly creates and associates the new rule              |
40 +---------------------------------------+---------------+-----------------------------------------------------------+
41 | test_remove_rule_by_id                | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#remove_rule() method   |
42 |                                       | Neutron 2     | properly deletes and disassociates the old rule via its ID|
43 +---------------------------------------+---------------+-----------------------------------------------------------+
44 | test_remove_rule_by_setting           | Keysone 2 & 3 | Ensures the OpenStackSecurityGroup#remove_rule() method   |
45 |                                       | Neutron 2     | properly deletes and disassociates the old rule via its   |
46 |                                       |               | setting object                                            |
47 +---------------------------------------+---------------+-----------------------------------------------------------+
48
49 create_image_tests.py - CreateImageSuccessTests
50 -----------------------------------------------
51
52 +---------------------------------------+---------------+-----------------------------------------------------------+
53 | Test Name                             | Glance API    | Description                                               |
54 +=======================================+===============+===========================================================+
55 | test_create_image_clean_url           | 1 & 2         | Ensures the OpenStackImage class can create an image from |
56 |                                       |               | a download URL location                                   |
57 +---------------------------------------+---------------+-----------------------------------------------------------+
58 | test_create_image_clean_url_properties| 1 & 2         | Ensures the OpenStackImage class can create an image from |
59 |                                       |               | a download URL location with custom properties            |
60 +---------------------------------------+---------------+-----------------------------------------------------------+
61 | test_create_image_clean_file          | 1 & 2         | Ensures the OpenStackImage class can create an image from |
62 |                                       |               | a locally sourced image file                              |
63 +---------------------------------------+---------------+-----------------------------------------------------------+
64 | test_create_delete_image              | 1 & 2         | Ensures the OpenStackImage.clean() method deletes an image|
65 |                                       |               | and does not raise an exception on subsequent calls to the|
66 |                                       |               | clean() method                                            |
67 +---------------------------------------+---------------+-----------------------------------------------------------+
68 | test_create_same_image                | 1 & 2         | Ensures the OpenStackImage.create() method does not create|
69 |                                       |               | another image when one already exists with the same name  |
70 +---------------------------------------+---------------+-----------------------------------------------------------+
71 | test_create_same_image_new_settings   | 1 & 2         | Tests the creation of an OpenStack image when the image   |
72 |                                       |               | already exists and the configuration only contains the    |
73 |                                       |               | the name.                                                 |
74 +---------------------------------------+---------------+-----------------------------------------------------------+
75
76 create_image_tests.py - CreateImageNegativeTests
77 ------------------------------------------------
78
79 +---------------------------------------+---------------+-----------------------------------------------------------+
80 | Test Name                             | Glance API    | Description                                               |
81 +=======================================+===============+===========================================================+
82 | test_bad_image_name                   | 1 & 2         | Ensures OpenStackImage.create() results in an Exception   |
83 |                                       |               | being raised when the ImageConfig.name attribute has    |
84 |                                       |               | not been set                                              |
85 +---------------------------------------+---------------+-----------------------------------------------------------+
86 | test_bad_image_url                    | 1 & 2         | Ensures OpenStackImage.create() results in an Exception   |
87 |                                       |               | being raised when the download URL is invalid             |
88 +---------------------------------------+---------------+-----------------------------------------------------------+
89 | test_bad_image_type                   | 1 & 2         | Ensures OpenStackImage.create() results in an Exception   |
90 |                                       |               | being raised when the image format is 'foo'               |
91 +---------------------------------------+---------------+-----------------------------------------------------------+
92 | test_bad_image_file                   | 1 & 2         | Ensures OpenStackImage.create() results in an Exception   |
93 |                                       |               | being raised when the image file does not exist           |
94 +---------------------------------------+---------------+-----------------------------------------------------------+
95
96 create_image_tests.py - CreateMultiPartImageTests
97 -------------------------------------------------
98
99 +----------------------------------------+---------------+-----------------------------------------------------------+
100 | Test Name                              | Glance API    | Description                                               |
101 +========================================+===============+===========================================================+
102 | test_create_three_part_image_from_url  | 1 & 2         | Ensures that a 3-part image can be created when each part |
103 |                                        |               | is being sourced from URLs                                |
104 +----------------------------------------+---------------+-----------------------------------------------------------+
105 | test_create_three_part_image_from_file | 1 & 2         | Ensures that a 3-part image can be created when each part |
106 | _3_creators                            |               | is being sourced from local files and 3 creators are used |
107 +----------------------------------------+---------------+-----------------------------------------------------------+
108 | test_create_three_part_image_from_url  | 1 & 2         | Ensures that a 3-part image can be created when each part |
109 | _3_creators                            |               | is being sourced from a URL and 3 creators are used       |
110 +----------------------------------------+---------------+-----------------------------------------------------------+
111
112 create_keypairs_tests.py - CreateKeypairsTests
113 ----------------------------------------------
114
115 +---------------------------------------+---------------+-----------------------------------------------------------+
116 | Test Name                             | Nova API      | Description                                               |
117 +=======================================+===============+===========================================================+
118 | test_create_keypair_only              | 2             | Ensures that a keypair object can be created simply by    |
119 |                                       |               | only configuring a name                                   |
120 +---------------------------------------+---------------+-----------------------------------------------------------+
121 | test_create_keypair_large_key         | 2             | Ensures that a keypair object can be created with a large |
122 |                                       |               | key of 10000 bytes                                        |
123 +---------------------------------------+---------------+-----------------------------------------------------------+
124 | test_create_delete_keypair            | 2             | Ensures that a keypair object is deleted via              |
125 |                                       |               | OpenStackKeypair.clean() and subsequent calls do not      |
126 |                                       |               | result in exceptions                                      |
127 +---------------------------------------+---------------+-----------------------------------------------------------+
128 | test_create_keypair_save_pub_only     | 2             | Ensures that a keypair object can be created when the only|
129 |                                       |               | the public key is cached to disk                          |
130 +---------------------------------------+---------------+-----------------------------------------------------------+
131 | test_create_keypair_save_both         | 2             | Ensures that a keypair object can be created when both the|
132 |                                       |               | public and private keys are cached to disk                |
133 +---------------------------------------+---------------+-----------------------------------------------------------+
134 | test_create_keypair_from_file         | 2             | Ensures that a keypair object can be created with an      |
135 |                                       |               | existing public key file                                  |
136 +---------------------------------------+---------------+-----------------------------------------------------------+
137
138 create_keypairs_tests.py - CreateKeypairsCleanupTests
139 -----------------------------------------------------
140
141 +---------------------------------------+---------------+-----------------------------------------------------------+
142 | Test Name                             | Nova API      | Description                                               |
143 +=======================================+===============+===========================================================+
144 | test_create_keypair_gen_files_delete_1| 2             | Ensures that new keypair files are deleted by default     |
145 |                                       |               | by OpenStackKeypair#clean()                               |
146 +---------------------------------------+---------------+-----------------------------------------------------------+
147 | test_create_keypair_gen_files_delete_2| 2             | Ensures that new keypair files are deleted by             |
148 |                                       |               | OpenStackKeypair#clean() when the settings delete_on_clean|
149 |                                       |               | attribute is set to True                                  |
150 +---------------------------------------+---------------+-----------------------------------------------------------+
151 | test_create_keypair_gen_files_keep    | 2             | Ensures that new keypair files are not deleted by         |
152 |                                       |               | OpenStackKeypair#clean()                                  |
153 +---------------------------------------+---------------+-----------------------------------------------------------+
154 | test_create_keypair_exist_files_keep  | 2             | Ensures that existing keypair files are not deleted by    |
155 |                                       |               | OpenStackKeypair#clean()                                  |
156 +---------------------------------------+---------------+-----------------------------------------------------------+
157 | test_create_keypair_exist_files_delete| 2             | Ensures that existing keypair files are deleted by        |
158 |                                       |               | OpenStackKeypair#clean()                                  |
159 +---------------------------------------+---------------+-----------------------------------------------------------+
160
161 create_network_tests.py - CreateNetworkSuccessTests
162 ---------------------------------------------------
163
164 +---------------------------------------+---------------+-----------------------------------------------------------+
165 | Test Name                             | Neutron API   | Description                                               |
166 +=======================================+===============+===========================================================+
167 | test_create_network_without_router    | 2             | Ensures that a network can be created via the             |
168 |                                       |               | OpenStackNetwork class without any routers                |
169 +---------------------------------------+---------------+-----------------------------------------------------------+
170 | test_create_delete_network            | 2             | Ensures that a router can be deleted via the              |
171 |                                       |               | OpenStackNetwork.clean() method                           |
172 +---------------------------------------+---------------+-----------------------------------------------------------+
173 | test_create_network_with_router       | 2             | Ensures that a network can be created via the             |
174 |                                       |               | OpenStackNetwork class with a router                      |
175 +---------------------------------------+---------------+-----------------------------------------------------------+
176 | test_create_networks_same_name        | 2             | Ensures that the OpenStackNetwork.create() method will not|
177 |                                       |               | create a network with the same name                       |
178 +---------------------------------------+---------------+-----------------------------------------------------------+
179 | test_create_networks_router_admin_user| 2             | Ensures that the networks, subnets, and routers can be    |
180 | _to_new_project                       |               | create created by an admin user and assigned to a new     |
181 |                                       |               | project ID                                                |
182 +---------------------------------------+---------------+-----------------------------------------------------------+
183 | test_create_networks_router_new_user  | 2             | Ensures that the networks, subnets, and routers can be    |
184 | _to_admin_project                     |               | create created by a new admin user and assigned to the    |
185 |                                       |               | 'admin' project ID                                        |
186 +---------------------------------------+---------------+-----------------------------------------------------------+
187
188 create_network_tests.py - CreateNetworkIPv6Tests
189 ------------------------------------------------
190
191 +---------------------------------------+---------------+-----------------------------------------------------------+
192 | Test Name                             | Neutron API   | Description                                               |
193 +=======================================+===============+===========================================================+
194 | test_create_network_one_ipv6_subnet   | 2             | Ensures that a network can be created with an IPv6 subnet |
195 +---------------------------------------+---------------+-----------------------------------------------------------+
196 | test_create_network_ipv4_ipv6_subnet  | 2             | Ensures that a network can be created with an IPv4 and    |
197 |                                       |               | IPv6 subnet                                               |
198 +---------------------------------------+---------------+-----------------------------------------------------------+
199
200 create_router_tests.py - CreateRouterSuccessTests
201 -------------------------------------------------
202
203 +---------------------------------------+---------------+-----------------------------------------------------------+
204 | Test Name                             | Neutron API   | Description                                               |
205 +=======================================+===============+===========================================================+
206 | test_create_router_vanilla            | 2             | Ensures that a router can be created via the              |
207 |                                       |               | OpenStackRouter class with minimal settings               |
208 +---------------------------------------+---------------+-----------------------------------------------------------+
209 | test_create_router_admin_user_to_new  | 2             | Ensures that a router can be created by an admin user and |
210 | _project                              |               | assigned to a new project                                 |
211 +---------------------------------------+---------------+-----------------------------------------------------------+
212 | test_create_router_new_user_to_admin  | 2             | Ensures that a router can be created by a new user and    |
213 | _project                              |               | assigned to the admin project                             |
214 +---------------------------------------+---------------+-----------------------------------------------------------+
215 | test_create_delete_router             | 2             | Ensures that a router can be deleted via the              |
216 |                                       |               | OpenStackRouter.clean() method                            |
217 +---------------------------------------+---------------+-----------------------------------------------------------+
218 | test_create_router_admin_state_false  | 2             | Ensures that a router can created with                    |
219 |                                       |               | admin_state_up = False                                    |
220 +---------------------------------------+---------------+-----------------------------------------------------------+
221 | test_create_router_admin_state_True   | 2             | Ensures that a router can created with                    |
222 |                                       |               | admin_state_up = True                                     |
223 +---------------------------------------+---------------+-----------------------------------------------------------+
224 | test_create_router_private_network    | 2             | Ensures that a router port can be created against a       |
225 |                                       |               | private network                                           |
226 +---------------------------------------+---------------+-----------------------------------------------------------+
227 | test_create_router_external_network   | 2             | Ensures that a router can be created that is connected to |
228 |                                       |               | both external and private internal networks               |
229 +---------------------------------------+---------------+-----------------------------------------------------------+
230
231 create_router_tests.py - CreateRouterNegativeTests
232 --------------------------------------------------
233
234 +----------------------------------------+---------------+-----------------------------------------------------------+
235 | Test Name                              | Neutron API   | Description                                               |
236 +========================================+===============+===========================================================+
237 | test_create_router_noname              | 2             | Ensures that an exception is raised when attempting to    |
238 |                                        |               | create a router without a name                            |
239 +----------------------------------------+---------------+-----------------------------------------------------------+
240 | test_create_router_invalid_gateway_name| 2             | Ensures that an exception is raised when attempting to    |
241 |                                        |               | create a router to an external network that does not exist|
242 +----------------------------------------+---------------+-----------------------------------------------------------+
243
244 create_qos_tests.py - CreateQoSTests
245 ------------------------------------
246
247 +----------------------------------------+---------------+-----------------------------------------------------------+
248 | Test Name                              |  Cinder API   | Description                                               |
249 +========================================+===============+===========================================================+
250 | test_create_qos                        | 2 & 3         | Tests the creation of a QoS Spec with the class           |
251 |                                        |               | OpenStackQoS                                              |
252 +----------------------------------------+---------------+-----------------------------------------------------------+
253 | test_create_delete_qos                 | 2 & 3         | Tests the creation of a QoS Spec with the class           |
254 |                                        |               | OpenStackQoS, its deletion with cinder_utils.py the       |
255 |                                        |               | the attempts to use the clean() method to ensure an       |
256 |                                        |               | exception is not called                                   |
257 +----------------------------------------+---------------+-----------------------------------------------------------+
258 | test_create_same_qos                   | 2 & 3         | Tests the creation of a QoS Spec with the class           |
259 |                                        |               | OpenStackQoS then instantiates another OpenStackQoS       |
260 |                                        |               | object with the same configuration to ensure the second   |
261 |                                        |               | instance returns the ID of the original                   |
262 +----------------------------------------+---------------+-----------------------------------------------------------+
263
264 create_volume_type_tests.py - CreateSimpleVolumeTypeSuccessTests
265 ----------------------------------------------------------------
266
267 +----------------------------------------+---------------+-----------------------------------------------------------+
268 | Test Name                              |  Cinder API   | Description                                               |
269 +========================================+===============+===========================================================+
270 | test_create_volume_type                | 2 & 3         | Tests the creation of a Volume Type with the class        |
271 |                                        |               | OpenStackVolumeType                                       |
272 +----------------------------------------+---------------+-----------------------------------------------------------+
273 | test_create_delete_volume_type         | 2 & 3         | Tests the creation of a Volume Type with the class        |
274 |                                        |               | OpenStackVolumeType, its deletion with cinder_utils.py,   |
275 |                                        |               | then attempts to use the clean() method to ensure an      |
276 |                                        |               | exception is not raised                                   |
277 +----------------------------------------+---------------+-----------------------------------------------------------+
278 | test_create_same_volume_type           | 2 & 3         | Tests the creation of a Volume Type with the class        |
279 |                                        |               | OpenStackVolumeType then instantiates another             |
280 |                                        |               | OpenStackVolumeType object with the same configuration to |
281 |                                        |               | ensure the second instance returns the ID of the original |
282 +----------------------------------------+---------------+-----------------------------------------------------------+
283
284 create_volume_type_tests.py - CreateSimpleVolumeTypeComplexTests
285 ----------------------------------------------------------------
286
287 +-----------------------------------------+---------------+-----------------------------------------------------------+
288 | Test Name                               |  Cinder API   | Description                                               |
289 +=========================================+===============+===========================================================+
290 | test_volume_type_with_qos               | 2 & 3         | Tests the creation of a Volume Type with the class        |
291 |                                         |               | OpenStackVolumeType with a QoSSpec                        |
292 +-----------------------------------------+---------------+-----------------------------------------------------------+
293 | test_volume_type_with_encryption        | 2 & 3         | Tests the creation of a Volume Type with the class        |
294 |                                         |               | OpenStackVolumeType with encryption                       |
295 +-----------------------------------------+---------------+-----------------------------------------------------------+
296 | test_volume_type_with_qos_and_encryption| 2 & 3         | Tests the creation of a Volume Type with the class        |
297 |                                         |               | OpenStackVolumeType with encryption and QoS Spec          |
298 +-----------------------------------------+---------------+-----------------------------------------------------------+
299
300 create_volume_tests.py - CreateSimpleVolumeSuccessTests
301 -------------------------------------------------------
302
303 +----------------------------------------+---------------+-----------------------------------------------------------+
304 | Test Name                              |  Cinder API   | Description                                               |
305 +========================================+===============+===========================================================+
306 | test_create_volume_simple              | 2 & 3         | Tests the creation of a Volume Type with the class        |
307 |                                        |               | OpenStackVolume                                           |
308 +----------------------------------------+---------------+-----------------------------------------------------------+
309 | test_create_delete_volume              | 2 & 3         | Tests the creation of a Volume with the class             |
310 |                                        |               | OpenStackVolume, its deletion with cinder_utils.py, then  |
311 |                                        |               | attempts to use the clean() method to ensure an           |
312 |                                        |               | exception is not raised                                   |
313 +----------------------------------------+---------------+-----------------------------------------------------------+
314 | test_create_same_volume                | 2 & 3         | Tests the creation of a Volume with the class             |
315 |                                        |               | OpenStackVolume then instantiates another                 |
316 |                                        |               | OpenStackVolume object with the same configuration to     |
317 |                                        |               | ensure the second instance returns the ID of the original |
318 +----------------------------------------+---------------+-----------------------------------------------------------+
319
320 create_volume_tests.py - CreateSimpleVolumeFailureTests
321 -------------------------------------------------------
322
323 +----------------------------------------+---------------+-----------------------------------------------------------+
324 | Test Name                              |  Cinder API   | Description                                               |
325 +========================================+===============+===========================================================+
326 | test_create_volume_bad_size            | 2 & 3         | Tests to ensure that attempting to create a volume with a |
327 |                                        |               | size of -1 raises a BadRequest exception                  |
328 +----------------------------------------+---------------+-----------------------------------------------------------+
329 | test_create_volume_bad_type            | 2 & 3         | Tests to ensure that attempting to create a volume with a |
330 |                                        |               | type that does not exist raises a NotFound exception      |
331 +----------------------------------------+---------------+-----------------------------------------------------------+
332 | test_create_volume_bad_image           | 2 & 3         | Tests to ensure that attempting to create a volume with an|
333 |                                        |               | image that does not exist raises a BadRequest exception   |
334 +----------------------------------------+---------------+-----------------------------------------------------------+
335 | test_create_volume_bad_zone            | 2 & 3         | Tests to ensure that attempting to create a volume with an|
336 |                                        |               | invalid availability zone raises a BadRequest exception   |
337 +----------------------------------------+---------------+-----------------------------------------------------------+
338
339 create_volume_tests.py - CreateVolumeWithTypeTests
340 --------------------------------------------------
341
342 +----------------------------------------+---------------+-----------------------------------------------------------+
343 | Test Name                              |  Cinder API   | Description                                               |
344 +========================================+===============+===========================================================+
345 | test_bad_volume_type                   | 2 & 3         | Tests to ensure the creation of a Volume with the         |
346 |                                        |               | OpenStackVolume#create() method raises a NotFound         |
347 |                                        |               | exception when attempting to apply a VolumeType that does |
348 |                                        |               | not exist                                                 |
349 +----------------------------------------+---------------+-----------------------------------------------------------+
350 | test_valid_volume_type                 | 2 & 3         | Tests to ensure the creation of a Volume with the         |
351 |                                        |               | OpenStackVolume#create() method properly creates the      |
352 |                                        |               | volume when associating with a valid VolumeType           |
353 +----------------------------------------+---------------+-----------------------------------------------------------+
354
355 create_volume_tests.py - CreateVolumeWithImageTests
356 ---------------------------------------------------
357
358 +----------------------------------------+---------------+-----------------------------------------------------------+
359 | Test Name                              |  Cinder API   | Description                                               |
360 +========================================+===============+===========================================================+
361 | test_bad_image_name                    | 2 & 3         | Tests to ensure the creation of a Volume with the         |
362 |                                        |               | OpenStackVolume#create() method raises a BadRequest       |
363 |                                        |               | exception when attempting to apply an image that does not |
364 |                                        |               | exist                                                     |
365 +----------------------------------------+---------------+-----------------------------------------------------------+
366 | test_valid_volume_image                | 2 & 3         | Tests to ensure the creation of a Volume with the         |
367 |                                        |               | OpenStackVolume#create() method properly creates the      |
368 |                                        |               | volume when associating with a valid image                |
369 +----------------------------------------+---------------+-----------------------------------------------------------+
370
371 create_stack_tests.py - CreateStackSuccessTests
372 -----------------------------------------------
373
374 +---------------------------------------+---------------+-----------------------------------------------------------+
375 | Test Name                             |   Heat API    | Description                                               |
376 +=======================================+===============+===========================================================+
377 | test_create_stack_template_file       | 1-3           | Ensures that a Heat stack can be created with a file-based|
378 |                                       |               | Heat template file                                        |
379 +---------------------------------------+---------------+-----------------------------------------------------------+
380 | test_create_stack_template_dict       | 1-3           | Ensures that a Heat stack can be created with a dictionary|
381 |                                       |               | Heat template                                             |
382 +---------------------------------------+---------------+-----------------------------------------------------------+
383 | test_create_delete_stack              | 1-3           | Ensures that a Heat stack can be created and deleted      |
384 |                                       |               | while having clean() called 2x without an exception       |
385 +---------------------------------------+---------------+-----------------------------------------------------------+
386 | test_create_same_stack                | 1-3           | Ensures that a Heat stack with the same name cannot be    |
387 |                                       |               | created 2x                                                |
388 +---------------------------------------+---------------+-----------------------------------------------------------+
389 | test_retrieve_network_creators        | 1-3           | Ensures that an OpenStackHeatStack instance can return an |
390 |                                       |               | OpenStackNetwork instance configured as deployed          |
391 +---------------------------------------+---------------+-----------------------------------------------------------+
392 | test_retrieve_vm_inst_creators        | 1-3           | Ensures that an OpenStackHeatStack instance can return an |
393 |                                       |               | OpenStackVmInstance instance configured as deployed       |
394 +---------------------------------------+---------------+-----------------------------------------------------------+
395
396 create_stack_tests.py - CreateStackVolumeTests
397 ----------------------------------------------
398
399 +---------------------------------------+---------------+-----------------------------------------------------------+
400 | Test Name                             |   Heat API    | Description                                               |
401 +=======================================+===============+===========================================================+
402 | test_retrieve_volume_creator          | 1-3           | Ensures that an OpenStackHeatStack instance can return a  |
403 |                                       |               | OpenStackVolume instance that it was responsible for      |
404 |                                       |               | deploying                                                 |
405 +---------------------------------------+---------------+-----------------------------------------------------------+
406 | test_retrieve_volume_type_creator     | 1-3           | Ensures that an OpenStackHeatStack instance can return a  |
407 |                                       |               | OpenStackVolumeType instance that it was responsible for  |
408 |                                       |               | deploying                                                 |
409 +---------------------------------------+---------------+-----------------------------------------------------------+
410
411 create_stack_tests.py - CreateStackFloatingIpTests
412 --------------------------------------------------
413
414 +---------------------------------------+---------------+-----------------------------------------------------------+
415 | Test Name                             |   Heat API    | Description                                               |
416 +=======================================+===============+===========================================================+
417 | test_connect_via_ssh_heat_vm          | 1             | Ensures that an OpenStackHeatStack instance can create a  |
418 |                                       |               | VM with a floating IP that can be accessed via            |
419 |                                       |               | OpenStackVmInstance                                       |
420 +---------------------------------------+---------------+-----------------------------------------------------------+
421
422 create_stack_tests.py - CreateStackNestedResourceTests
423 ------------------------------------------------------
424
425 +---------------------------------------+---------------+-----------------------------------------------------------+
426 | Test Name                             |   Heat API    | Description                                               |
427 +=======================================+===============+===========================================================+
428 | test_nested                           | 1             | Ensures that an OpenStackHeatStack with an external       |
429 |                                       |               | resource file with VMs with floating IPs can be accessed  |
430 |                                       |               | in the class OpenStackVmInstance and return the associated|
431 |                                       |               | initialized OpenStackVmInstance objects                   |
432 +---------------------------------------+---------------+-----------------------------------------------------------+
433
434 create_stack_tests.py - CreateStackRouterTests
435 ----------------------------------------------
436
437 +---------------------------------------+---------------+-----------------------------------------------------------+
438 | Test Name                             |   Heat API    | Description                                               |
439 +=======================================+===============+===========================================================+
440 | test_retrieve_router_creator          | 1             | Ensures that an OpenStackHeatStack instance can return a  |
441 |                                       |               | OpenStackRouter instance that it was responsible for      |
442 |                                       |               | deploying                                                 |
443 +---------------------------------------+---------------+-----------------------------------------------------------+
444
445 create_stack_tests.py - CreateStackFlavorTests
446 ----------------------------------------------
447
448 +---------------------------------------+---------------+-----------------------------------------------------------+
449 | Test Name                             |   Heat API    | Description                                               |
450 +=======================================+===============+===========================================================+
451 | test_retrieve_flavor_creator          | 1-3           | Ensures that an OpenStackHeatStack instance can return a  |
452 |                                       |               | OpenStackFlavor instance that it was responsible for      |
453 |                                       |               | deploying                                                 |
454 +---------------------------------------+---------------+-----------------------------------------------------------+
455
456 create_stack_tests.py - CreateStackKeypairTests
457 -----------------------------------------------
458
459 +---------------------------------------+---------------+-----------------------------------------------------------+
460 | Test Name                             |   Heat API    | Description                                               |
461 +=======================================+===============+===========================================================+
462 | test_retrieve_keypair_creator         | 1-3           | Ensures that an OpenStackHeatStack instance can return a  |
463 |                                       |               | OpenStackKeypair instance that it was responsible for     |
464 |                                       |               | deploying                                                 |
465 +---------------------------------------+---------------+-----------------------------------------------------------+
466
467 create_stack_tests.py - CreateStackSecurityGroupTests
468 -----------------------------------------------------
469
470 +---------------------------------------+---------------+-----------------------------------------------------------+
471 | Test Name                             |   Heat API    | Description                                               |
472 +=======================================+===============+===========================================================+
473 | test_retrieve_security_group_creator  | 1-3           | Ensures that an OpenStackHeatStack instance can return a  |
474 |                                       |               | OpenStackSecurityGroup instance that it was responsible   |
475 |                                       |               | for deploying                                             |
476 +---------------------------------------+---------------+-----------------------------------------------------------+
477
478 create_stack_tests.py - CreateComplexStackTests
479 -----------------------------------------------
480
481 +---------------------------------------+---------------+-----------------------------------------------------------+
482 | Test Name                             |   Heat API    | Description                                               |
483 +=======================================+===============+===========================================================+
484 | test_connect_via_ssh_heat_vm          | 1-3           | Ensures that two OpenStackHeatStack instances can return  |
485 |                                       |               | OpenStackVmInstance instances one configured with a       |
486 |                                       |               | floating IP and keypair and can be access via SSH         |
487 +---------------------------------------+---------------+-----------------------------------------------------------+
488
489 create_stack_tests.py - CreateStackNegativeTests
490 ------------------------------------------------
491
492 +----------------------------------------+---------------+-----------------------------------------------------------+
493 | Test Name                              |   Heat API    | Description                                               |
494 +========================================+===============+===========================================================+
495 | test_missing_dependencies              | 1-3           | Ensures that a Heat template fails to deploy when expected|
496 |                                        |               | dependencies are missing                                  |
497 +----------------------------------------+---------------+-----------------------------------------------------------+
498 | test_bad_stack_file                    | 1-3           | Ensures that a Heat template fails to deploy when the Heat|
499 |                                        |               | template file does not exist                              |
500 +----------------------------------------+---------------+-----------------------------------------------------------+
501
502 create_stack_tests.py - CreateStackFailureTests
503 -----------------------------------------------
504
505 +----------------------------------------+---------------+-----------------------------------------------------------+
506 | Test Name                              |   Heat API    | Description                                               |
507 +========================================+===============+===========================================================+
508 | test_stack_failure                     | 1-3           | Ensures that a Heat template fails to deploy when expected|
509 |                                        |               | dependencies are missing                                  |
510 +----------------------------------------+---------------+-----------------------------------------------------------+
511
512 create_instance_tests.py - CreateInstanceSimpleTests
513 ----------------------------------------------------
514
515 +---------------------------------------+---------------+-----------------------------------------------------------+
516 | Test Name                             | API Versions  | Description                                               |
517 +=======================================+===============+===========================================================+
518 | test_create_delete_instance           | Nova 2        | Ensures that the OpenStackVmInstance.clean() method       |
519 |                                       | Neutron 2     | deletes the instance                                      |
520 +---------------------------------------+---------------+-----------------------------------------------------------+
521
522 create_instance_tests.py - SimpleHealthCheck
523 --------------------------------------------
524
525 +---------------------------------------+---------------+-----------------------------------------------------------+
526 | Test Name                             | API Versions  | Description                                               |
527 +=======================================+===============+===========================================================+
528 | test_check_vm_ip_dhcp                 | Nova 2        | Tests the creation of an OpenStack instance with a single |
529 |                                       | Neutron 2     | port and it's assigned IP address                         |
530 +---------------------------------------+---------------+-----------------------------------------------------------+
531
532 create_instance_tests.py - CreateInstanceTwoNetTests
533 ----------------------------------------------------
534
535 +---------------------------------------+---------------+-----------------------------------------------------------+
536 | Test Name                             | API Versions  | Description                                               |
537 +=======================================+===============+===========================================================+
538 | test_ping_via_router                  | Nova 2        | Tests the ability of two VMs on different private overlay |
539 |                                       | Neutron 2     | networks tied together with a router to ping each other   |
540 +---------------------------------------+---------------+-----------------------------------------------------------+
541
542 create_instance_tests.py - CreateInstanceSingleNetworkTests
543 -----------------------------------------------------------
544
545 +---------------------------------------+---------------+-----------------------------------------------------------+
546 | Test Name                             | API Versions  | Description                                               |
547 +=======================================+===============+===========================================================+
548 | test_single_port_static               | Nova 2        | Ensures that an instance with a single port/NIC with a    |
549 |                                       | Neutron 2     | static IP can be created                                  |
550 +---------------------------------------+---------------+-----------------------------------------------------------+
551 | test_ssh_client_fip_before_active     | Nova 2        | Ensures that an instance can be reached over SSH when the |
552 |                                       | Neutron 2     | floating IP is assigned prior to the VM becoming ACTIVE   |
553 +---------------------------------------+---------------+-----------------------------------------------------------+
554 | test_ssh_client_fip_after_active      | Nova 2        | Ensures that an instance can be reached over SSH when the |
555 |                                       | Neutron 2     | floating IP is assigned after to the VM becoming ACTIVE   |
556 +---------------------------------------+---------------+-----------------------------------------------------------+
557 | test_ssh_client_fip_after_init        | Nova 2        | Ensures that an instance can have a floating IP assigned  |
558 |                                       | Neutron 2     | added after initialization                                |
559 +---------------------------------------+---------------+-----------------------------------------------------------+
560 | test_ssh_client_fip_reverse_engineer  | Nova 2        | Ensures that an instance can be reverse engineered and    |
561 |                                       | Neutron 2     | allows for a floating IP to be added after initialization |
562 +---------------------------------------+---------------+-----------------------------------------------------------+
563 | test_ssh_client_fip_after_reboot      | Nova 2        | Ensures that an instance can be reached over SSH after    |
564 |                                       | Neutron 2     | a reboot call has been issued                             |
565 +---------------------------------------+---------------+-----------------------------------------------------------+
566 | test_ssh_client_fip_second_creator    | Nova 2        | Ensures that an instance can be reached over SSH via a    |
567 |                                       | Neutron 2     | second identical creator object                           |
568 +---------------------------------------+---------------+-----------------------------------------------------------+
569
570 create_instance_tests.py - CreateInstancePortManipulationTests
571 --------------------------------------------------------------
572
573 +---------------------------------------+---------------+-----------------------------------------------------------+
574 | Test Name                             | API Versions  | Description                                               |
575 +=======================================+===============+===========================================================+
576 | test_set_custom_valid_ip_one_subnet   | Nova 2        | Ensures that an instance's can have a valid static IP is  |
577 |                                       | Neutron 2     | properly assigned                                         |
578 +---------------------------------------+---------------+-----------------------------------------------------------+
579 | test_set_custom_invalid_ip_one_subnet | Nova 2        | Ensures that an instance's port with an invalid static IP |
580 |                                       | Neutron 2     | raises an exception                                       |
581 +---------------------------------------+---------------+-----------------------------------------------------------+
582 | test_set_custom_valid_mac             | Nova 2        | Ensures that an instance's port can have a valid MAC      |
583 |                                       | Neutron 2     | address properly assigned                                 |
584 +---------------------------------------+---------------+-----------------------------------------------------------+
585 | test_set_custom_invalid_mac           | Nova 2        | Ensures that an instance's port with an invalid MAC       |
586 |                                       | Neutron 2     | address raises and exception                              |
587 +---------------------------------------+---------------+-----------------------------------------------------------+
588 | test_set_custom_mac_and_ip            | Nova 2        | Ensures that an instance's port with a valid static IP and|
589 |                                       | Neutron 2     | MAC are properly assigned                                 |
590 +---------------------------------------+---------------+-----------------------------------------------------------+
591 | test_set_allowed_address_pairs        | Nova 2        | Ensures the configured allowed_address_pairs is properly  |
592 |                                       | Neutron 2     | set on a VMs port                                         |
593 +---------------------------------------+---------------+-----------------------------------------------------------+
594 | test_set_allowed_address_pairs_bad_mac| Nova 2        | Ensures the port cannot be created when a bad MAC address |
595 |                                       | Neutron 2     | format is used in the allowed_address_pairs port attribute|
596 +---------------------------------------+---------------+-----------------------------------------------------------+
597 | test_set_allowed_address_pairs_bad_ip | Nova 2        | Ensures the port cannot be created when a bad IP address  |
598 |                                       | Neutron 2     | format is used in the allowed_address_pairs port attribute|
599 +---------------------------------------+---------------+-----------------------------------------------------------+
600
601 create_instance_tests.py - CreateInstanceOnComputeHost
602 ------------------------------------------------------
603
604 +---------------------------------------+---------------+-----------------------------------------------------------+
605 | Test Name                             | API Versions  | Description                                               |
606 +=======================================+===============+===========================================================+
607 | test_deploy_vm_to_each_compute_node   | Nova 2        | Tests to ensure that one can fire up an instance on each  |
608 |                                       | Neutron 2     | active compute node                                       |
609 +---------------------------------------+---------------+-----------------------------------------------------------+
610
611 create_instance_tests.py - CreateInstanceFromThreePartImage
612 -----------------------------------------------------------
613
614 +-----------------------------------------------------+---------------+-----------------------------------------------------------+
615 | Test Name                                           | API Versions  | Description                                               |
616 +=====================================================+===============+===========================================================+
617 | test_create_delete_instance_from_three_part_image   | Nova 2        | Tests to ensure that one can fire up an instance then     |
618 |                                                     | Neutron 2     | delete it when using a 3-part image                       |
619 +-----------------------------------------------------+---------------+-----------------------------------------------------------+
620
621 create_instance_tests.py - CreateInstanceIPv6NetworkTests (Staging)
622 -------------------------------------------------------------------
623
624 +---------------------------------------+---------------+-----------------------------------------------------------+
625 | Test Name                             | API Versions  | Description                                               |
626 +=======================================+===============+===========================================================+
627 | test_v4fip_v6overlay                  | Nova 2        | Expects a BadRequest exception to be raised when          |
628 |                                       | Neutron 2     | attempting to add an IPv4 floating IP to a VM with an IPv6|
629 |                                       |               | port                                                      |
630 +---------------------------------------+---------------+-----------------------------------------------------------+
631 | test_fip_v4and6_overlay               | Nova 2        | Connects to a VM via a floating IP joined to a port that  |
632 |                                       | Neutron 2     | has been confiured with both IPv4 and IPv6 addresses      |
633 +---------------------------------------+---------------+-----------------------------------------------------------+
634
635 create_instance_tests.py - InstanceSecurityGroupTests
636 -----------------------------------------------------
637
638 +---------------------------------------+---------------+-----------------------------------------------------------+
639 | Test Name                             | API Versions  | Description                                               |
640 +=======================================+===============+===========================================================+
641 | test_add_security_group               | Nova 2        | Ensures that a VM instance can have security group added  |
642 |                                       | Neutron 2     | to it while its running                                   |
643 +---------------------------------------+---------------+-----------------------------------------------------------+
644 | test_add_invalid_security_group       | Nova 2        | Ensures that a VM instance does not accept the addition of|
645 |                                       | Neutron 2     | a security group that no longer exists                    |
646 +---------------------------------------+---------------+-----------------------------------------------------------+
647 | test_remove_security_group            | Nova 2        | Ensures that a VM instance accepts the removal of a       |
648 |                                       | Neutron 2     | security group                                            |
649 +---------------------------------------+---------------+-----------------------------------------------------------+
650 | test_remove_security_group_never_added| Nova 2        | Ensures that a VM instance does not accept the removal of |
651 |                                       | Neutron 2     | a security group that was never added in the first place  |
652 +---------------------------------------+---------------+-----------------------------------------------------------+
653 | test_add_same_security_group          | Nova 2        | Ensures that a VM instance does not add a security group  |
654 |                                       | Neutron 2     | that has already been added to the instance               |
655 +---------------------------------------+---------------+-----------------------------------------------------------+
656
657 create_instance_tests.py - CreateInstanceVolumeTests
658 ----------------------------------------------------
659
660 +---------------------------------------+---------------+-----------------------------------------------------------+
661 | Test Name                             | API Versions  | Description                                               |
662 +=======================================+===============+===========================================================+
663 | test_create_instance_with_one_volume  | Nova 2        | Ensures that a VM instance can have one volume attached   |
664 |                                       | Cinder 2 & 3  | to it                                                     |
665 +---------------------------------------+---------------+-----------------------------------------------------------+
666 | test_create_instance_with_two_volumes | Nova 2        | Ensures that a VM instance can have two volumes attached  |
667 |                                       | Cinder 2 & 3  | to it                                                     |
668 +---------------------------------------+---------------+-----------------------------------------------------------+
669
670 ansible_utils_tests.py - AnsibleProvisioningTests
671 -------------------------------------------------
672
673 +---------------------------------------+---------------+-----------------------------------------------------------+
674 | Test Name                             | API Versions  | Description                                               |
675 +=======================================+===============+===========================================================+
676 | test_apply_simple_playbook            | Nova 2        | Ensures that an instance assigned with a floating IP will |
677 |                                       | Neutron 2     | apply a simple Ansible playbook                           |
678 +---------------------------------------+---------------+-----------------------------------------------------------+
679 | test_apply_template_playbook          | Nova 2        | Ensures that an instance assigned with a floating IP will |
680 |                                       | Neutron 2     | apply a Ansible playbook containing Jinga2 substitution   |
681 |                                       |               | values                                                    |
682 +---------------------------------------+---------------+-----------------------------------------------------------+
683
684 cluster_template_tests.py - CreateClusterTemplateTests
685 ------------------------------------------------------
686
687 +----------------------------------------+---------------+-----------------------------------------------------------+
688 | Test Name                              |  Magnum API   | Description                                               |
689 +========================================+===============+===========================================================+
690 | test_create_cluster_template           | 1             | Tests the creation of a Cluster template with the class   |
691 |                                        |               | OpenStackClusterTemplate                                  |
692 +----------------------------------------+---------------+-----------------------------------------------------------+
693 | test_create_delete_cluster_template    | 1             | Tests the creation and deletiong of a Cluster template    |
694 |                                        |               | with the class OpenStackClusterTemplate                   |
695 +----------------------------------------+---------------+-----------------------------------------------------------+
696 | test_create_same_cluster_template      | 1             | Tests the creation of a Cluster template 2x using the same|
697 |                                        |               | config object to ensure it was only created once          |
698 +----------------------------------------+---------------+-----------------------------------------------------------+
699 | test_create_cluster_template_bad_flavor| 1             | Tests to ensure OpenStackClusterTemplate#create() will    |
700 |                                        |               | raise an exception when the flavor is invalid             |
701 +----------------------------------------+---------------+-----------------------------------------------------------+
702 | test_create_cluster_template_bad_master| 1             | Tests to ensure OpenStackClusterTemplate#create() will    |
703 | _flavor                                |               | raise an exception when the master flavor is invalid      |
704 +----------------------------------------+---------------+-----------------------------------------------------------+
705 | test_create_cluster_template_bad_image | 1             | Tests to ensure OpenStackClusterTemplate#create() will    |
706 |                                        |               | raise an exception when the image is invalid              |
707 +----------------------------------------+---------------+-----------------------------------------------------------+
708 | test_create_cluster_template_bad       | 1             | Tests to ensure OpenStackClusterTemplate#create() will    |
709 | _network_driver                        |               | raise an exception when the network driver is invalid     |
710 +----------------------------------------+---------------+-----------------------------------------------------------+
711 | test_create_cluster_template_bad       | 1             | Tests to ensure OpenStackClusterTemplate#create() will    |
712 | _volume_driver                         |               | raise an exception when the volume driver is invalid      |
713 +----------------------------------------+---------------+-----------------------------------------------------------+
714