The compliance and certification test cases can be defined under the ``/dovetail/cert``
directory, which is defined in yaml format.
-A sample file named ``basic.yml`` is provided as follows:
+A sample file named ``compliance_set.yml`` is provided as follows:
::
- certification_basic:
- name: certification_basic
- testcase_list:
+ certification_compliance_set:
+ name: certification_compliance_set
+ testcases_list:
- dovetail.ipv6.tc001
The testcase listed here is dovetail.ipv6.tc001, defined within ``dovetail/testcase``.
Note: if a new test case yaml file is created, its name should start with ``certification_``,
-in similiar fashion as the sample file ``certification_basic``.
+in similiar fashion as the sample file ``certification_compliance_set``.
Running Dovetail tool
---------------------
::
- python run.py --scenario basic
+ python run.py --scenario compliance_set
-The value ``basic`` passed to the ``scenario`` flag can be replaced with the test cases yaml file.
-If not argument is given, the basic scenario will be run as the default.
+The value ``compliance_set`` passed to the ``scenario`` flag can be replaced with the test cases yaml file.
+If not argument is given, the compliance_set scenario will be run as the default.
Running Dovetail in a Docker container
########################################
Inside the container the following commands can be executed to trigger the testcases ::
cd /home/opnfv/dovetail/dovetail
- python run.py --scenario basic
+ python run.py --scenario compliance_set
Results Output
###############
+++ /dev/null
-certification_basic:
- name: certification_basic
- testcase_list:
- - dovetail.ipv6.tc001
--- /dev/null
+certification_compliance_set:
+ name: certification_compliance_set
+ testcases_list:
+ # Temporarily, one test case kept here as default to run
+ # for use of software development/debug
+ # TO DO: will amend when compliance set is settled
+ - dovetail.ipv6.tc001
--- /dev/null
+certification_proposed_tests:
+ name: certification_proposed_tests
+ testcases_list:
+ # TO DO: will adjust the dovetail tool to support in later patches
+ # run.py --name1 {**/proposed/compliance} --name2 {**/vim/ipv6,etc}
+ # vim operations
+ - dovetail.vimops.tc001
+ - dovetail.vimops.tc002
+ - dovetail.vimops.tc003
+ - dovetail.vimops.tc004
+ - dovetail.vimops.tc005
+ - dovetail.vimops.tc006
+ # ipv6
+ - dovetail.ipv6.tc001
+ - dovetail.ipv6.tc002
+ - dovetail.ipv6.tc003
+ - dovetail.ipv6.tc004
+ - dovetail.ipv6.tc005
+ - dovetail.ipv6.tc006
+ - dovetail.ipv6.tc007
+ - dovetail.ipv6.tc008
+ - dovetail.ipv6.tc009
+ - dovetail.ipv6.tc010
+ - dovetail.ipv6.tc011
+ - dovetail.ipv6.tc012
+ - dovetail.ipv6.tc013
+ - dovetail.ipv6.tc014
+ - dovetail.ipv6.tc015
+ - dovetail.ipv6.tc016
+ - dovetail.ipv6.tc017
+ - dovetail.ipv6.tc018
+ - dovetail.ipv6.tc010
+ - dovetail.ipv6.tc019
+ - dovetail.ipv6.tc020
+ - dovetail.ipv6.tc021
+ - dovetail.ipv6.tc022
+ - dovetail.ipv6.tc023
+ - dovetail.ipv6.tc024
+ - dovetail.ipv6.tc025
+ # nfvi compute/network/storage
+ - dovetail.nfvi.tc001
+ - dovetail.nfvi.tc002
flags:
- '--scenario'
- '-s'
- default: 'basic'
+ default: 'compliance_set'
help: 'certification scenario.'
| report |\n'
report += split_line
report += '|scenario: %s\n' % scenario_yaml['name']
- for testcase_name in scenario_yaml['testcase_list']:
+ for testcase_name in scenario_yaml['testcases_list']:
testcase = Testcase.get(testcase_name)
if testcase is None:
report += '| [testcase]: %s\t\t\t\t[Undefined]\n' % \
def run_test(scenario):
- for testcase_name in scenario['testcase_list']:
+ for testcase_name in scenario['testcases_list']:
logger.info('>>[testcase]: %s' % (testcase_name))
testcase = Testcase.get(testcase_name)
if testcase is None:
dovetail.ipv6.tc001:
name: dovetail.ipv6.tc001
- objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation
+ objective: Bulk creation and deletion of IPv6 networks, ports and subnets
scripts:
type: functest
testcase: tempest_smoke_serial
--- /dev/null
+dovetail.ipv6.tc002:
+ name: dovetail.ipv6.tc002
+ objective: VIM ipv6 operations, to create/update/delete an IPv6 network and subnet
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_networks.NetworksIpV6Test.test_create_update_delete_network_subnet
+ - tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_create_update_delete_network_subnet
--- /dev/null
+dovetail.ipv6.tc003:
+ name: dovetail.ipv6.tc003
+ objective: VIM ipv6 operations, to check external network visibility
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility
+ - tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_external_network_visibility
--- /dev/null
+dovetail.ipv6.tc004:
+ name: dovetail.ipv6.tc004
+ objective: VIM ipv6 operations, to list IPv6 networks and subnets of a tenant
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_networks.NetworksIpV6Test.test_list_networks
+ - tempest.api.network.test_networks.NetworksIpV6Test.test_list_subnets
+ - tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_list_networks
+ - tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_list_subnets
--- /dev/null
+dovetail.ipv6.tc005:
+ name: dovetail.ipv6.tc005
+ objective: VIM ipv6 operations, to show information of an IPv6 network and subnet
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_networks.NetworksIpV6Test.test_show_network
+ - tempest.api.network.test_networks.NetworksIpV6Test.test_show_subnet
+ - tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_show_network
+ - tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_show_subnet
--- /dev/null
+dovetail.ipv6.tc006:
+ name: dovetail.ipv6.tc006
+ objective: VIM ipv6 operations, to create an IPv6 port in allowed allocation pools
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_in_allowed_allocation_pools
--- /dev/null
+dovetail.ipv6.tc007:
+ name: dovetail.ipv6.tc007
+ objective: VIM ipv6 operations, to create an IPv6 port without security groups
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_with_no_securitygroups
--- /dev/null
+dovetail.ipv6.tc008:
+ name: dovetail.ipv6.tc008
+ objective: VIM ipv6 operations, to create/update/delete an IPv6 port
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_update_delete_port
--- /dev/null
+dovetail.ipv6.tc009:
+ name: dovetail.ipv6.tc009
+ objective: VIM ipv6 operations, to list IPv6 ports of a tenant
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_ports.PortsIpV6TestJSON.test_list_ports
--- /dev/null
+dovetail.ipv6.tc010:
+ name: dovetail.ipv6.tc010
+ objective: VIM ipv6 operations, to show information of an IPv6 port
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_ports.PortsIpV6TestJSON.test_show_port
--- /dev/null
+dovetail.ipv6.tc011:
+ name: dovetail.ipv6.tc011
+ objective: VIM ipv6 operations, to add multiple interfaces for an IPv6 router
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_routers.RoutersIpV6Test.test_add_multiple_router_interfaces
--- /dev/null
+dovetail.ipv6.tc012:
+ name: dovetail.ipv6.tc012
+ objective: VIM ipv6 operations, to add and remove an IPv6 router interface with port_id
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_port_id
--- /dev/null
+dovetail.ipv6.tc013:
+ name: dovetail.ipv6.tc013
+ objective: VIM ipv6 operations, to add and remove an IPv6 router interface with subnet_id
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_subnet_id
--- /dev/null
+dovetail.ipv6.tc014:
+ name: dovetail.ipv6.tc014
+ objective: VIM ipv6 operations, to create, update, delete, list and show an IPv6 router
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_routers.RoutersIpV6Test.test_create_show_list_update_delete_router
--- /dev/null
+dovetail.ipv6.tc015:
+ name: dovetail.ipv6.tc015
+ objective: VIM ipv6 operations, to create, update, delete, list and show an IPv6 security group
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_list_update_show_delete_security_group
--- /dev/null
+dovetail.ipv6.tc016:
+ name: dovetail.ipv6.tc016
+ objective: VIM ipv6 operations, to create, delete and show security group rules
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_show_delete_security_group_rule
--- /dev/null
+dovetail.ipv6.tc017:
+ name: dovetail.ipv6.tc017
+ objective: VIM ipv6 operations, to list all security groups
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_security_groups.SecGroupIPv6Test.test_list_security_groups
--- /dev/null
+dovetail.ipv6.tc018:
+ name: dovetail.ipv6.tc018
+ objective: VIM ipv6 operations, to show information of an IPv6 port
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.scenario.test_network_v6.TestGettingAddress.test_dhcp6_stateless_from_os
--- /dev/null
+dovetail.ipv6.tc019:
+ name: dovetail.ipv6.tc019
+ objective: VIM ipv6 operations, to do IPv6 address assignment - dual stack, DHCPv6 stateless
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os
--- /dev/null
+dovetail.ipv6.tc020:
+ name: dovetail.ipv6.tc020
+ objective: VIM ipv6 operations, to do IPv6 Address Assignment - Multiple Prefixes, DHCPv6 Stateless
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_dhcpv6_stateless
--- /dev/null
+dovetail.ipv6.tc021:
+ name: dovetail.ipv6.tc021
+ objective: VIM ipv6 operations, to do IPv6 Address Assignment - Dual Stack, Multiple Prefixes, DHCPv6 Stateless
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_dhcpv6_stateless
--- /dev/null
+dovetail.ipv6.tc022:
+ name: dovetail.ipv6.tc022
+ objective: VIM ipv6 operations, to do IPv6 Address Assignment - SLAAC
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.scenario.test_network_v6.TestGettingAddress.test_slaac_from_os
--- /dev/null
+dovetail.ipv6.tc023:
+ name: dovetail.ipv6.tc023
+ objective: VIM ipv6 operations, to do IPv6 Address Assignment - Dual Stack, SLAAC
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os
--- /dev/null
+dovetail.ipv6.tc024:
+ name: dovetail.ipv6.tc024
+ objective: VIM ipv6 operations, to do IPv6 address assignment - multiple prefixes, SLAAC
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_slaac
--- /dev/null
+dovetail.ipv6.tc025:
+ name: dovetail.ipv6.tc025
+ objective: VIM ipv6 operations, to do IPv6 address assignment - dual stack, multiple prefixes, SLAAC
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac
--- /dev/null
+dovetail.nfvi.tc001:
+ name: dovetail.nfvi.tc001
+ objective: testing for vping using ssh
+ scripts:
+ type: functest
+ testcase: vping_ssh
+ sub_testcase_list:
--- /dev/null
+dovetail.nfvi.tc002:
+ name: dovetail.nfvi.tc002
+ objective: testing for vping using userdata
+ scripts:
+ type: functest
+ testcase: vping_userdata
+ sub_testcase_list:
--- /dev/null
+dovetail.vimops.tc001:
+ name: dovetail.vimops.tc001
+ objective: Glance images v2 index
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.api.image.v2.test_images.ListImagesTest.test_list_no_params
--- /dev/null
+dovetail.vimops.tc002:
+ name: dovetail.vimops.tc002
+ objective: Glance Images v2 Delete
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_delete_image
+ - tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_delete_image_null_id
+ - tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_delete_non_existing_image
+ - tempest.api.image.v2.test_images_tags_negative.ImagesTagsNegativeTest.test_delete_non_existing_tag
--- /dev/null
+dovetail.vimops.tc003:
+ name: dovetail.vimops.tc003
+ objective: Glance images v2 list
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.api.image.v2.test_images.ListImagesTest.test_get_image_schema
+ - tempest.api.image.v2.test_images.ListImagesTest.test_get_images_schema
+ - tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_delete_deleted_image
+ - tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_image_null_id
+ - tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_non_existent_image
--- /dev/null
+dovetail.vimops.tc004:
+ name: dovetail.vimops.tc004
+ objective: Glance images v2 list
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_container_format
+ - tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_disk_format
+ - tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_limit
+ - tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_min_max_size
+ - tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_size
+ - tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_status
+ - tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_visibility
--- /dev/null
+dovetail.vimops.tc005:
+ name: dovetail.vimops.tc005
+ objective: Glance images v2 import
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_register_upload_get_image_file
+ - tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_register_with_invalid_container_format
+ - tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_register_with_invalid_disk_format
--- /dev/null
+dovetail.vimops.tc006:
+ name: dovetail.vimops.tc006
+ objective: Glance images v2 update
+ scripts:
+ type: functest
+ testcase: tempest_full_parallel
+ sub_testcase_list:
+ - tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_update_image
+ - tempest.api.image.v2.test_images_tags.ImagesTagsTest.test_update_delete_tags_for_image
+ - tempest.api.image.v2.test_images_tags_negative.ImagesTagsNegativeTest.test_update_tags_for_non_existing_image