X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=blobdiff_plain;f=functest%2Fci%2Ftestcases.yaml;h=bc27f6f8f0ce93d0ac9b45d515dcb8922e68394b;hb=a752aff9c3f597ec28c52e7a3f350de8daa45246;hp=12d791f08b3aa8d7880b125b489eb48ecf4e0d50;hpb=f6f929ae158be1b52cd0ffc0c1f02eb7709504f1;p=functest.git diff --git a/functest/ci/testcases.yaml b/functest/ci/testcases.yaml index 12d791f08..bc27f6f8f 100755 --- a/functest/ci/testcases.yaml +++ b/functest/ci/testcases.yaml @@ -8,17 +8,58 @@ tiers: operations in the VIM. testcases: - - name: healthcheck + name: snaps_health_check criteria: 'status == "PASS"' blocking: true + clean_flag: false description: >- - This test case verifies the basic OpenStack services like - Keystone, Glance, Cinder, Neutron and Nova. - + This test case creates executes the SimpleHealthCheck + Python test class which creates an, image, flavor, network, + and Cirros VM instance and observes the console output to + validate the single port obtains the correct IP address. dependencies: installer: '' scenario: '^((?!lxd).)*$' + run: + module: 'functest.opnfv_tests.openstack.snaps.health_check' + class: 'HealthCheck' + - + name: connection_check + criteria: 'status == "PASS"' + blocking: true + clean_flag: false + description: >- + This test case verifies the retrieval of OpenStack clients: + Keystone, Glance, Neutron and Nova and may perform some + simple queries. When the config value of + snaps.use_keystone is True, functest must have access to + the cloud's private network. + dependencies: + installer: '^((?!netvirt).)*$' + scenario: '' + run: + module: 'functest.opnfv_tests.openstack.snaps.connection_check' + class: 'ConnectionCheck' + + - + name: api_check + criteria: 'status == "PASS"' + blocking: true + clean_flag: false + description: >- + This test case verifies the retrieval of OpenStack clients: + Keystone, Glance, Neutron and Nova and may perform some + simple queries. When the config value of + snaps.use_keystone is True, functest must have access to + the cloud's private network. + + dependencies: + installer: '^((?!netvirt).)*$' + scenario: '' + run: + module: 'functest.opnfv_tests.openstack.snaps.api_check' + class: 'ApiCheck' - name: smoke order: 1 @@ -30,13 +71,14 @@ tiers: name: vping_ssh criteria: 'status == "PASS"' blocking: true + clean_flag: true description: >- This test case verifies: 1) SSH to an instance using floating IPs over the public network. 2) Connectivity between 2 instances over a private network. dependencies: installer: '' - scenario: '^((?!bgpvpn|odl_l3).)*$' + scenario: '^((?!odl_l3).)*$' run: module: 'functest.opnfv_tests.openstack.vping.vping_ssh' class: 'VPingSSH' @@ -45,6 +87,7 @@ tiers: name: vping_userdata criteria: 'status == "PASS"' blocking: true + clean_flag: true description: >- This test case verifies: 1) Boot a VM with given userdata. 2) Connectivity between 2 instances over a private network. @@ -59,6 +102,7 @@ tiers: name: tempest_smoke_serial criteria: 'success_rate == 100%' blocking: false + clean_flag: false description: >- This test case runs the smoke subset of the OpenStack Tempest suite. The list of test cases is generated by @@ -75,12 +119,13 @@ tiers: name: rally_sanity criteria: 'success_rate == 100%' blocking: false + clean_flag: false description: >- This test case runs a sub group of tests of the OpenStack Rally suite in smoke mode. dependencies: installer: '' - scenario: '^((?!bgpvpn).)*$' + scenario: '' run: module: 'functest.opnfv_tests.openstack.rally.rally' class: 'RallySanity' @@ -89,10 +134,11 @@ tiers: name: odl criteria: 'success_rate == 100%' blocking: true + clean_flag: false description: >- - Test Suite for the OpenDaylight SDN Controller. It integrates - some test suites from upstream using Robot as the test - framework. + Test Suite for the OpenDaylight SDN Controller. It + integrates some test suites from upstream using + Robot as the test framework. dependencies: installer: '' scenario: 'odl' @@ -105,57 +151,67 @@ tiers: - /home/opnfv/repos/odl_test/csit/suites/openstack/neutron - - name: onos - criteria: 'status == "PASS"' + name: odl_netvirt + criteria: 'success_rate == 100%' blocking: true + clean_flag: false description: >- - Test Suite for the ONOS SDN Controller. It integrates - some test suites from upstream using TestON as the test - framework. + Test Suite for the OpenDaylight SDN Controller when + the NetVirt features are installed. It integrates + some test suites from upstream using Robot as the + test framework. dependencies: installer: '' - scenario: 'onos' + scenario: 'netvirt' + run: + module: 'functest.opnfv_tests.sdn.odl.odl' + class: 'ODLTests' + args: + suites: + - /home/opnfv/repos/odl_test/csit/suites/integration/basic + - /home/opnfv/repos/odl_test/csit/suites/openstack/neutron + - /home/opnfv/repos/odl_test/csit/suites/openstack/connectivity - - name: connection_check - criteria: 'status == "PASS"' + name: fds + criteria: 'success_rate == 100%' blocking: false + clean_flag: false description: >- - This test case verifies the retrieval of OpenStack clients: - Keystone, Glance, Neutron and Nova and may perform some - simple queries. When the config value of - snaps.use_keystone is True, functest must have access to - the cloud's private network. - + Test Suite for the OpenDaylight SDN Controller when GBP features are + installed. It integrates some test suites from upstream using + Robot as the test framework. dependencies: - installer: '^((?!netvirt).)*$' - scenario: '' + installer: 'apex' + scenario: 'odl_l2-fdio' run: - module: 'functest.opnfv_tests.openstack.snaps.connection_check' - class: 'ConnectionCheck' + module: 'functest.opnfv_tests.sdn.odl.odl' + class: 'ODLTests' + args: + suites: + - /home/opnfv/repos/fds/testing/robot - - name: api_check + name: onos criteria: 'status == "PASS"' - blocking: false + blocking: true + clean_flag: true description: >- - This test case verifies the retrieval of OpenStack clients: - Keystone, Glance, Neutron and Nova and may perform some - simple queries. When the config value of - snaps.use_keystone is True, functest must have access to - the cloud's private network. - + Test Suite for the ONOS SDN Controller. It integrates + some test suites from upstream using TestON as the test + framework. dependencies: - installer: '^((?!netvirt).)*$' - scenario: '' + installer: '' + scenario: 'onos' run: - module: 'functest.opnfv_tests.openstack.snaps.api_check' - class: 'ApiCheck' + module: 'functest.opnfv_tests.sdn.onos.onos' + class: 'Onos' - name: snaps_smoke criteria: 'status == "PASS"' blocking: false + clean_flag: false description: >- This test case contains tests that setup and destroy environments with VMs with and without Floating IPs @@ -167,7 +223,7 @@ tiers: dependencies: installer: '^((?!netvirt).)*$' - scenario: '' + scenario: '^((?!lxd).)*$' run: module: 'functest.opnfv_tests.openstack.snaps.smoke' class: 'SnapsSmoke' @@ -184,6 +240,7 @@ tiers: name: promise criteria: 'success_rate == 100%' blocking: false + clean_flag: true description: >- Test suite from Promise project. dependencies: @@ -197,6 +254,7 @@ tiers: name: doctor criteria: 'status == "PASS"' blocking: false + clean_flag: true description: >- Test suite from Doctor project. dependencies: @@ -210,6 +268,7 @@ tiers: name: bgpvpn criteria: 'status == "PASS"' blocking: false + clean_flag: true description: >- Test suite from SDNVPN project. dependencies: @@ -223,15 +282,20 @@ tiers: name: security_scan criteria: 'status == "PASS"' blocking: false + clean_flag: true description: >- - Simple security Scan + Simple Security Scan dependencies: installer: 'apex' scenario: '^((?!fdio).)*$' + run: + module: 'functest.opnfv_tests.features.security_scan' + class: 'SecurityScan' # - # name: copper # criteria: 'status == "PASS"' # blocking: false +# clean_flag: true # description: >- # Test suite for policy management based on OpenStack Congress # dependencies: @@ -240,19 +304,11 @@ tiers: # run: # module: 'functest.opnfv_tests.features.copper' # class: 'Copper' - - - name: moon - criteria: 'status == "PASS"' - blocking: false - description: >- - Security management system for OPNFV - dependencies: - installer: 'compass' - scenario: '(odl)*(moon)' - name: multisite criteria: 'success_rate == 100%' blocking: false + clean_flag: false description: >- Test suite from kingbird dependencies: @@ -265,6 +321,7 @@ tiers: name: odl-sfc criteria: 'status == "PASS"' blocking: false + clean_flag: true description: >- Test suite for odl-sfc to test two chains and two SFs dependencies: @@ -277,15 +334,20 @@ tiers: name: onos_sfc criteria: 'status == "PASS"' blocking: true + clean_flag: true description: >- Test Suite for onos-sfc to test sfc function. dependencies: installer: '' scenario: 'onos-sfc' + run: + module: 'functest.opnfv_tests.sdn.onos.onos' + class: 'OnosSfc' - name: parser criteria: 'ret == 0' blocking: false + clean_flag: true description: >- Test suite from Parser project. dependencies: @@ -295,52 +357,114 @@ tiers: module: 'functest.opnfv_tests.vnf.rnc.parser' class: 'Parser' - - name: orchestra - criteria: 'ret == 0' + name: domino + criteria: 'status == "PASS"' blocking: false + clean_flag: true description: >- - Test OpenBaton (Orchestra) stack + Test suite from Domino project. dependencies: - installer: 'joid' - scenario: 'unknown' + installer: '' + scenario: '' run: - module: 'functest.opnfv_tests.features.orchestrator.orchestra' - class: 'OpenbatonOrchestrator' + module: 'functest.opnfv_tests.features.domino' + class: 'Domino' + - + name: netready + criteria: 'status == "PASS"' + blocking: false + clean_flag: true + description: >- + Test suite from Netready project. + dependencies: + installer: 'apex' + scenario: 'gluon' + run: + module: 'functest.opnfv_tests.features.netready' + class: 'GluonVping' + - + name: barometer + criteria: 'status == "PASS"' + blocking: false + clean_flag: true + description: >- + Test suite for the Barometer project. Separate tests verify the + proper configuration and functionality of the following + collectd plugins Ceilometer, Hugepages, Memory RAS (mcelog), + and OVS Events + dependencies: + installer: 'fuel' + scenario: 'kvm_ovs_dpdk_bar' + run: + module: 'functest.opnfv_tests.features.barometer' + class: 'BarometerCollectd' - name: components order: 3 - ci_loop: 'weekly' + ci_loop: 'daily' description : >- Extensive testing of OpenStack API. testcases: +# - +# name: tempest_full_parallel +# criteria: 'success_rate >= 80%' +# blocking: false +# clean_flag: false +# description: >- +# The list of test cases is generated by +# Tempest automatically and depends on the parameters of +# the OpenStack deplopyment. +# dependencies: +# installer: '^((?!netvirt).)*$' +# scenario: '' +# run: +# module: 'functest.opnfv_tests.openstack.tempest.tempest' +# class: 'TempestFullParallel' - - name: tempest_full_parallel - criteria: 'success_rate >= 80%' + name: tempest_defcore + criteria: 'success_rate == 100%' blocking: false + clean_flag: false description: >- - The list of test cases is generated by - Tempest automatically and depends on the parameters of - the OpenStack deplopyment. + This is the set of Tempest test cases created by OpenStack + Interop Working Group for certification purposes. dependencies: - installer: '^((?!netvirt).)*$' - scenario: '' + installer: '' + scenario: 'nosdn-nofeature-ha' run: module: 'functest.opnfv_tests.openstack.tempest.tempest' - class: 'TempestFullParallel' - + class: 'TempestDefcore' - - name: rally_full - criteria: 'success_rate >= 90%' + name: tempest_custom + criteria: 'success_rate == 100%' blocking: false + clean_flag: false description: >- - This test case runs the full suite of scenarios of the OpenStack - Rally suite using several threads and iterations. + The test case allows running a customized list of tempest + test cases defined in a file under + /functest/opnfv_tests/openstack/ + /tempest/custom_tests/test_list.txt + The file is empty and can be customized with the desired tests. dependencies: - installer: '^((?!netvirt).)*$' - scenario: '' + installer: 'unknown' + scenario: 'unknown' run: - module: 'functest.opnfv_tests.openstack.rally.rally' - class: 'RallyFull' + module: 'functest.opnfv_tests.openstack.tempest.tempest' + class: 'TempestCustom' +# - +# name: rally_full +# criteria: 'success_rate >= 90%' +# blocking: false +# clean_flag: false +# description: >- +# This test case runs the full suite of scenarios of the OpenStack +# Rally suite using several threads and iterations. +# dependencies: +# installer: '^((?!netvirt).)*$' +# scenario: '' +# run: +# module: 'functest.opnfv_tests.openstack.rally.rally' +# class: 'RallyFull' - name: vnf @@ -353,6 +477,7 @@ tiers: name: cloudify_ims criteria: 'status == "PASS"' blocking: false + clean_flag: true description: >- This test case deploys an OpenSource vIMS solution from Clearwater using the Cloudify orchestrator. It also runs some signaling traffic. @@ -362,54 +487,57 @@ tiers: run: module: 'functest.opnfv_tests.vnf.ims.cloudify_ims' class: 'ImsVnf' +# - +# name: aaa +# criteria: 'ret == 0' +# blocking: false +# clean_flag: true +# description: >- +# Test suite from Parser project. +# dependencies: +# installer: '' +# scenario: '' +# run: +# module: 'functest.opnfv_tests.vnf.aaa.aaa' +# class: 'AaaVnf' - - name: aaa + name: orchestra_ims criteria: 'ret == 0' blocking: false + clean_flag: true description: >- - Test suite from Parser project. + VNF deployment with OpenBaton (Orchestra) dependencies: installer: '' scenario: '' run: - module: 'functest.opnfv_tests.vnf.aaa.aaa' - class: 'AaaVnf' - - - - name: juju_epc - criteria: 'ret == 0' - blocking: false - description: >- - Test suite from OAI project, vEPC deployed with Juju. - dependencies: - installer: 'unknown' - scenario: 'unknown' - run: - module: 'functest.opnfv_tests.vnf.epc.epc' - class: 'EpcVnf' + module: 'functest.opnfv_tests.vnf.ims.orchestra_ims' + class: 'ImsVnf' - - name: orchestra_ims + name: opera_ims criteria: 'ret == 0' blocking: false + clean_flag: true description: >- - VNF deployment with OpenBaton (Orchestra) + Evolution of vIMS dependencies: installer: 'unknown' scenario: 'unknown' run: - module: 'functest.opnfv_tests.vnf.ims.orchestra_ims' + module: 'functest.opnfv_tests.vnf.ims.opera_ims' class: 'ImsVnf' - - name: opera_ims - criteria: 'ret == 0' + name: vyos_vrouter + criteria: 'status == "PASS"' blocking: false + clean_flag: true description: >- - Evolution of vIMS + This test case is vRouter testing. dependencies: - installer: 'unknown' - scenario: 'unknown' + installer: 'fuel' + scenario: 'nosdn-nofeature' run: - module: 'functest.opnfv_tests.vnf.ims.opera_ims' - class: 'ImsVnf' + module: 'functest.opnfv_tests.vnf.router.vyos_vrouter' + class: 'VrouterVnf'