-healthcheck:
- order: 0
- description : |
- This is the optional healthcheck
- that can provided by the installer
- or the internal in functest
- testcases:
- healthcheck:
- installer: any
- sdn: any
- feat: any
- mode: any
-smoke:
- order: 1
- description : |
- This is the set of basic functest
- tests and smoke tempest in serial mode.
- testcases:
- vping_ssh:
- installer: any
- sdn: any
- feat: any
- mode: any
+tiers:
+ -
+ name: healthcheck
+ order: 0
+ ci: daily
+ description : >-
+ This is the first tier to be executed to verify the basic
+ operations in the VIM.
+ testcases:
+ -
+ name: healthcheck
+ description: >-
+ This test case verifies the basic OpenStack services like
+ Keystone, Glance, Cinder, Neutron and Nova.
- vping_userdata:
- installer: any
- sdn: any
- feat: any
- mode: any
+ dependencies:
+ installer: ''
+ scenario: ''
- tempest_smoke_serial:
- installer: any
- sdn: any
- feat: any
- mode: any
+ -
+ name: smoke
+ order: 1
+ ci: daily
+ description : >-
+ This is the second tier in Functest and consist of a set of basic
+ Functional tests to validate the OpenStack deployment.
+ testcases:
+ -
+ name: vping_ssh
+ description: |-
+ This test case verifies:
+ ····1) SSH to an instance using floating IPs over the public network.
+ ····2) Connectivity between 2 instances over a private network.
+ dependencies:
+ installer: ''
+ scenario: '^((?!bgpvpn).)*$'
- rally_smoke:
- installer: any
- sdn: any
- feat: any
- mode: any
+ -
+ name: vping_userdata
+ description: |-
+ This test case verifies:
+ ····1) Boot a VM with given userdata.
+ ····2) Connectivity between 2 instances over a private network.
+ dependencies:
+ installer: ''
+ scenario: ''
- security_groups:
- installer: any
- sdn: any
- feat: any
- mode: any
-sdn_suites:
- order: 2
- description : |
- test cases for the SDN controllers
- testcases:
- odl:
- installer: any
- sdn: odl_l2|odl_l3
- feat: any
- mode: any
- onos:
- installer: any
- sdn: onos
- feat: any
- mode: any
- ovno:
- installer: any
- sdn: ocl
- feat: any
- mode: any
-features:
- order: 4
- description : |
- test from feature projects integrated in functest
- testcases:
- promise:
- installer: fuel|joid
- sdn: any
- feat: any
- mode: any
- sdnvpn:
- installer: fuel|apex
- sdn: odl_l2
- feat: bgpvpn
- mode: any
- policy-test:
- installer: any
- sdn: odl_l2|odl_l3
- feat: any
- mode: any
-tempest:
- order: 5
- description : |
- this is the execution of the full tempest suite in parallel
- testcases:
- tempest_full_parallel:
- installer: any
- sdn: any
- feat: any
- mode: any
-rally:
- order: 6
- description : |
- Full Rally suite
- testcases:
- tempest_full_parallel:
- installer: any
- sdn: any
- feat: any
- mode: any
-vnf:
- order: 7
- description : |
- collection of VNF test cases
- testcases:
- vims:
- installer: any
- sdn: any
- feat: any
- mode: any
\ No newline at end of file
+ -
+ name: tempest_smoke_serial
+ description: >-
+ This test case runs the smoke subset of the OpenStack
+ Tempest suite. The list of test cases is generated by
+ Tempest automatically and depend on the parameters of
+ the OpenStack deplopyment.
+ dependencies:
+ installer: ''
+ scenario: ''
+
+ -
+ name: rally_sanity
+ description: >-
+ This test case runs a sub group of tests of the OpenStack
+ Rally suite in smoke mode.
+ dependencies:
+ installer: ''
+ scenario: ''
+
+ -
+ name: security_groups
+ description: >-
+ This test case verifies the functionality of the OpenStack
+ security groups and that the port rules created are
+ fullfilled.
+ dependencies:
+ installer: ''
+ scenario: ''
+
+ -
+ name: sdn_suites
+ order: 2
+ ci: daily
+ description : >-
+ This tier contains the test suites corresponding to the different
+ SDN Controllers existing in OPNFV.
+ testcases:
+ -
+ name: odl
+ description: >-
+ Test Suite for the OpenDaylight SDN Controller. It integrates
+ some test suites from upstream using Robot as the test
+ framework.
+ dependencies:
+ installer: ''
+ scenario: 'odl'
+
+ -
+ name: onos
+ description: >-
+ Test Suite for the ONOS SDN Controller. It integrates
+ some test suites from upstream using TestON as the test
+ framework.
+ dependencies:
+ installer: ''
+ scenario: 'onos'
+
+ -
+ name: ovno
+ description: >-
+ Test Suite for the Open Contrail SDN Controller.
+ framework.
+ dependencies:
+ installer: ''
+ scenario: 'ocl'
+
+ -
+ name: features
+ order: 3
+ ci: daily
+ description : >-
+ This tier contains the test suites from feature projects
+ integrated in functest
+ testcases:
+ -
+ name: promise
+ description: >-
+ Test suite from Promise project.
+ dependencies:
+ installer: '(fuel)|(joid)'
+ scenario: ''
+
+ -
+ name: doctor
+ description: >-
+ Test suite from Dcotor project.
+ dependencies:
+ installer: 'apex'
+ scenario: ''
+
+ -
+ name: sdnvpn
+ description: >-
+ Test suite from SDNVPN project.
+ dependencies:
+ installer: '(fuel)|(apex)'
+ scenario: 'bgpvpn'
+
+ -
+ name: tempest
+ order: 4
+ ci: weekly
+ description : >-
+ This tier contains the test suites from feature projects
+ integrated in functest
+ testcases:
+ -
+ name: tempest_full_parallel
+ description: >-
+ This test case runs the full set of the OpenStack
+ Tempest suite. The list of test cases is generated by
+ Tempest automatically and depend on the parameters of
+ the OpenStack deplopyment.
+ dependencies:
+ installer: ''
+ scenario: ''
+
+ -
+ name: rally
+ order: 5
+ ci: weekly
+ description : >-
+ This tier contains the Rally suite from the OpenStack community.
+ testcases:
+ -
+ name: rally_full
+ description: >-
+ This test case runs the full suite of scenarios of the OpenStack
+ Rally suite using several threads and iterations.
+ dependencies:
+ installer: ''
+ scenario: ''
+
+ -
+ name: vnf
+ order: 6
+ ci: weekly
+ description : >-
+ This tier contains a collection of VNF test cases.
+ testcases:
+ -
+ name: vims
+ description: >-
+ This test case deploys an OpenSource vIMS solution from Clearwater
+ using the Cloudify orchestrator. It also runs some signaling traffic.
+ dependencies:
+ installer: ''
+ scenario: '(ocl)|(nosdn)|^(os-odl)((?!bgpvpn).)*$'
--- /dev/null
+#!/usr/bin/env python
+#
+# jose.lausuch@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+
+import yaml
+
+import tier_handler as th
+
+
+class TierBuilder:
+ def __init__(self, ci_installer, ci_scenario, testcases_file):
+ self.ci_installer = ci_installer
+ self.ci_scenario = ci_scenario
+ self.testcases_file = testcases_file
+ self.dic_tier_array = None
+ self.tier_objects = []
+ self.testcases_yaml = None
+ self.generate_tiers()
+
+ def read_test_yaml(self):
+ with open(self.testcases_file) as f:
+ self.testcases_yaml = yaml.safe_load(f)
+
+ self.dic_tier_array = []
+ for tier in self.testcases_yaml.get("tiers"):
+ self.dic_tier_array.append(tier)
+
+ def generate_tiers(self):
+ if self.dic_tier_array is None:
+ self.read_test_yaml()
+
+ del self.tier_objects[:]
+ for dic_tier in self.dic_tier_array:
+ tier = th.Tier(name=dic_tier['name'],
+ order=dic_tier['order'],
+ ci=dic_tier['ci'],
+ description=dic_tier['description'])
+
+ for dic_testcase in dic_tier['testcases']:
+ installer = dic_testcase['dependencies']['installer']
+ scenario = dic_testcase['dependencies']['scenario']
+ dep = th.Dependency(installer, scenario)
+
+ testcase = th.TestCase(name=dic_testcase['name'],
+ dependency=dep,
+ description=dic_testcase['description'])
+ if testcase.is_compatible(self.ci_installer, self.ci_scenario):
+ tier.add_test(testcase)
+
+ self.tier_objects.append(tier)
+
+ def __str__(self):
+ output = ""
+ for i in range(0, len(self.tier_objects)):
+ output += str(self.tier_objects[i]) + "\n"
+ return output
--- /dev/null
+#!/usr/bin/env python
+#
+# jose.lausuch@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+
+
+import re
+
+
+class Tier:
+ def __init__(self, name, order, ci, description=""):
+ self.tests_array = []
+ self.name = name
+ self.order = order
+ self.ci = ci
+ self.description = description
+
+ def add_test(self, testcase):
+ self.tests_array.append(testcase)
+
+ def get_tests(self):
+ array_str = []
+ for test in self.tests_array:
+ array_str.append(test.name)
+ return array_str
+
+ def __str__(self):
+ return ("Tier info:\n"
+ " Name: " + self.name + "\n"
+ " Description: " + self.description + "\n"
+ " Order: " + str(self.order) + "\n"
+ " Test cases: " + str(self.get_tests()) + "\n")
+
+
+class TestCase:
+ def __init__(self, name, dependency, description=""):
+ self.name = name
+ self.dependency = dependency
+ self.description = description
+
+ def is_compatible(self, ci_installer, ci_scenario):
+ if re.search(self.dependency.get_installer(), ci_installer) is None:
+ return False
+
+ if re.search(self.dependency.get_scenario(), ci_scenario) is None:
+ return False
+
+ return True
+
+ def __str__(self):
+ return ("Testcase info:\n"
+ " Name: " + self.name + "\n"
+ " Description: " + self.description + "\n"
+ " " + str(self.dependency) + "\n")
+
+
+class Dependency:
+ def __init__(self, installer, scenario):
+ self.installer = installer
+ self.scenario = scenario
+
+ def get_installer(self):
+ return self.installer
+
+ def get_scenario(self):
+ return self.scenario
+
+ def __str__(self):
+ return ("Dependency info:\n"
+ " installer: " + self.installer + "\n"
+ " scenario: " + self.scenario + "\n")
+++ /dev/null
-#!/usr/bin/env python
-#
-# jose.lausuch@ericsson.com
-# All rights reserved. This program and the accompanying materials
-# are made available under the terms of the Apache License, Version 2.0
-# which accompanies this distribution, and is available at
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-
-
-class Tier:
- def __init__(self, name, description, order):
- self.tests_array = []
- self.name = name
- self.description = description
- self.order = order
-
- def add_test(self, testcase):
- self.tests_array.append(testcase)
-
- def get_tests(self):
- array_str = []
- for test in self.tests_array:
- array_str.append(test.name)
- return array_str
-
- def __str__(self):
- return "Tier info:\n" + \
- "\tName: " + self.name + "\n" + \
- "\tDescription: " + self.description + "\n" + \
- "\tOrder: " + self.order + "\n" + \
- "\tTest cases: " + str(self.get_tests()) + "\n"
-
-
-class Testcase:
- def __init__(self, name, description, dependency):
- self.name = name
- self.description = description
- self.dependency = dependency
-
- def __str__(self):
- return "Testcase info:\n" + \
- "\tName: " + self.name + "\n" + \
- "\tName: " + self.name + "\n" + \
- "\tDescription: " + self.description + "\n" + \
- "\tDependencies: " + str(self.dependency) + "\n"
-
-
-class Dependency:
- def __init__(self, installer, sdn, feature, mode):
- self.installer = installer
- self.sdn = sdn
- self.feature = feature
- self.mode = mode
-
- def __str__(self):
- return "Dependency info:\n" + \
- "\t" + self.installer + " os-[" + self.sdn + "]-[" + \
- self.feature + "]-[" + self.mode + "]" + "\n" + \
- "\t\t- installer: " + self.installer + "\n" + \
- "\t\t- sdn Controller: " + self.sdn + "\n" + \
- "\t\t- feature: " + self.feature + "\n" + \
- "\t\t- mode: " + self.mode + "\n"
Code Review / functest-xtesting.git / commitdiff