3 created by [Cédric Ollivier](mailto:cedric.ollivier@orange.com)
9 - Functest integrates lots of heterogeneous testcases:
11 - internal vs external
12 - it aims to benefit from object programming
13 - to define common operations
14 - to avoid conditional instructions regarding the testcases
15 - to avoid duplicating code
16 - to ease the integration of third-party testcases (written in Bash or Python)
23 ### Functest function calls
25 - **CI** calls *run_tests.py* (please see [jenkins jobs](https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=tree;f=jjb/functest))
26 - *run_tests.py* parses *functest/ci/testcases.yaml* to:
27 - check which testcase(s) must be run
28 - execute the common operations on every testcase (run, push its results to db...)
29 <!-- .element: class="fragment highlight-red"-->
30 - return the right status code to **CI**
35 - limit run_tests.py instructions by defining:
36 - the basic testcase attributes
37 - all common operations
38 - the status codes expected
39 - avoid duplicating codes between testcases
40 - ease the development of third-party testcases (aka features)
46 base model for single test case
49 ### instance attributes
51 - project_name (default: 'functest')
63 |-------------------|------------------------------|
64 | run(**kwargs) | run the test case |
65 | is_successful() | interpret the results |
66 | get_duration() | return the duration |
67 | push_to_db() | push the results to the DB |
68 | create_snapshot() | save the testing environment |
69 | clean() | clean the resources |
74 - the subclasses must override the default implementation which is false on purpose
75 - the new implementation must set the following attributes to push the results to DB:
83 | Status code | Returned when |
84 |--------------------|---------------------|
85 | EX_OK | everything is OK |
86 | EX_RUN_ERROR | run() failed |
87 | EX_TESTCASE_FAILED | results are false |
88 | EX_PUSH_TO_DB_ERROR| push_to_db() failed |
94 module = importlib.import_module(run_dict['module'])
95 cls = getattr(module, run_dict['class'])
96 test_dict = ft_utils.get_dict_by_test(test_name)
97 test_case = cls(**test_dict)
99 kwargs = run_dict['args']
100 result = test_case.run(**kwargs)
102 result = test_case.run()
103 if result == testcase.TestCase.EX_OK:
104 if GlobalVariables.REPORT_FLAG:
105 test_case.push_to_db()
106 result = test_case.is_successful()
111 ## Your first test case
117 #!/usr/bin/env python
121 from functest.core import testcase
123 class Test(testcase.TestCase):
125 def run(self, **kwargs):
126 self.start_time = time.time()
129 self.stop_time = time.time()
130 return testcase.TestCase.EX_OK
134 ### functest/ci/testcases.yaml
138 project_name: functest
155 base model for single feature
161 |-------------------|---------------------------|
162 | run(**kwargs) | run the feature |
163 | execute(**kwargs) | execute the Python method |
168 - allows executing any Python method by calling execute()
169 - sets the following attributes required to push the results to DB:
173 - doesn't fulfill details when pushing the results to the DB.
176 ### execute(**kwargs)
178 - the subclasses must override the default implementation which is false on purpose
179 - the new implementation must return 0 if success or anything else if failure.
183 ## Your second test case
189 #!/usr/bin/env python
191 from functest.core import feature
193 class Test(feature.Feature):
195 def execute(self, **kwargs):
201 ### functest/ci/testcases.yaml
205 project_name: functest
222 class designed to run any bash command
225 ### execute(**kwargs)
227 execute the cmd passed as arg.
231 ## Your third test case
234 ### functest/ci/testcases.yaml
238 project_name: functest
246 module: 'functest.core.feature'
249 cmd: 'echo Hello World; exit 0'
257 base model for running unittest.TestSuite
262 - allows running any unittest.TestSuite
263 - sets the following attributes required to push the results to DB:
271 ## Your fourth test case
277 #!/usr/bin/env python
281 class TestStringMethods(unittest.TestCase):
283 def test_upper(self):
284 self.assertEqual('Hello World'.upper(),
289 ### functest/ci/testcases.yaml
293 project_name: functest
301 module: 'functest.core.unit'
312 base model for VNF onboarding testing
318 |-----------------------|---------------------------------------------------|
319 | prepare() | prepare VNF env (user, tenant, security group,..) |
320 | run(**kwargs) | run VNF test case |
321 | deploy_orchestrator() | deploy cloudify, ONAP, OpenBaton,... (optional) |
322 | deploy_vnf() | deploy the VNF |
323 | test_vnf() | run tests on the VNF |
328 - deploys an orchestrator if needed (e.g. heat, OpenBaton, Cloudify, ONAP, Juju)
330 - performs tests on the VNF
336 - creates a Tenant/Project
337 - allocates admin role to the user on this tenant
340 ### deploy_orchestrator()
342 - deploys an orchestrator (optional)
343 - if this function is overridden then raise orchestratorDeploymentException if error during orchestrator deployment
348 - **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases.
349 - The deployment can be executed via a specific orchestrator or using build-in orchestrators such as heat, openbaton, cloudify, juju, ONAP, ...
351 True if the VNF is properly deployed
352 False if the VNF is not deployed
353 - raises VnfDeploymentException if error during VNF deployment
358 - **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases.
359 - Once a VNF is deployed, it is assumed that specific test suite can be run to validate the VNF.
361 True if VNF tests are PASS
362 False if test suite is FAIL
363 - raises VnfTestException if error during VNF tests
367 ## Your fifth test case
373 #!/usr/bin/env python
375 from functest.core import vnf
377 class Vnf(vnf.VnfOnBoarding):
379 def deploy_vnf(self):
380 print "Deploy your VNF here"
381 print "Feed orchestrator with VNF descriptor"
385 print "Test your VNF here"
390 ### functest/ci/testcases.yaml
394 project_name: functest