3 created by [Cédric Ollivier](mailto:cedric.ollivier@orange.com)
9 - Functest integrates lots of heterogeneous testcases:
11 - internal vs external
12 - it aims to benefit from object programming
13 - to define common operations
14 - to avoid conditional instructions regarding the testcases
15 - to avoid duplicating code
16 - to ease the integration of third-party testcases (written in Bash or Python)
23 ### Functest function calls
25 - **CI** calls *run_tests.py* (please see [jenkins jobs](https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=tree;f=jjb/functest))
26 - *run_tests.py* parses *functest/ci/testcases.yaml* to:
27 - check which testcase(s) must be run
28 - execute the common operations on every testcase (run, push its results to db...)
29 <!-- .element: class="fragment highlight-red"-->
30 - return the right status code to **CI**
35 - limit run_tests.py instructions by defining:
36 - the basic testcase attributes
37 - all common operations
38 - the status codes expected
39 - avoid duplicating codes between testcases
40 - ease the development of third-party testcases (aka features)
46 base model for single test case
49 ### instance attributes
51 - project_name (default: 'functest')
63 |-------------------|------------------------------|
64 | run(**kwargs) | run the test case |
65 | is_successful() | interpret the results |
66 | get_duration() | return the duration |
67 | push_to_db() | push the results to the DB |
68 | clean() | clean the resources |
73 - the subclasses must override the default implementation which is false on purpose
74 - the new implementation must set the following attributes to push the results to DB:
82 | Status code | Returned when |
83 |--------------------|---------------------|
84 | EX_OK | everything is OK |
85 | EX_RUN_ERROR | run() failed |
86 | EX_TESTCASE_FAILED | results are false |
87 | EX_PUSH_TO_DB_ERROR| push_to_db() failed |
93 module = importlib.import_module(run_dict['module'])
94 cls = getattr(module, run_dict['class'])
95 test_dict = ft_utils.get_dict_by_test(test_name)
96 test_case = cls(**test_dict)
98 kwargs = run_dict['args']
99 result = test_case.run(**kwargs)
101 result = test_case.run()
102 if result == testcase.TestCase.EX_OK:
103 if GlobalVariables.REPORT_FLAG:
104 test_case.push_to_db()
105 result = test_case.is_successful()
110 ## Your first test case
116 #!/usr/bin/env python
120 from functest.core import testcase
122 class Test(testcase.TestCase):
124 def run(self, **kwargs):
125 self.start_time = time.time()
128 self.stop_time = time.time()
129 return testcase.TestCase.EX_OK
133 ### functest/ci/testcases.yaml
137 project_name: functest
154 base model for single feature
160 |-------------------|---------------------------|
161 | run(**kwargs) | run the feature |
162 | execute(**kwargs) | execute the Python method |
167 - allows executing any Python method by calling execute()
168 - sets the following attributes required to push the results to DB:
172 - doesn't fulfill details when pushing the results to the DB.
175 ### execute(**kwargs)
177 - the subclasses must override the default implementation which is false on purpose
178 - the new implementation must return 0 if success or anything else if failure.
182 ## Your second test case
188 #!/usr/bin/env python
190 from functest.core import feature
192 class Test(feature.Feature):
194 def execute(self, **kwargs):
200 ### functest/ci/testcases.yaml
204 project_name: functest
221 class designed to run any bash command
224 ### execute(**kwargs)
226 execute the cmd passed as arg.
230 ## Your third test case
233 ### functest/ci/testcases.yaml
237 project_name: functest
245 module: 'functest.core.feature'
248 cmd: 'echo Hello World; exit 0'
256 base model for running unittest.TestSuite
261 - allows running any unittest.TestSuite
262 - sets the following attributes required to push the results to DB:
270 ## Your fourth test case
276 #!/usr/bin/env python
280 class TestStringMethods(unittest.TestCase):
282 def test_upper(self):
283 self.assertEqual('Hello World'.upper(),
288 ### functest/ci/testcases.yaml
292 project_name: functest
300 module: 'functest.core.unit'
311 base model for VNF onboarding testing
317 |-----------------------|---------------------------------------------------|
318 | prepare() | prepare VNF env (user, tenant, security group,..) |
319 | run(**kwargs) | run VNF test case |
320 | deploy_orchestrator() | deploy cloudify, ONAP, OpenBaton,... (optional) |
321 | deploy_vnf() | deploy the VNF |
322 | test_vnf() | run tests on the VNF |
327 - deploys an orchestrator if needed (e.g. heat, OpenBaton, Cloudify, ONAP, Juju)
329 - performs tests on the VNF
335 - creates a Tenant/Project
336 - allocates admin role to the user on this tenant
339 ### deploy_orchestrator()
341 - deploys an orchestrator (optional)
342 - if this function is overridden then raise orchestratorDeploymentException if error during orchestrator deployment
347 - **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases.
348 - The deployment can be executed via a specific orchestrator or using build-in orchestrators such as heat, openbaton, cloudify, juju, ONAP, ...
350 True if the VNF is properly deployed
351 False if the VNF is not deployed
352 - raises VnfDeploymentException if error during VNF deployment
357 - **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases.
358 - Once a VNF is deployed, it is assumed that specific test suite can be run to validate the VNF.
360 True if VNF tests are PASS
361 False if test suite is FAIL
362 - raises VnfTestException if error during VNF tests
366 ## Your fifth test case
372 #!/usr/bin/env python
374 from functest.core import vnf
376 class Vnf(vnf.VnfOnBoarding):
378 def deploy_vnf(self):
379 print "Deploy your VNF here"
380 print "Feed orchestrator with VNF descriptor"
384 print "Test your VNF here"
389 ### functest/ci/testcases.yaml
393 project_name: functest