3 created by [Cédric Ollivier](mailto:cedric.ollivier@orange.com)
9 - Functest integrates lots of heterogeneous testcases:
11 - internal vs external
12 - it aims to benefit from object programming
13 - to define common operations
14 - to avoid conditional instructions regarding the testcases
15 - to avoid duplicating code
16 - to ease the integration of third-party testcases (written in Bash or Python)
23 ### Functest function calls
25 - **CI** calls *run_tests.py* (please see [jenkins jobs](https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=tree;f=jjb/functest))
26 - *run_tests.py* parses *functest/ci/testcases.yaml* to:
27 - check which testcase(s) must be run
28 - execute the common operations on every testcase (run, push its results to db...)
29 <!-- .element: class="fragment highlight-red"-->
30 - return the right status code to **CI**
35 - limit run_tests.py instructions by defining:
36 - the basic testcase attributes
37 - all common operations
38 - the status codes expected
39 - avoid duplicating codes between testcases
40 - ease the development of third-party testcases (aka features)
46 base model for single test case
49 ### instance attributes
51 - project_name (default: 'functest')
63 |-------------------|------------------------------|
64 | run(**kwargs) | run the test case |
65 | is_successful() | interpret the results |
66 | get_duration() | return the duration |
67 | push_to_db() | push the results to the DB |
68 | create_snapshot() | save the testing environment |
69 | clean() | clean the resources |
74 - the subclasses must override the default implementation which is false on purpose
75 - the new implementation must set the following attributes to push the results to DB:
83 | Status code | Returned when |
84 |--------------------|---------------------|
85 | EX_OK | everything is OK |
86 | EX_RUN_ERROR | run() failed |
87 | EX_TESTCASE_FAILED | results are false |
88 | EX_PUSH_TO_DB_ERROR| push_to_db() failed |
94 module = importlib.import_module(run_dict['module'])
95 cls = getattr(module, run_dict['class'])
96 test_dict = ft_utils.get_dict_by_test(test_name)
97 test_case = cls(**test_dict)
99 kwargs = run_dict['args']
100 result = test_case.run(**kwargs)
102 result = test_case.run()
103 if result == testcase.TestCase.EX_OK:
104 if GlobalVariables.REPORT_FLAG:
105 test_case.push_to_db()
106 result = test_case.is_successful()
111 ## Your first test case
117 #!/usr/bin/env python
121 from functest.core import testcase
123 class Test(testcase.TestCase):
125 def run(self, **kwargs):
126 self.start_time = time.time()
129 self.stop_time = time.time()
130 return testcase.TestCase.EX_OK
134 ### functest/ci/testcases.yaml
138 project_name: functest
155 base model for single feature
161 |-------------------|---------------------------|
162 | run(**kwargs) | run the feature |
163 | execute(**kwargs) | execute the Python method |
168 - allows executing any Python method by calling execute()
169 - sets the following attributes required to push the results to DB:
173 - doesn't fulfill details when pushing the results to the DB.
176 ### execute(**kwargs)
178 - the subclasses must override the default implementation which is false on purpose
179 - the new implementation must return 0 if success or anything else if failure.
183 ## Your second test case
189 #!/usr/bin/env python
191 from functest.core import feature
193 class Test(feature.Feature):
195 def execute(self, **kwargs):
201 ### functest/ci/testcases.yaml
205 project_name: functest
222 class designed to run any bash command
225 ### execute(**kwargs)
227 execute the cmd passed as arg.
231 ## Your third test case
234 ### functest/ci/testcases.yaml
238 project_name: functest
246 module: 'functest.core.feature'
249 cmd: 'echo Hello World; exit 0'
257 base model for running unittest.TestSuite
262 - allows running any unittest.TestSuite
263 - sets the following attributes required to push the results to DB:
271 ## Your fourth test case
277 #!/usr/bin/env python
281 class TestStringMethods(unittest.TestCase):
283 def test_upper(self):
284 self.assertEqual('Hello World'.upper(),
289 ### functest/ci/testcases.yaml
293 project_name: functest
301 module: 'functest.core.unit'
314 - __to finish VNF abstraction (coverage + pylint)__
316 - to manage criteria as written in testcases.yaml
318 Please see [Functest Euphrates page](https://wiki.opnfv.org/display/functest/Functest+Euphrates+page) for more details