created by [Cédric Ollivier](mailto:cedric.ollivier@orange.com)
-2017/04/24
+2017/06/05
Note:
-- Functest integrates lots of heteregeounous testcases:
+- Functest integrates lots of heterogeneous testcases:
- python vs bash
- internal vs external
- it aims to benefit from object programming
### Our target
- limit run_tests.py instructions by defining:
- - the basic testcase attritutes
+ - the basic testcase attributes
- all common operations
- the status codes expected
- avoid duplicating codes between testcases
-- ease the developpement of third-party testcases (aka features)
+- ease the development of third-party testcases (aka features)
- project_name (default: 'functest')
- case_name
- criteria
+- result
- start_time
- stop_time
- details
### methods
-| Method | Purpose |
-|-------------------|--------------------------------------------|
-| run(**kwargs) | run the test case |
-| check_criteria() | interpret the results of the test case |
-| push_to_db() | push the results of the test case to the DB|
+| Method | Purpose |
+|-------------------|------------------------------|
+| run(**kwargs) | run the test case |
+| is_successful() | interpret the results |
+| get_duration() | return the duration |
+| push_to_db() | push the results to the DB |
+| clean() | clean the resources |
### run(**kwargs)
- the subclasses must override the default implementation which is false on purpose
- the new implementation must set the following attributes to push the results to DB:
- - criteria
+ - result
- start_time
- stop_time
if result == testcase.TestCase.EX_OK:
if GlobalVariables.REPORT_FLAG:
test_case.push_to_db()
- result = test_case.check_criteria()
+ result = test_case.is_successful()
```
def run(self, **kwargs):
self.start_time = time.time()
print "Hello World"
- self.criteria = 'PASS'
+ self.result = 100
self.stop_time = time.time()
return testcase.TestCase.EX_OK
```
```yaml
case_name: first
project_name: functest
-criteria: 'status == "PASS"'
+criteria: 100
blocking: true
-clean_flag: false
description: ''
dependencies:
installer: ''
- allows executing any Python method by calling execute()
- sets the following attributes required to push the results to DB:
- - criteria
+ - result
- start_time
- stop_time
- doesn't fulfill details when pushing the results to the DB.
```yaml
case_name: second
project_name: functest
-criteria: 'status == "PASS"'
+criteria: 100
blocking: true
-clean_flag: false
description: ''
dependencies:
installer: ''
```
case_name: third
project_name: functest
-criteria: 'status == "PASS"'
+criteria: 100
blocking: true
-clean_flag: false
description: ''
dependencies:
installer: ''
-## Euphrates
+## class Suite
+bases: TestCase
+
+base model for running unittest.TestSuite
+
+
+### run(**kwargs)
+
+- allows running any unittest.TestSuite
+- sets the following attributes required to push the results to DB:
+ - result
+ - start_time
+ - stop_time
+ - details
+
+
+
+## Your fourth test case
+
+
+### fourth.py
+
+```python
+#!/usr/bin/env python
+
+import unittest
+
+class TestStringMethods(unittest.TestCase):
+
+ def test_upper(self):
+ self.assertEqual('Hello World'.upper(),
+ 'HELLO WORLD')
+```
+
+
+### functest/ci/testcases.yaml
+
+```
+case_name: fourth
+project_name: functest
+criteria: 100
+blocking: true
+description: ''
+dependencies:
+ installer: ''
+ scenario: ''
+run:
+ module: 'functest.core.unit'
+ class: 'Suite'
+ args:
+ name: 'fourth'
+```
+
+
+
+## class VNF
+bases: TestCase
+
+base model for VNF onboarding testing
+
+
+### methods
+
+| Method | Purpose |
+|-----------------------|---------------------------------------------------|
+| prepare() | prepare VNF env (user, tenant, security group,..) |
+| run(**kwargs) | run VNF test case |
+| deploy_orchestrator() | deploy cloudify, ONAP, OpenBaton,... (optional) |
+| deploy_vnf() | deploy the VNF |
+| test_vnf() | run tests on the VNF |
+
+
+### run(**kwargs)
+
+- deploys an orchestrator if needed (e.g. heat, OpenBaton, Cloudify, ONAP, Juju)
+- deploys the VNF
+- performs tests on the VNF
+
+
+### prepare()
+
+- creates a user
+- creates a Tenant/Project
+- allocates admin role to the user on this tenant
+
+
+### deploy_orchestrator()
+
+- deploys an orchestrator (optional)
+- if this function is overridden then raise orchestratorDeploymentException if error during orchestrator deployment
+
+
+### deploy_vnf()
+
+- **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases.
+- The deployment can be executed via a specific orchestrator or using build-in orchestrators such as heat, openbaton, cloudify, juju, ONAP, ...
+- returns:
+ True if the VNF is properly deployed
+ False if the VNF is not deployed
+- raises VnfDeploymentException if error during VNF deployment
+
+
+### test_vnf()
+
+- **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases.
+- Once a VNF is deployed, it is assumed that specific test suite can be run to validate the VNF.
+- returns:
+ True if VNF tests are PASS
+ False if test suite is FAIL
+- raises VnfTestException if error during VNF tests
+
+
+
+## Your fifth test case
-### Next actions
+### fifth.py
-- __to finish VNF abstraction (coverage + pylint)__
-- to publish doc API
-- to manage criteria as written in testcases.yaml
+```python
+#!/usr/bin/env python
+
+from functest.core import vnf
+
+class Vnf(vnf.VnfOnBoarding):
+
+ def deploy_vnf(self):
+ print "Deploy your VNF here"
+ print "Feed orchestrator with VNF descriptor"
+ return 0
-Please see [Functest Euphrates page](https://wiki.opnfv.org/display/functest/Functest+Euphrates+page) for more details
+ def test_vnf(self):
+ print "Test your VNF here"
+ return 0
+```
+
+
+### functest/ci/testcases.yaml
+
+```yaml
+case_name: fifth
+project_name: functest
+criteria: 100
+blocking: true
+description: ''
+dependencies:
+ installer: ''
+ scenario: ''
+run:
+ module: 'fifth'
+ class: 'Vnf'
+```
Code Review / functest-xtesting.git / blobdiff