functest.git
7 years agoConform constants.py with the minimal standards 77/34877/5
Cédric Ollivier [Wed, 17 May 2017 05:44:27 +0000 (07:44 +0200)]
Conform constants.py with the minimal standards

It adds the shebang to constants.py and its dependencies and stops
importing from a relative path. It stops printing obsolete values too.

It adds python3 support to them by the way.

Change-Id: Ie7bddcf253eb245844d9e11c17acc69943c1ba63
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoFix an unorderable types error (if python 3) 75/34875/1
Cédric Ollivier [Wed, 17 May 2017 05:34:02 +0000 (07:34 +0200)]
Fix an unorderable types error (if python 3)

Change-Id: I07c5a548f10a1779faead4d9c596a1976aff8866
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoAdd Python3 support to the ODL testcase 73/34873/2
Cédric Ollivier [Wed, 17 May 2017 04:06:51 +0000 (06:06 +0200)]
Add Python3 support to the ODL testcase

It also adds this support to its dependencies.

Change-Id: I0534f0c7b0e15a9ee89f522f314cf5200874454c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoUpdate/Add requirements to run ODL robot CSIT 69/34569/3
Jamo Luhrsen [Wed, 10 May 2017 17:27:14 +0000 (10:27 -0700)]
Update/Add requirements to run ODL robot CSIT

It conforms test_odl.py with the new RobotFramework API too.
It also updates test-requirements.txt as required by the current
implementation of run_unit_tests.sh.

Co-Authored-By: Cédric Ollivier <cedric.ollivier@orange.com>
Change-Id: Ibca0150a40d3b8a3f736be1e9dcbf9ce56af0299
Signed-off-by: Jamo Luhrsen <jluhrsen@redhat.com>
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMerge "Fix one minor typo"
Jose Lausuch [Tue, 16 May 2017 16:05:16 +0000 (16:05 +0000)]
Merge "Fix one minor typo"

7 years agoMerge "Fully cover the decorator"
Jose Lausuch [Tue, 16 May 2017 16:03:12 +0000 (16:03 +0000)]
Merge "Fully cover the decorator"

7 years agoMerge "Assign the wrapped function’s attributes"
Jose Lausuch [Tue, 16 May 2017 16:03:10 +0000 (16:03 +0000)]
Merge "Assign the wrapped function’s attributes"

7 years agoMerge "Additional Unit Tests for core modules"
Jose Lausuch [Tue, 16 May 2017 15:23:41 +0000 (15:23 +0000)]
Merge "Additional Unit Tests for core modules"

7 years agoMerge "Get a logger before printing"
Jose Lausuch [Tue, 16 May 2017 15:23:27 +0000 (15:23 +0000)]
Merge "Get a logger before printing"

7 years agoAdd enabled flag in testcases.yaml 33/34833/9
jose.lausuch [Tue, 16 May 2017 09:28:14 +0000 (11:28 +0200)]
Add enabled flag in testcases.yaml

This will allow enable/disable test cases directly
using true/false.

Change-Id: Iaa7d051ebb8dd9e5a6f835a823f58f1e17a87b8d
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
7 years agoSwitch from generate_report to PrettyTable 21/34621/8
Cédric Ollivier [Thu, 11 May 2017 11:01:16 +0000 (13:01 +0200)]
Switch from generate_report to PrettyTable

run_tests.py now relies on PrettyTable as most of the openstack clients.
generate_report.py and its related unit tests are simply removed.

It sets padding_width=5 in testcase.py too to conform with
run_tests.py.

Now report is printed in every case.

Change-Id: Id9ce93f984503f25d6a2150482f397853fa3dd64
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoUpdate framework pres 71/34471/5
Cédric Ollivier [Sat, 6 May 2017 17:54:38 +0000 (19:54 +0200)]
Update framework pres

It takes into account the next framework modifications:
  - Switch TestCase attribute criteria to result [1]
  - Manage criteria in TestCase [2]
  - Print the real testcase duration [3]
  - Rename TestCase check_result() to is_successful() [4]

[1] https://gerrit.opnfv.org/gerrit/#/c/33695/
[2] https://gerrit.opnfv.org/gerrit/#/c/33697/
[3] https://gerrit.opnfv.org/gerrit/#/c/34369/
[4] https://gerrit.opnfv.org/gerrit/#/c/34373/

Change-Id: I3a6790e4233ef4c55b0535618dacf702f779f28b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoFix how to get tier_name 41/34541/7
Cédric Ollivier [Wed, 10 May 2017 09:38:32 +0000 (11:38 +0200)]
Fix how to get tier_name

The tier name was incorrect when passing a test case as run_test.py arg.

Change-Id: I6f439fc7b90facb3f0553798bd1fe06ac9f09475
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMerge "Modify TestCase.__str__() to use PrettyTable"
Jose Lausuch [Tue, 16 May 2017 11:47:14 +0000 (11:47 +0000)]
Merge "Modify TestCase.__str__() to use PrettyTable"

7 years agoFix one minor typo 75/34775/1
Cédric Ollivier [Mon, 15 May 2017 14:09:01 +0000 (16:09 +0200)]
Fix one minor typo

Change-Id: Id17070985adfbe7817f3be25b47f583cb9ef90b6
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoAdditional Unit Tests for core modules 45/28545/14
ashishk1994 [Mon, 13 Feb 2017 17:55:58 +0000 (23:25 +0530)]
Additional Unit Tests for core modules

This patch adds unit tests for vnf and pytest_suite_runner.
It also fixes several pylint warnings/errors.

Additional tests are required to check args values.

Co-Authored-By: Cédric Ollivier <cedric.ollivier@orange.com>
Change-Id: I75a829233481655c794402830e1da037ab7cdb8d
Signed-off-by: ashishk1994 <ashishk.iiit@gmail.com>
7 years agoGet a logger before printing 71/34771/1
Cédric Ollivier [Mon, 15 May 2017 13:47:33 +0000 (15:47 +0200)]
Get a logger before printing

It also fixes pylint errors.

Change-Id: I0d973360574a517724d0be3986c788c3ce7f54d4
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoFully cover the decorator 69/34769/1
Cédric Ollivier [Mon, 15 May 2017 12:55:12 +0000 (14:55 +0200)]
Fully cover the decorator

Change-Id: I1f1c938f0715e6afb7edbb8242f9a6c9cbfa65b5
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoAssign the wrapped function’s attributes 47/34747/1
Cédric Ollivier [Sun, 14 May 2017 06:50:32 +0000 (08:50 +0200)]
Assign the wrapped function’s attributes

It mainly assigns to the wrapper push_results_to_db's __name__,
__module__ and __doc__ and the documentation string.

Change-Id: Idc882ac4e743699a4887e46c5066e0983c90afd9
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoUpgrade robotframework-sshlibrary to 2.1.3 31/31931/2
George Paraskevopoulos [Mon, 27 Mar 2017 16:43:52 +0000 (19:43 +0300)]
Upgrade robotframework-sshlibrary to 2.1.3

Robotframework sshlibrary monkeypatches paramiko runtime and version
2.1.1 causes
2017-03-27 15:24:23,196 - SSH utils - ERROR - _custom_start_client() got
an unexpected keyword argument 'timeout'

errors in SFC test because of the issue
https://github.com/robotframework/SSHLibrary/issues/158

This issue is resolved in version 2.1.3

Change-Id: I07f570c1a90f358b9f17aa6c959642d67e07a7e4
Signed-off-by: George Paraskevopoulos <geopar@intracom-telecom.com>
7 years agoMerge "Add Energy recording support"
Morgan Richomme [Fri, 12 May 2017 06:33:24 +0000 (06:33 +0000)]
Merge "Add Energy recording support"

7 years agoModify TestCase.__str__() to use PrettyTable 87/34587/3
Cédric Ollivier [Thu, 11 May 2017 08:03:47 +0000 (10:03 +0200)]
Modify TestCase.__str__() to use PrettyTable

It adds PrettyTable as requirement even if it's already defined in
OpenStack client dependencies.

If the TestCase object is considered as invalid, it simply returns the
default str.

Change-Id: Iee788aef2a13694d9482560977cbbf21c7f2c967
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMerge "Conform ODL with last framework updates"
Morgan Richomme [Wed, 10 May 2017 07:54:08 +0000 (07:54 +0000)]
Merge "Conform ODL with last framework updates"

7 years agoMerge changes from topic 'add_testcase_str'
Morgan Richomme [Wed, 10 May 2017 07:53:46 +0000 (07:53 +0000)]
Merge changes from topic 'add_testcase_str'

* changes:
  Remove the former results in DB url
  Implement TestCase __str__()

7 years agoMerge "Bugfix: initialize the logger for snaps"
Cedric Ollivier [Tue, 9 May 2017 17:26:33 +0000 (17:26 +0000)]
Merge "Bugfix: initialize the logger for snaps"

7 years agoRemove the former results in DB url 01/34501/2
Cédric Ollivier [Tue, 9 May 2017 14:59:49 +0000 (16:59 +0200)]
Remove the former results in DB url

It conforms generate_report.py with the previous change [1].

[1] https://gerrit.opnfv.org/gerrit/#/c/33511/

Change-Id: I677ebd416f462a591439fc7b637c02d7964e6173
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoAdd Energy recording support 31/33931/11
Benoit HERARD [Thu, 27 Apr 2017 09:23:00 +0000 (11:23 +0200)]
Add Energy recording support

It adds helpers to send notifications to Energy recording API and
related unit tests.
It requires a dedicated section in functest config file
to set connectivity parameters to Energy recording API.

It is using shared API Recording at http://161.105.253.100:8888

Change-Id: Idcb74d1bf7341ccce7cc1c3926f22338ce24f714
Signed-off-by: Benoit HERARD <benoit.herard@orange.com>
7 years agoConform ODL with last framework updates 79/34479/2
Cédric Ollivier [Tue, 9 May 2017 10:06:57 +0000 (12:06 +0200)]
Conform ODL with last framework updates

Now ODL TestCase calculates the ratio between critical tests passed
and failed and saves it in result. Non-critical test cases can fail as
result doesn't take them into account [1].

It also updates default attribute values in TestCase and allows result
to be float.

[1] http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#setting-criticality

Change-Id: Id4a89271b5b1a90cd3c1e2b08591ff26ffaffee0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoRename VnfOnBoardingBase to VnfOnBoarding 69/34469/1
Cédric Ollivier [Tue, 9 May 2017 07:55:02 +0000 (09:55 +0200)]
Rename VnfOnBoardingBase to VnfOnBoarding

Change-Id: I514340fc6180e29b31ef11bdea9f719aa2bbb738
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoRename vnf_base to vnf 67/34467/1
Cédric Ollivier [Tue, 9 May 2017 07:50:02 +0000 (09:50 +0200)]
Rename vnf_base to vnf

Change-Id: I0b8b9b2e85717fc92a233e4f7344b3419421778d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoBugfix: initialize the logger for snaps 15/34415/3
Linda Wang [Mon, 8 May 2017 06:33:13 +0000 (06:33 +0000)]
Bugfix: initialize the logger for snaps

Testcases about snaps failed due to the absence of attribute 'logger'.

Change-Id: I78c85e51f63afa767fa73aa12faea066ad432371
Signed-off-by: Linda Wang <wangwulin@huawei.com>
7 years agoImplement TestCase __str__() 95/34395/2
Cédric Ollivier [Sat, 6 May 2017 07:32:36 +0000 (09:32 +0200)]
Implement TestCase __str__()

It's designed to be called by run_tests.py when printing the results of
test cases and when generating the global report.

Change-Id: If4a6f023ef2344bbc4f940d07dde4b776dce5d68
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMerge "Rename TestCase check_result() to is_successful()"
Cedric Ollivier [Sat, 6 May 2017 07:20:55 +0000 (07:20 +0000)]
Merge "Rename TestCase check_result() to is_successful()"

7 years agoMerge "Print the real testcase duration"
Jose Lausuch [Fri, 5 May 2017 14:33:49 +0000 (14:33 +0000)]
Merge "Print the real testcase duration"

7 years agoRename TestCase check_result() to is_successful() 73/34373/2
Cédric Ollivier [Fri, 5 May 2017 13:24:54 +0000 (15:24 +0200)]
Rename TestCase check_result() to is_successful()

Change-Id: I41cea33b4c1dae79e75231e7c7e069fc89736f34
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoPrint the real testcase duration 69/34369/3
Cédric Ollivier [Fri, 5 May 2017 12:38:27 +0000 (14:38 +0200)]
Print the real testcase duration

TestCase offers a new public method to calculate and print the duration.
It also adds the related unit tests and adapts run_tests.py.

Change-Id: Ib6a7e637dafacb9027146199aeb033c2dcb986c6
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMerge "Replace CONST.* by getattribute/setattr"
Jose Lausuch [Fri, 5 May 2017 13:50:54 +0000 (13:50 +0000)]
Merge "Replace CONST.* by getattribute/setattr"

7 years agoMerge "Extended the timeout for Opera"
Jose Lausuch [Fri, 5 May 2017 13:50:10 +0000 (13:50 +0000)]
Merge "Extended the timeout for Opera"

7 years agoReplace CONST.* by getattribute/setattr 27/34127/7
jose.lausuch [Wed, 3 May 2017 12:11:38 +0000 (14:11 +0200)]
Replace CONST.* by getattribute/setattr

Directories affected:
 - ci
 - core

and respective unit tests

Change-Id: I6a3d5aa68de29fc5a37ae543a067ff797eba33e6
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
7 years agoMerge "Add iniparse to requirements.txt"
Jose Lausuch [Fri, 5 May 2017 12:49:00 +0000 (12:49 +0000)]
Merge "Add iniparse to requirements.txt"

7 years agoMerge "Check open args in test_odl"
Jose Lausuch [Fri, 5 May 2017 12:22:20 +0000 (12:22 +0000)]
Merge "Check open args in test_odl"

7 years agoMerge "Replace project_name by case_name in result file name"
Morgan Richomme [Fri, 5 May 2017 11:35:44 +0000 (11:35 +0000)]
Merge "Replace project_name by case_name in result file name"

7 years agoReplace project_name by case_name in result file name 61/34361/1
Cédric Ollivier [Fri, 5 May 2017 11:01:20 +0000 (13:01 +0200)]
Replace project_name by case_name in result file name

It avoids erasing functest.log when project_name = functest.

Change-Id: Icae898abb8b51c89b79bb1124adfadab8b0b3b99
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoAdd iniparse to requirements.txt 59/34359/1
Linda Wang [Fri, 5 May 2017 10:11:50 +0000 (10:11 +0000)]
Add iniparse to requirements.txt

Package iniparse is absent for functest, while it is required by
rally.py

Change-Id: Ia50216cfebdc9033f6a5f4abc6338951aaa2e8ab
Signed-off-by: Linda Wang <wangwulin@huawei.com>
7 years agoCheck open args in test_odl 55/34355/1
Cédric Ollivier [Fri, 5 May 2017 09:23:18 +0000 (11:23 +0200)]
Check open args in test_odl

Change-Id: I4b0060502621c7491820b9aa42d8f2f63b06a9ae
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoAdd Danube 2.0 reporting status 53/34353/1
Morgan Richomme [Fri, 5 May 2017 09:19:15 +0000 (11:19 +0200)]
Add Danube 2.0 reporting status

Change-Id: Icc09b824a4e7906dfab031b679ba3d0aa86ab0cd
Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
7 years agoMerge "Forbid calling generate_report from cmdline"
Jose Lausuch [Fri, 5 May 2017 07:51:39 +0000 (07:51 +0000)]
Merge "Forbid calling generate_report from cmdline"

7 years agoMerge "Define loggers as class-private members"
Jose Lausuch [Fri, 5 May 2017 07:51:35 +0000 (07:51 +0000)]
Merge "Define loggers as class-private members"

7 years agoMerge "Delete functest.utils.functest_logger"
Jose Lausuch [Fri, 5 May 2017 07:50:31 +0000 (07:50 +0000)]
Merge "Delete functest.utils.functest_logger"

7 years agoForbid calling generate_report from cmdline 43/34143/8
Cédric Ollivier [Wed, 3 May 2017 13:58:23 +0000 (15:58 +0200)]
Forbid calling generate_report from cmdline

Change-Id: Ic9ea7507dc5f1f781b57c4573e971a2c8a8ca9ef
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoDefine loggers as class-private members 13/34113/10
Cédric Ollivier [Tue, 2 May 2017 18:57:08 +0000 (20:57 +0200)]
Define loggers as class-private members

This mangling ensures that all info messages printed from core packages
are shown in console. It also avoids sphinx to print them.

Change-Id: I07db9f33060c195bce3b48b06a6640eb6c56c2eb
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoDelete functest.utils.functest_logger 19/34019/13
Cédric Ollivier [Sun, 30 Apr 2017 06:52:55 +0000 (08:52 +0200)]
Delete functest.utils.functest_logger

It deletes functest.utils.functest_logger and the related unit tests.
Then it modifies all functest modules to get all loggers via
logging.getLogger(). __name__ is mainly used as getLogger arg as
proposed by logging [1].

All loggers and handlers are now defined via functest/ci/logging.ini
instead of a dict loaded by an external json file.

Now only warn messages and info messages from ci and core packages are
printed in console.

[1] https://docs.python.org/2/library/logging.html

Change-Id: Ic192855e0f9bf94825d8f7ec73549a0f3b8d44c5
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMerge "Update tempest.conf to enable tempest from external server"
Jose Lausuch [Thu, 4 May 2017 12:13:08 +0000 (12:13 +0000)]
Merge "Update tempest.conf to enable tempest from external server"

7 years agoMerge "Update rally_conf.json for creating deployment"
Jose Lausuch [Thu, 4 May 2017 12:12:38 +0000 (12:12 +0000)]
Merge "Update rally_conf.json for creating deployment"

7 years agoMerge "Add the choice of interface for keystoneclient"
Jose Lausuch [Thu, 4 May 2017 12:11:59 +0000 (12:11 +0000)]
Merge "Add the choice of interface for keystoneclient"

7 years agoMerge "Publish framework presentation"
Cedric Ollivier [Thu, 4 May 2017 07:45:56 +0000 (07:45 +0000)]
Merge "Publish framework presentation"

7 years agoMerge changes from topic 'remove_openstack_dir'
Morgan Richomme [Thu, 4 May 2017 07:23:17 +0000 (07:23 +0000)]
Merge changes from topic 'remove_openstack_dir'

* changes:
  Remove openstack directory
  Remove create_instance_and_ip.py

7 years agoMerge "refstack-client userguide"
Morgan Richomme [Thu, 4 May 2017 07:22:41 +0000 (07:22 +0000)]
Merge "refstack-client userguide"

7 years agoExtended the timeout for Opera 91/34191/1
helenyao [Thu, 4 May 2017 02:05:29 +0000 (10:05 +0800)]
Extended the timeout for Opera

Change-Id: Ifeb4aecf66d75c9bb98d5dafdbe20be78423565e
Signed-off-by: helenyao <yaohelan@huawei.com>
7 years agoMerge "Update INFO file for Euphrates"
Jose Lausuch [Wed, 3 May 2017 10:34:56 +0000 (10:34 +0000)]
Merge "Update INFO file for Euphrates"

7 years agoRemove openstack directory 21/34121/1
jose.lausuch [Wed, 3 May 2017 10:32:49 +0000 (12:32 +0200)]
Remove openstack directory

It is useles right now and won't be used in the future.

Change-Id: I9aa121544051f49f7e7e794ae3b2c4a9ea516e47
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
7 years agoUpdate tempest.conf to enable tempest from external server 05/34105/3
Linda Wang [Wed, 3 May 2017 07:08:03 +0000 (07:08 +0000)]
Update tempest.conf to enable tempest from external server

1. The endpoint type to use for OpenStack Identity API v3
   "v3_endpoint_type" is set to be "adminURL" by default, which should
   be updated to the env OS_ENDPOINT_TYPE with the value "publicURL"
2. Reset the identity region name to "RegionOne"
3. For keystone api v3 in Newton, 5 testcases aimming at identity admin
   v2 test should be omitted.

JIRA: FUNCTEST-801

Change-Id: I47ae41a80de0a52d32d25c9783860955f465099c
Signed-off-by: Linda Wang <wangwulin@huawei.com>
7 years agoMerge "Updated basic attributes for opera testcase"
Jose Lausuch [Wed, 3 May 2017 07:54:16 +0000 (07:54 +0000)]
Merge "Updated basic attributes for opera testcase"

7 years agorefstack-client userguide 49/33049/5
MatthewLi [Thu, 6 Apr 2017 08:26:37 +0000 (04:26 -0400)]
refstack-client userguide

JIRA: FUNCTEST-758

container setup steps are skipped, since they already
included in the functest userguide documents.

Change-Id: I729d4c3174b03706624a7235da24cb3043b28c2c
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
7 years agoUpdated basic attributes for opera testcase 37/34037/2
helenyao [Tue, 2 May 2017 07:03:22 +0000 (15:03 +0800)]
Updated basic attributes for opera testcase

The original value as unknown is not reasonable.

Change-Id: I23672ea70fff9b3badb2d2cf4f32dc3e038d7bb3
Signed-off-by: helenyao <yaohelan@huawei.com>
7 years agoMerge "Remove missing statements in test_odl"
Jose Lausuch [Tue, 2 May 2017 21:06:17 +0000 (21:06 +0000)]
Merge "Remove missing statements in test_odl"

7 years agoMerge "Restrict coverage output to all functest packages"
Jose Lausuch [Tue, 2 May 2017 21:05:49 +0000 (21:05 +0000)]
Merge "Restrict coverage output to all functest packages"

7 years agoMerge "Fix pylint warnings in snaps"
Jose Lausuch [Tue, 2 May 2017 14:29:51 +0000 (14:29 +0000)]
Merge "Fix pylint warnings in snaps"

7 years agoRemove create_instance_and_ip.py 51/34051/1
jose.lausuch [Tue, 2 May 2017 11:58:51 +0000 (13:58 +0200)]
Remove create_instance_and_ip.py

JIRA: FUNCTEST-817

Change-Id: Ib297ec656a19c9ea4e06c6f2e133ed514a1dbf35
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
7 years agoMerge "rename tempest.conf to refstack_tempest.conf"
Jose Lausuch [Tue, 2 May 2017 11:50:20 +0000 (11:50 +0000)]
Merge "rename tempest.conf to refstack_tempest.conf"

7 years agoPublish framework presentation 39/34039/2
Cédric Ollivier [Thu, 20 Apr 2017 14:58:41 +0000 (16:58 +0200)]
Publish framework presentation

This is the first version prepared before merging "Switch TestCase
attribute criteria to result" [1].
It will also be updated to integrate VNF classes as soon as they are
ready (UT, pylint, docstrings...).

[1] https://gerrit.opnfv.org/gerrit/#/c/33695/

Change-Id: I2fc8f4ae377afee70e083cd51e5725eccda15431
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoAdd the choice of interface for keystoneclient 19/33919/4
Linda Wang [Thu, 27 Apr 2017 07:01:24 +0000 (07:01 +0000)]
Add the choice of interface for keystoneclient

1. When functest is running on external host not jumpserver,
   keystoneclient is used to retrieve or create projects/tenants
   and users, which would fail to establish connection to the admin
   endpoint.
2. Keystoneclient will connect to the public endpoint by setting the
   env OS_INTERFACE to be public.
3. Remove the admin endpoint check in check_os.sh.

JIRA: FUNCTEST-801

Change-Id: I324f5c8fdf61447319070a23f6b6bc6b1bbc1d48
Signed-off-by: Linda Wang <wangwulin@huawei.com>
7 years agoRemove missing statements in test_odl 17/34017/1
Cédric Ollivier [Sun, 30 Apr 2017 08:29:29 +0000 (10:29 +0200)]
Remove missing statements in test_odl

It mainly removes _fake_url_for() which is useless since get_endpoint()
is not called for identity.

Coverage for functest/tests/unit/odl/test_odl.py : 100%

Change-Id: I5809c5eac4021c44113e70379621ab311187c669
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoRestrict coverage output to all functest packages 15/34015/1
Cédric Ollivier [Sun, 30 Apr 2017 07:38:55 +0000 (09:38 +0200)]
Restrict coverage output to all functest packages

The next packages/modules are not yet covered:
  - functest/cli/setup.py
  - functest/core/pytest_suite_runner.py
  - functest/opnfv_tests/mano
  - functest/opnfv_tests/openstack/examples
  - functest/opnfv_tests/openstack/snaps
  - functest/opnfv_tests/openstack/vping
  - functest/opnfv_tests/sdn/onos
  - functest/opnfv_tests/vnf/aaa
  - functest/opnfv_tests/vnf/ims
  - functest/opnfv_tests/vnf/router
  - functest/utils/openstack

Change-Id: Id62749796864537615f0ca8b162ce77b6c1a3309
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoFix pylint warnings in snaps 13/34013/1
Cédric Ollivier [Sun, 30 Apr 2017 07:35:05 +0000 (09:35 +0200)]
Fix pylint warnings in snaps

Pylint only warns about missing module docstrings now.

Change-Id: I989ff800cd2628fed80a2d3510ef37ce2876cbd4
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoUpdate rally_conf.json for creating deployment 21/33921/1
Linda Wang [Thu, 27 Apr 2017 07:36:32 +0000 (07:36 +0000)]
Update rally_conf.json for creating deployment

1. The rally_conf.json is obtained via def get_credentials_for_rally(),
   where the parameter endpoint_type is supposed to get from the env
   OS_ENDPOINT_TYPE or OS_INTERFACE, just as the def
   get_creds_from_env_vars() in rally project [1].

[1] https://github.com/openstack/rally/blob/master/rally/cli/envutils.py

JIRA: FUNCTEST-801

Change-Id: I6b6be82b9ff7f04948b47c4fca836d415fe13f8b
Signed-off-by: Linda Wang <wangwulin@huawei.com>
7 years agoFix the instance initialization error for OnosSfc 33/33833/4
helenyao [Wed, 26 Apr 2017 09:10:47 +0000 (17:10 +0800)]
Fix the instance initialization error for OnosSfc

Change-Id: I1820bed1bbb7dbddd3923ea6439f58592595f5c8
Signed-off-by: helenyao <yaohelan@huawei.com>
7 years agoLogger must be a singleton 69/33869/1
Cédric Ollivier [Wed, 26 Apr 2017 13:31:19 +0000 (15:31 +0200)]
Logger must be a singleton

It avoids creating a python object for every message printed.

Change-Id: Ia42b59db9f7ce4eadc7e91da4d193db646b54d85
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoFix false positives in test_tempest 23/33823/2
Cédric Ollivier [Tue, 25 Apr 2017 12:33:41 +0000 (14:33 +0200)]
Fix false positives in test_tempest

It fixes the return values and the side effects expected from the next
methods:
  - configure_tempest
  - create_tempest_resources
  - generate_test_list
  - apply_tempest_blacklist
  - run_verifier_tests
  - parse_verifier_result

JIRA: FUNCTEST-803

Change-Id: Ib7dbd28e003cf0f27f497ad8e63cf7ae5ce564de
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoCatch os.makedirs() exceptions in tempest.py 21/33821/2
Cédric Ollivier [Wed, 26 Apr 2017 07:18:04 +0000 (09:18 +0200)]
Catch os.makedirs() exceptions in tempest.py

It can raise an error exception if the leaf directory already exists or
cannot be created [1].

[1] https://docs.python.org/2/library/os.html

JIRA: FUNCTEST-804

Change-Id: Ie410bccf1bb2dfd36df6e918c152ecf94844b5d7
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoManage criteria in TestCase 97/33697/3
Cédric Ollivier [Mon, 24 Apr 2017 06:58:20 +0000 (08:58 +0200)]
Manage criteria in TestCase

It converts all criteria values to the corresponding percent in
functest/ci/testcases.yaml.

Result is expected to be equal or greater than criteria. If both are 0,
result is considered as false.

It is compatible with the old behavior but warns to update.
It will allow a safer remove.

It also fixes a bug in test_tempest to allow merging [1] and
tier_handler.py which required that type criteria was str.

[1] https://gerrit.opnfv.org/gerrit/#/c/27949/

Change-Id: Ib6edcfa3103b7d51b0bdc83119f1cea2a8be9fbc
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoSwitch TestCase attribute criteria to result 95/33695/2
Cédric Ollivier [Sun, 23 Apr 2017 06:59:43 +0000 (08:59 +0200)]
Switch TestCase attribute criteria to result

It mainly avoids mixing input and output.
Criteria is now an input set in functest/ci/testcases.yaml and then
must be passed as __init__() args (which will be proposed in an
additional change).

Then it also renames the related TestCase method to
check_result().

Change-Id: Ifc3c8e3ea6cde7e3edf7174bed4bf2bf0894e8e3
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMerge "Remove the useless opnfv_tests dir in tests"
Cedric Ollivier [Wed, 26 Apr 2017 07:32:47 +0000 (07:32 +0000)]
Merge "Remove the useless opnfv_tests dir in tests"

7 years agoRemove the useless opnfv_tests dir in tests 11/33711/1
Cédric Ollivier [Mon, 24 Apr 2017 13:34:10 +0000 (15:34 +0200)]
Remove the useless opnfv_tests dir in tests

Change-Id: I41a8db181adf6c0c67b9de8380c3ccdd1ad3b529
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoRemove useless lines of code in barometer.py 09/33709/1
Cédric Ollivier [Mon, 24 Apr 2017 12:35:04 +0000 (14:35 +0200)]
Remove useless lines of code in barometer.py

Change-Id: Ie312dbd296a85a35096294331c28a45f8dc10d0c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agorename tempest.conf to refstack_tempest.conf 65/33665/1
MatthewLi [Sat, 22 Apr 2017 09:30:51 +0000 (05:30 -0400)]
rename tempest.conf to refstack_tempest.conf

since for tempest smoke/full, etc, will also use a file named
tempest.conf this wil cover the refstack tempest.conf,
rename it to refstack_tempest.conf, then easily for users to
debug to check if the configure file is correct or not.

Change-Id: Ic2f93fb05effb1a61276cb6ba755076a0404caad
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
7 years agoSwitch testcase to test case in docstrings 25/33625/1
Cédric Ollivier [Fri, 21 Apr 2017 09:55:36 +0000 (11:55 +0200)]
Switch testcase to test case in docstrings

Change-Id: Iea64673788a3d5ab5eb3028522f659de183440ed
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMerge "Minor changes in docstrings"
Serena Feng [Fri, 21 Apr 2017 09:05:15 +0000 (09:05 +0000)]
Merge "Minor changes in docstrings"

7 years agoMerge "Ignore _build (sphinx output dir)"
Jose Lausuch [Fri, 21 Apr 2017 09:03:48 +0000 (09:03 +0000)]
Merge "Ignore _build (sphinx output dir)"

7 years agoMerge "Remove exec right for ci config files"
Cedric Ollivier [Fri, 21 Apr 2017 07:42:16 +0000 (07:42 +0000)]
Merge "Remove exec right for ci config files"

7 years agoMerge "Fix pylint warnings in decorators.py"
Cedric Ollivier [Fri, 21 Apr 2017 07:42:03 +0000 (07:42 +0000)]
Merge "Fix pylint warnings in decorators.py"

7 years agoMerge "Stop hardcoding results suffix in push_to_db"
Cedric Ollivier [Fri, 21 Apr 2017 07:41:56 +0000 (07:41 +0000)]
Merge "Stop hardcoding results suffix in push_to_db"

7 years ago[VNF_base] Support existing tenant and user 89/33589/3
boucherv [Thu, 20 Apr 2017 14:20:57 +0000 (16:20 +0200)]
[VNF_base] Support existing tenant and user

Support existing tenant and user with a get_or_create_tenant method.
If the tenant and user already exist it only get the id of those

JIRA: FUNCTEST-799

Change-Id: Id5daf2904592e42199471bdd3547e861e6db6e32
Signed-off-by: boucherv <valentin.boucher@orange.com>
7 years agoRemove exec right for ci config files 95/33595/1
Cédric Ollivier [Thu, 20 Apr 2017 15:07:50 +0000 (17:07 +0200)]
Remove exec right for ci config files

Change-Id: Ibf7e549252234fb69dd6e0cc7920b66759469aea
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoIgnore _build (sphinx output dir) 93/33593/1
Cédric Ollivier [Thu, 20 Apr 2017 15:04:46 +0000 (17:04 +0200)]
Ignore _build (sphinx output dir)

Change-Id: I8afb00fb2942d2144968896981532324ac8e322d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoFix pylint warnings in decorators.py 85/33585/1
Cédric Ollivier [Thu, 20 Apr 2017 13:24:47 +0000 (15:24 +0200)]
Fix pylint warnings in decorators.py

Change-Id: Ie6c53feb97053248c909d641753b44b5f51c8719
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMinor changes in docstrings 79/33579/2
Cédric Ollivier [Thu, 20 Apr 2017 12:52:47 +0000 (14:52 +0200)]
Minor changes in docstrings

It removes case_name from the list of attributes that run() must set
because it's now managed by run_tests.py (see __init__())

Change-Id: Id6228880257d0e9fb27483c56aa9985197feb04e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoStop hardcoding results suffix in push_to_db 11/33511/2
MatthewLi [Wed, 19 Apr 2017 06:21:14 +0000 (02:21 -0400)]
Stop hardcoding results suffix in push_to_db

Adding results in url is useless as it is defined in results
section in config_functest.yaml.

It allows defining the full name in case of dumping to file (see
decorator.py). The commented local file is the value required by
dovetail.

Co-Authored-By: Cédric Ollivier <cedric.ollivier@orange.com>
Change-Id: I35ba9b34d99e138292f099934e28585ee8b85604
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoAdd mock to requirements.txt 05/33505/1
Linda Wang [Wed, 19 Apr 2017 02:18:26 +0000 (02:18 +0000)]
Add mock to requirements.txt

Package mock is absent for functest, while it is required by
decorators.py.

Change-Id: I0d042def0335e369c4e22a349e3068829e1d2c59
Signed-off-by: Linda Wang <wangwulin@huawei.com>
7 years agoUpdate INFO file for Euphrates 03/33503/1
jose.lausuch [Tue, 18 Apr 2017 19:56:15 +0000 (21:56 +0200)]
Update INFO file for Euphrates

Change-Id: Iddc36c0c9d3ce897c15393479e7ccb2d2a3230c9
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>