functest-xtesting.git
8 years agoMerge "Use Kingbird Bind host URL for Tempest"
mei mei [Tue, 9 Aug 2016 01:34:26 +0000 (01:34 +0000)]
Merge "Use Kingbird Bind host URL for Tempest"

8 years agoUpdate test with the new SF and new tacker script
Manuel Buil [Mon, 8 Aug 2016 15:11:11 +0000 (17:11 +0200)]
Update test with the new SF and new tacker script

- A new SF image was created with a new name. Several files must be updated
- There is a new tacker branch for OPNFV Colorado, an update was made

Change-Id: Ib5ed364467baca32fa204155655318cb1ec189f5
Signed-off-by: Manuel Buil <manuel.buil@ericsson.com>
8 years agoUse Kingbird Bind host URL for Tempest
Ashish Singh [Mon, 8 Aug 2016 15:41:18 +0000 (21:11 +0530)]
Use Kingbird Bind host URL for Tempest

Earlier, there was external URL for accessing KB services
in tempest.conf which is not accessible from functest docker.

Use bind host URL for KB service to access from the docker.

Change-Id: I147ffc2cd9883a6913080f9e1ab245807e0ca707
Signed-off-by: ashish singh <ashishsingh2k8@gmail.com>
8 years agoMerge "Enable Copper functest under JOID"
Morgan Richomme [Mon, 8 Aug 2016 15:11:16 +0000 (15:11 +0000)]
Merge "Enable Copper functest under JOID"

8 years agoMerge "Bug fixes odl-sfc + new features"
valentin boucher [Sun, 7 Aug 2016 11:03:01 +0000 (11:03 +0000)]
Merge "Bug fixes odl-sfc + new features"

8 years agoBug fixes odl-sfc + new features
Manuel Buil [Wed, 3 Aug 2016 15:59:07 +0000 (17:59 +0200)]
Bug fixes odl-sfc + new features

* Doing the test with private IPs as floating are currently not working in ODL
* New SF takes more time to start the service. A while loop is added.
* Configuration of the compute set-up due to ODL limitations added

Change-Id: I5449af940caee456262dac66bc47b66fd05c2a70
Signed-off-by: Manuel Buil <manuel.buil@ericsson.com>
8 years agoReduced timeout for SSH connection attempt in tempest.conf
vitikkan [Fri, 5 Aug 2016 04:18:18 +0000 (07:18 +0300)]
Reduced timeout for SSH connection attempt in tempest.conf

Some tempest test cases try to establish SSH connection to
Cirros VMs using default 300s timeout. Since Cirros VMs send
only three DHCP requests with 1 minute interval, future SSH
connection attempts will most probably fail if DHCP response
will not be received right after third request. For that reason
the optimal timeout is slightly above 2 minutes.

JIRA: FUNCTEST-378

Change-Id: I57415ad37a85cd84887d3cac4f7a44f7e145d820
Signed-off-by: vitikkan <viktor.tikkanen@nokia.com>
8 years agoMerge "Re-generating tempest.conf file before starting tests"
valentin boucher [Thu, 4 Aug 2016 08:22:56 +0000 (08:22 +0000)]
Merge "Re-generating tempest.conf file before starting tests"

8 years agoMerge "unify test result check for feature project and apply to parser"
valentin boucher [Thu, 4 Aug 2016 08:21:59 +0000 (08:21 +0000)]
Merge "unify test result check for feature project and apply to parser"

8 years agoMerge "support "" equals to None check in Testcase.is_compatible"
valentin boucher [Thu, 4 Aug 2016 08:20:52 +0000 (08:20 +0000)]
Merge "support "" equals to None check in Testcase.is_compatible"

8 years agoRe-generating tempest.conf file before starting tests
vitikkan [Thu, 4 Aug 2016 08:04:38 +0000 (11:04 +0300)]
Re-generating tempest.conf file before starting tests

"rally verify genconfig" command doesn't generate tempest.conf
file if it already exists in the used deployment. This can cause
troubles if some resources (images, flavors etc.) change between
consecutive runs of tempest. The change ensures that tempest.conf
is deleted before executing "rally verify genconfig".

JIRA: FUNCTEST-403

Change-Id: Icf5e380da3cb8757f4177b49028937355e2cf1d7
Signed-off-by: vitikkan <viktor.tikkanen@nokia.com>
8 years agounify test result check for feature project and apply to parser
SerenaFeng [Thu, 4 Aug 2016 03:25:15 +0000 (11:25 +0800)]
unify test result check for feature project and apply to parser

add check_test_result() method to unify test result process

JIRA: FUNCTEST-405

Change-Id: Iee4d2876cbbf372f8c3e3bf94448036669bc7f7e
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agosupport "" equals to None check in Testcase.is_compatible
SerenaFeng [Thu, 4 Aug 2016 03:07:05 +0000 (11:07 +0800)]
support "" equals to None check in Testcase.is_compatible

when check ci_installer/ci_scenario is None, we also check if ci_installer/ci_scenario is ""

JIRA: FUNCTEST-404

Change-Id: Ib8b7b9a98f6eb064d2b3648e5d1334990c69cb2f
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agoSet lnfo logging level for Doctor tests
Carlos Goncalves [Wed, 3 Aug 2016 18:22:32 +0000 (20:22 +0200)]
Set lnfo logging level for Doctor tests

JIRA: DOCTOR-63

Change-Id: Iaa8fb5989f69a8853c0e3e03ce87d58fb2d4309b
Signed-off-by: Carlos Goncalves <carlos.goncalves@neclab.eu>
8 years agoEnable Copper functest under JOID
blsaws [Wed, 3 Aug 2016 17:00:13 +0000 (10:00 -0700)]
Enable Copper functest under JOID

JIRA: COPPER-4

Change-Id: Ief49cea34ed15ba9c877bf4acd512f4d77cb5016
Signed-off-by: blsaws <bryan.sullivan@att.com>
8 years agoMerge "Shorter test case names are accepted for tempest blacklist file"
vitikkan [Wed, 3 Aug 2016 11:37:55 +0000 (11:37 +0000)]
Merge "Shorter test case names are accepted for tempest blacklist file"

8 years agoShorter test case names are accepted for tempest blacklist file
vitikkan [Wed, 3 Aug 2016 06:19:21 +0000 (09:19 +0300)]
Shorter test case names are accepted for tempest blacklist file

Blacklist file handling is changed so that also partial test case
names are accepted. For example, "tempest.api.object_storage"
entry will exclude all the test cases containing this string.

JIRA: FUNCTEST-401

Change-Id: I620be7dfd5d980a343d2f7cc7c0dbe40598bf65c
Signed-off-by: vitikkan <viktor.tikkanen@nokia.com>
8 years agofunctest utils - get_resolvconf_ns() Ensure that dns works with dns name resolution
boucherv [Wed, 3 Aug 2016 09:17:10 +0000 (11:17 +0200)]
functest utils - get_resolvconf_ns() Ensure that dns works with dns name resolution

JIRA: FUNCTEST-402

Change-Id: I5fa9204aafb13330eb810950aa7bc728f1a69162
Signed-off-by: boucherv <valentin.boucher@orange.com>
8 years agoMerge "Fix for the endpoint URL for kingbird"
mei mei [Wed, 3 Aug 2016 04:13:48 +0000 (04:13 +0000)]
Merge "Fix for the endpoint URL for kingbird"

8 years agoMerge "modfied to mute object storage test on joid."
valentin boucher [Tue, 2 Aug 2016 18:59:43 +0000 (18:59 +0000)]
Merge "modfied to mute object storage test on joid."

8 years agoFix for the endpoint URL for kingbird
Ashish Singh [Tue, 2 Aug 2016 07:41:35 +0000 (13:11 +0530)]
Fix for the endpoint URL for kingbird

Currently, there is internal IP address which is getting populated
in tempest.conf. Since the tempest are run inside functest docker container
we need to use public IP address for accessing Kingbird services.

Now, there will not be any installer specific endpoints as we had earlier.
All should communicate with public endpoints.

Change-Id: Id22e7e4d9ece9a0000a1186113627b82d6ba400a
Signed-off-by: ashish singh <ashishsingh2k8@gmail.com>
8 years agouse configured success_rate to check tempest test status
SerenaFeng [Tue, 2 Aug 2016 05:48:51 +0000 (13:48 +0800)]
use configured success_rate to check tempest test status

when checking tempest test results using success_rate
configured in testcases.yaml rather than written it in the code

JIRA: FUNCTEST-400

Change-Id: I2a34e3e032a0d94c6a4605fcf09a66c8f8f9b78c
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agoFix missing directory when calling bgpvpn tests
jose.lausuch [Mon, 1 Aug 2016 15:00:24 +0000 (17:00 +0200)]
Fix missing directory when calling bgpvpn tests

Change-Id: Ic9f6b9592202c3bed9d6835eeea7c14971e1c01f
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoMerge "Several bug fixes in ODL-SFC functest testcase"
Jose Lausuch [Mon, 1 Aug 2016 14:46:23 +0000 (14:46 +0000)]
Merge "Several bug fixes in ODL-SFC functest testcase"

8 years agoSeveral bug fixes in ODL-SFC functest testcase
Manuel Buil [Mon, 1 Aug 2016 08:09:10 +0000 (10:09 +0200)]
Several bug fixes in ODL-SFC functest testcase

JIRA: SFC-42

- SSH options added so that the ssh key is not needed
- IPTABLES change done before contacting nova, neutron...
- Fix a problem in the CI test (we need to CD the BASEDIR)

Change-Id: Id63c02a614bf31e5a33637c2bf95270bd1399ff8
Signed-off-by: Manuel Buil <manuel.buil@ericsson.com>
8 years agomodfied to mute object storage test on joid.
Narinder Gupta [Mon, 1 Aug 2016 03:33:39 +0000 (22:33 -0500)]
modfied to mute object storage test on joid.

In case of joid we implement ceph-radosgw for object storage which creates
swift api for object storage. Manually i can verify we can upload the
object. But functest is failing other installer removed the API but we
need in JOID for juju based deployment on Openstack. So this patch will
mute the object storage test cases for functest.

Change-Id: I21d2bdc665a9589630cfcacb278cfb65a1c15f11
Signed-off-by: Narinder Gupta <narinder.gupta@canonical.com>
8 years agoWrap call to the rest api with try-except while generating report
jose.lausuch [Sun, 31 Jul 2016 20:15:44 +0000 (22:15 +0200)]
Wrap call to the rest api with try-except while generating report

When the Rest api is not available for some reason
(see associated JIRA), the script fails and it gives also a false
output to Jenkins.

JIRA: FUNCTEST-399

Change-Id: I81622a2e22661889afcf49526c2421af257920a4
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoAdd onos_sfc scenario
wuwenbin2 [Fri, 29 Jul 2016 09:18:53 +0000 (17:18 +0800)]
Add onos_sfc scenario

JIRA:ONOSFW-172

Change-Id: I019edf79f8d343f794aa7af151d8e30c18e2c5f6
Signed-off-by: wuwenbin2 <wuwenbin2@huawei.com>
8 years agoAdd dir_repo_domino to fix bug
Ulas Kozat [Fri, 29 Jul 2016 17:09:05 +0000 (10:09 -0700)]
Add dir_repo_domino to fix bug

JIRA: DOMINO-19

Change-Id: Ib054b859569ef69e32cec650c7dd223199762763
Signed-off-by: Ulas Kozat <ulas.kozat@gmail.com>
8 years agoReturn with exit code in Doctor testcase
Carlos Goncalves [Fri, 29 Jul 2016 13:07:40 +0000 (15:07 +0200)]
Return with exit code in Doctor testcase

JIRA: DOCTOR-63

Change-Id: Ib142a6102a705ee571081651c650ced05f7d5d4f
Signed-off-by: Carlos Goncalves <carlos.goncalves@neclab.eu>
8 years agoReturn with exit code in Promise testcase
Carlos Goncalves [Fri, 29 Jul 2016 12:54:55 +0000 (14:54 +0200)]
Return with exit code in Promise testcase

JIRA: PROMISE-77

Change-Id: Ie856d7b58f72a253abf4b446e33f3c98d06c50c1
Signed-off-by: Carlos Goncalves <carlos.goncalves@neclab.eu>
8 years agoReturn -1 in Rally if the success criteria is not met
Markus Kullberg [Fri, 29 Jul 2016 08:06:29 +0000 (11:06 +0300)]
Return -1 in Rally if the success criteria is not met

JIRA: FUNCTEST-382

Change-Id: Iaf475ecd3dbfcf6fb639ff146fe96e54c5393a1e
Signed-off-by: Markus Kullberg <markus.kullberg@nokia.com>
8 years agoMerge "Select test cases in tempest to be blacklisted per each scenario"
jose.lausuch [Fri, 29 Jul 2016 10:13:58 +0000 (10:13 +0000)]
Merge "Select test cases in tempest to be blacklisted per each scenario"

8 years agocorrect rally criteria check
SerenaFeng [Fri, 29 Jul 2016 09:07:44 +0000 (17:07 +0800)]
correct rally criteria check

add method check_success_rate() to unify criteria check by success_rate

JIRA: FUNCTEST-398

Change-Id: I1b3f0ccd7061b80ac88356206f29844e31e4101b
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agounify test results logger and apply to parser
SerenaFeng [Fri, 29 Jul 2016 07:16:14 +0000 (15:16 +0800)]
unify test results logger and apply to parser

refine a unified method to log the test results
and use it to parser.

JIRA: FUNCTEST-397

Change-Id: I5ed6b04107be5590b1d9d5683c8f1846fef5a4d4
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agoSelect test cases in tempest to be blacklisted per each scenario
jose.lausuch [Thu, 28 Jul 2016 19:09:39 +0000 (21:09 +0200)]
Select test cases in tempest to be blacklisted per each scenario

JIRA: FUNCTEST-396

Added example for SDNVPN test cases

Change-Id: I623130fc330c95239d982097552ca09f2a7ea34a
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoMerge "Bug fix of the CI script in the ODL-SFC test case"
Jose Lausuch [Thu, 28 Jul 2016 15:49:27 +0000 (15:49 +0000)]
Merge "Bug fix of the CI script in the ODL-SFC test case"

8 years agoBug fix of the CI script in the ODL-SFC test case
Manuel Buil [Thu, 28 Jul 2016 15:02:22 +0000 (17:02 +0200)]
Bug fix of the CI script in the ODL-SFC test case

Change-Id: Id3db14d1f26a7dd75cdaaf4d7ca0f54268a89faf
Signed-off-by: Manuel Buil <manuel.buil@ericsson.com>
8 years agoPrevent exiting the overall execution when there is a failed test case
jose.lausuch [Thu, 28 Jul 2016 14:44:22 +0000 (16:44 +0200)]
Prevent exiting the overall execution when there is a failed test case

Unless it is a test with the flag blocking=true

Change-Id: Iada54319db1c81c73781f1b135eb496bc9f2af1b
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoScripts added to enable the SFC-ODL test in CI
Manuel Buil [Wed, 27 Jul 2016 16:03:55 +0000 (18:03 +0200)]
Scripts added to enable the SFC-ODL test in CI

We need to install Tacker in the server before running the test cases

Change-Id: Id7daee80627bc71c335fcb3efa5d389594641b65
Signed-off-by: Manuel Buil <manuel.buil@ericsson.com>
8 years agoAdd configure options for healthcheck
Ciprian Barbu [Mon, 25 Jul 2016 11:43:00 +0000 (14:43 +0300)]
Add configure options for healthcheck

The healthcheck tier used hard-coded x86 image for testing glance and nova.
This breaks functest for ARM based pods, because x86 instances cannot run on
ARM and even more, if healthcheck fails no other testcases are executed.

Change-Id: I8a4cc754569a26d98bff57470574ed28eebbf2da
Signed-off-by: Ciprian Barbu <ciprian.barbu@enea.com>
8 years agoAdd region name to OpenStack credentials
Dimitri Mazmanov [Thu, 28 Jul 2016 09:27:13 +0000 (11:27 +0200)]
Add region name to OpenStack credentials

Currently functest doesn't account for regions when it tries to
instantieate OpenStack clients. As a result, the code fails when
functest is used in a multi-region environment.
The fix forces functest account for region name.

Signed-off-by: Dimitri Mazmanov <dimitri.mazmanov@ericsson.com>
Change-Id: Iebbd20042548c02da9d5ed7e58a454347a555778

8 years agoFunctional testing for tosca2heat translation
SerenaFeng [Thu, 28 Jul 2016 04:20:21 +0000 (12:20 +0800)]
Functional testing for tosca2heat translation

in Dockerfile, add parser related work
in config_functest.yaml/testcase.yaml, add parser related configuration
in exec_test.sh to run parser test
add vnf/RNC/parser.py script

JIRA: FUNCTEST-187

Change-Id: I8b53774da00f13b630ec5d9f750817db892a82bb
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agoMerge "Bugfix: dont stop execution when a test case is not blocking"
jose.lausuch [Thu, 28 Jul 2016 09:51:07 +0000 (09:51 +0000)]
Merge "Bugfix: dont stop execution when a test case is not blocking"

8 years agoBugfix: dont stop execution when a test case is not blocking
jose.lausuch [Thu, 28 Jul 2016 07:14:48 +0000 (09:14 +0200)]
Bugfix: dont stop execution when a test case is not blocking

Change-Id: I3564b6f2d8f8057ec2c9753837e3a5fe16a4a866
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoAdd endpoint type support on functest
boucherv [Wed, 27 Jul 2016 12:57:58 +0000 (14:57 +0200)]
Add endpoint type support on functest

By default public endpoint was used but now you can use internal or admin

JIRA: FUNCTEST-387

Change-Id: If1adaa88248d379ad6d74b20f2dffd76278b67b9
Signed-off-by: boucherv <valentin.boucher@orange.com>
8 years agologger the push result of rally test
SerenaFeng [Wed, 27 Jul 2016 09:44:33 +0000 (17:44 +0800)]
logger the push result of rally test

JIRA: FUNCTEST-393

Change-Id: I94158531739e1e05c694a785a8bbca7f80684af5
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agoMerge "[bugfix] issures in multisite testcase"
meimei [Thu, 28 Jul 2016 02:00:39 +0000 (02:00 +0000)]
Merge "[bugfix] issures in multisite testcase"

8 years agoDisable security scan tests
jose.lausuch [Wed, 27 Jul 2016 19:51:51 +0000 (21:51 +0200)]
Disable security scan tests

Change-Id: Ic6800f4b1401c44dbded160f45fdd17c706f14f8
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoMake openstack_clean be aware of the names and not only the IDs
jose.lausuch [Wed, 27 Jul 2016 12:43:04 +0000 (14:43 +0200)]
Make openstack_clean be aware of the names and not only the IDs

Sometimes, when using the same container for a different deployment,
the IDs of the openstack resources are different but the names are
the same. This will avoid cleaning for example the user 'admin'
regardless of the ID it has.

JIRA: FUNCTEST-383

Change-Id: If27603f8f6d0cb7ad0aeba6e3c89ca19e2b8aed2
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoFix the proper version output for kingbird.
Dimitri Mazmanov [Wed, 27 Jul 2016 10:40:04 +0000 (12:40 +0200)]
Fix the proper version output for kingbird.

Signed-off-by: Dimitri Mazmanov <dimitri.mazmanov@ericsson.com>
8 years agoReturn -1 in Tempest if the success criteria is not passed.
jose.lausuch [Wed, 27 Jul 2016 09:59:09 +0000 (11:59 +0200)]
Return -1 in Tempest if the success criteria is not passed.

JIRA: FUNCTEST-381

Change-Id: I533bb50e26a3b97da9fd3bfa007f8ac7ce752a5b
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoReturn -1 if copper test fails
jose.lausuch [Wed, 27 Jul 2016 08:11:59 +0000 (10:11 +0200)]
Return -1 if copper test fails

Change-Id: Ib66d1b69c73603bab33950ca2c0fca3f07eb3d22
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years ago[bugfix] issures in multisite testcase
meimei [Wed, 27 Jul 2016 09:09:04 +0000 (17:09 +0800)]
[bugfix] issures in multisite testcase

Change-Id: I351311688742c04c2c2b7278015e221178267fda
Signed-off-by: meimei <meimei@huawei.com>
8 years agoEnable odl-sfc in CI
Manuel Buil [Wed, 27 Jul 2016 09:15:07 +0000 (11:15 +0200)]
Enable odl-sfc in CI

JIRA:FUNCTEST-392

Change-Id: If1c931df26062501695061e1155eb26ad007c8de
Signed-off-by: Manuel Buil <manuel.buil@ericsson.com>
8 years agoBump rally to 0.5.0 and add shyaml
Ciprian Barbu [Tue, 26 Jul 2016 16:15:11 +0000 (19:15 +0300)]
Bump rally to 0.5.0 and add shyaml

A newer version of rally brings in this change that is required by armband:
https://bugs.launchpad.net/rally/+bug/1580695

The shyaml cli parser can be used to make the healthcheck testcase
configurable, for instance to pass a different cirros image other than the one
for x86_64. Also required by armband to make functest execute all tests.

Change-Id: Ic4976bb6d4a4753e56deade35b641f1127af3162
Signed-off-by: Ciprian Barbu <ciprian.barbu@enea.com>
8 years agoMerge "[bugfix] miss a argument"
meimei [Wed, 27 Jul 2016 07:25:55 +0000 (07:25 +0000)]
Merge "[bugfix] miss a argument"

8 years ago[bugfix] miss a argument
meimei [Wed, 27 Jul 2016 07:08:35 +0000 (15:08 +0800)]
[bugfix] miss a argument

Change-Id: Id8cdf6a5dadee6a2e0ba10772db88c84e5ce13a3
Signed-off-by: meimei <meimei@huawei.com>
8 years agosubstitue misused variable 'total_success' with 'success_rate'
SerenaFeng [Wed, 27 Jul 2016 01:42:20 +0000 (09:42 +0800)]
substitue misused variable 'total_success' with 'success_rate'

JIRA: FUNCTEST-390

Change-Id: I35a2f72016573fc9e0a8403e436d5568937c5b14
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agoGenerate Functest report at the end of the execution
jose.lausuch [Fri, 22 Jul 2016 15:49:34 +0000 (17:49 +0200)]
Generate Functest report at the end of the execution

JIRA: FUNCTEST-89

Change-Id: I10c41efdd6a7f737f57ab15158c8c8d49f392271
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agodelete useless stop_time initialization in run_rally-cert.py
SerenaFeng [Tue, 26 Jul 2016 01:17:50 +0000 (09:17 +0800)]
delete useless stop_time initialization in run_rally-cert.py

stop_time is assigned twice, delete the first one

JIRA:FUNCTEST-384

Change-Id: I380113474e66a73c516f4411d8d7c5832e33921a
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agoBug fix for ODL-SFC testcase
Manuel Buil [Tue, 26 Jul 2016 16:56:05 +0000 (18:56 +0200)]
Bug fix for ODL-SFC testcase

The vxlan_tool was sometimes not started properly in the SFs. Now we check
that the process is started

Change-Id: I8b88b76ba30755a1ed339f9e4c374d43e5e0b422
Signed-off-by: Manuel Buil <manuel.buil@ericsson.com>
8 years agoBad status on rally_sanity
SerenaFeng [Tue, 26 Jul 2016 01:54:37 +0000 (09:54 +0800)]
Bad status on rally_sanity

format total_success to float when compared with numeric 90

JIRA: FUNCTEST-365

Change-Id: I473582053c107bb026bf0e8d76dab46b2e5e3eb6
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agoadd .idea directory to gitignore
SerenaFeng [Tue, 26 Jul 2016 02:39:57 +0000 (10:39 +0800)]
add .idea directory to gitignore

ignore .idea created by using pycharm

JIRA: FUNCTEST-386

Change-Id: I425b4525ea16cafc2883f165efd2293a8adb13fb
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
8 years agoTurn off vPing_ssh for odl_l3 scenarios due to odl bug
jose.lausuch [Mon, 25 Jul 2016 15:05:27 +0000 (17:05 +0200)]
Turn off vPing_ssh for odl_l3 scenarios due to odl bug

https://bugs.opendaylight.org/show_bug.cgi?id=5586
JIRA: FUNCTEST-363

Change-Id: I4d195ae7ebbe10cfc4fe2947eecf5f0c051fd4e3
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoMerge "Create a flag in the tests to indicate if they should be blocking or not"
jose.lausuch [Mon, 25 Jul 2016 11:42:07 +0000 (11:42 +0000)]
Merge "Create a flag in the tests to indicate if they should be blocking or not"

8 years agoCreate a flag in the tests to indicate if they should be blocking or not
jose.lausuch [Mon, 25 Jul 2016 09:43:01 +0000 (11:43 +0200)]
Create a flag in the tests to indicate if they should be blocking or not

JIRA: FUNCTEST-380

Change-Id: I737d71a47ddac80bf0222a1f004af05f509c5971
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoBug fix for SFC test case
Manuel Buil [Mon, 25 Jul 2016 09:20:05 +0000 (11:20 +0200)]
Bug fix for SFC test case

The path is fixed in the sfc_tacker script, some minor bug fixes in sfc.py are
made and the "FAILED" messages are not handled by log.error instead of debug

Change-Id: I2facbd985c1c0c0cac3865504b1bf65940b7ce50
Signed-off-by: Manuel Buil <manuel.buil@ericsson.com>
8 years agoAdd duration message after test execution
jose.lausuch [Fri, 22 Jul 2016 21:40:28 +0000 (23:40 +0200)]
Add duration message after test execution

Change-Id: Ida68009ba02b2a5135dc4f3ae958fc77088389d7
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agobugfix: Don't try to run a tier if it doesn't contain any enabled test
jose.lausuch [Fri, 22 Jul 2016 09:51:26 +0000 (11:51 +0200)]
bugfix: Don't try to run a tier if it doesn't contain any enabled test

- Add separation (blank lines) between the test cases output

JIRA: FUNCTEST-349

Change-Id: I54a5d4f33d5776b4754641168c3775d19ca50e4a
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoFix execute_command function to show all the output
jose.lausuch [Fri, 22 Jul 2016 09:23:41 +0000 (11:23 +0200)]
Fix execute_command function to show all the output

This function wasn't showing anything for some commands,
like 'rally show images' and 'rally show flavors'

JIRA: FUNCTEST-348

Change-Id: I75bf5f3504ea8ffea42c2fb237d6863c655259e1
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoAdd cd to test directory
blsaws [Wed, 20 Jul 2016 04:15:23 +0000 (21:15 -0700)]
Add cd to test directory

JIRA: COPPER-4

Fix pendantic formatting issue

Change-Id: Iaab44a990fc04addd4c66dc8432bc9428bb5a52f
Signed-off-by: blsaws <bryan.sullivan@att.com>
8 years agoBugfix: ODL report to DB fails
jose.lausuch [Wed, 20 Jul 2016 17:34:51 +0000 (19:34 +0200)]
Bugfix: ODL report to DB fails

JIRA: FUNCTEST-370

Change-Id: I48c8927f3490b86fcf1c95656324429b70201653
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoMerge "Add kingbird endpoint information to functest."
Jose Lausuch [Wed, 20 Jul 2016 14:42:31 +0000 (14:42 +0000)]
Merge "Add kingbird endpoint information to functest."

8 years agoRemove logger as input parameter of functions in openstack_utils
jose.lausuch [Wed, 20 Jul 2016 11:37:41 +0000 (13:37 +0200)]
Remove logger as input parameter of functions in openstack_utils

JIRA: FUNCTEST-376

Also some fixes to get rid of flake8 violations in onos-sfc

Change-Id: I52c2fd30a6b81af20ea779db2d73b9386524f7ce
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoAdd kingbird endpoint information to functest.
Dimitri Mazmanov [Wed, 20 Jul 2016 11:42:30 +0000 (13:42 +0200)]
Add kingbird endpoint information to functest.

The patch multihops to the fuel node and dynamically retrieves kingbird
endpoint information. It also resolves the controller ip instead of
using the hardcoded value from the config ONLY if the installer type is
Fuel.

Signed-off-by: Dimitri Mazmanov <dimitri.mazmanov@ericsson.com>
Change-Id: I5161df5df3da7283f6374efa6bd1108246c1095a

8 years agoAdd logger to openstack_utils to output error messages
jose.lausuch [Tue, 19 Jul 2016 17:30:02 +0000 (19:30 +0200)]
Add logger to openstack_utils to output error messages

This will help to see the exception error when something
fails

JIRA: FUNCTEST-376

Change-Id: Ibaa6954fc63af00d552ac32b3cf903b9218200d1
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoMerge "Show real time tempest test execution"
jose.lausuch [Tue, 19 Jul 2016 15:38:38 +0000 (15:38 +0000)]
Merge "Show real time tempest test execution"

8 years agoShow real time tempest test execution
jose.lausuch [Tue, 19 Jul 2016 13:18:28 +0000 (15:18 +0200)]
Show real time tempest test execution

JIRA: FUNCTEST-375

Change-Id: I4f0cb68639b927f2980389d9331ec260605c8caf
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoCatch timeout exception in vIMS
boucherv [Tue, 19 Jul 2016 14:05:11 +0000 (16:05 +0200)]
Catch timeout exception in vIMS

JIRA: FUNCTEST-368

Change-Id: Ifeec4b89e9a05ee02d5a0829ee787dbd6ef528b2
Signed-off-by: boucherv <valentin.boucher@orange.com>
8 years agoExit with -1 return code if something fails in vping tests
jose.lausuch [Tue, 19 Jul 2016 12:18:08 +0000 (14:18 +0200)]
Exit with -1 return code if something fails in vping tests

Also, turn off debug info from paramiko python module in vping_ssh

JIRA: FUNCTEST-373
JIRA: FUNCTEST-374

Change-Id: I6168cae85db91b2dd6ceac591f12dd9295786b59
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoExtend timeout to check vm in onos-sfc testcase
wuwenbin2 [Tue, 19 Jul 2016 11:50:03 +0000 (19:50 +0800)]
Extend timeout to check vm in onos-sfc testcase

Change-Id: I68ae154391d8098cab3d3cb05cebe6f1922ca701
Signed-off-by: wuwenbin2 <wuwenbin2@huawei.com>
8 years agoDon't take openstack snapshot when "-n" option is set
jose.lausuch [Fri, 15 Jul 2016 11:40:27 +0000 (13:40 +0200)]
Don't take openstack snapshot when "-n" option is set

JIRA: FUNCTEST-236

Change-Id: I37bfa4ebe88a8cf02f2cddc8fc4cce70ad87b775
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years ago[multisite] run on the env of fuel and compass
meimei [Fri, 15 Jul 2016 03:02:50 +0000 (11:02 +0800)]
[multisite] run on the env of fuel and compass

Change-Id: Ie590a99e8b571af66cc2164c291dd8e9eed47e4a
Signed-off-by: meimei <meimei@huawei.com>
8 years agoUse blacklist of Tempest test cases for SDNVPN scenario
jose.lausuch [Fri, 15 Jul 2016 12:14:24 +0000 (14:14 +0200)]
Use blacklist of Tempest test cases for SDNVPN scenario

This patch disables tempest smoke from being executed in bgpvpn scenarios.
Since there is a known upstream issue which won't be resolved before Colorado,
it doesn't make sense to run the test cases in Tempest that will fail due
to that issue.
The Tempest suite will be triggered along with the "features" tier, with
a given blacklist of the known test that will fail.

JIRA: FUNCTEST-369

Change-Id: Ifc533077207e9d62d49bcedcc3cd0936df312891
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoFixed typo in config.ini for reports.html
Luke Hinds [Fri, 8 Jul 2016 10:15:11 +0000 (11:15 +0100)]
Fixed typo in config.ini for reports.html

JIRA: FUNCTEST-359

Change-Id: Ide565cea987592446a44b168bd4cff0285864192
Signed-off-by: Luke Hinds <lukehinds@gmail.com>
8 years agoAdd functest logger in ODL script
jose.lausuch [Fri, 15 Jul 2016 11:28:12 +0000 (13:28 +0200)]
Add functest logger in ODL script

JIRA: FUNCTEST-367

Change-Id: I3a752ca78d274ca8c16f6179534d36f12facc202
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoPush sfc result into DB
wuwenbin2 [Fri, 15 Jul 2016 10:53:39 +0000 (18:53 +0800)]
Push sfc result into DB

Change-Id: I8ab9b5b9322fa3a6905c629e559aa78d9817958f
Signed-off-by: wuwenbin2 <wuwenbin2@huawei.com>
8 years agoRevert "Disable ODL test."
Jose Lausuch [Fri, 15 Jul 2016 10:12:34 +0000 (10:12 +0000)]
Revert "Disable ODL test."

This reverts commit adf9811e8c0f2fa92ca1c45658a3360e1de8be7e.

JIRA: FUNCTEST-371

Change-Id: I677ed08e017e2b147eec9433b050d3fa480565a2
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoDisable ODL test.
jose.lausuch [Fri, 15 Jul 2016 09:32:12 +0000 (11:32 +0200)]
Disable ODL test.

It triggers a lot of errors. See log in the JIRA ticket:
JIRA: FUNCTEST-371

Change-Id: Ia039d36773158850f7746ae6d3a2a65fb350e382
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoDisable ODL reporting to DB
jose.lausuch [Fri, 15 Jul 2016 09:23:15 +0000 (11:23 +0200)]
Disable ODL reporting to DB

The script odlreport2db.py fails and it needs more investigation
and troubleshooting. For now, we disable it to not block CI

JIRA:FUNCTEST-370

Change-Id: I9952fd7c5238212d6ed1437f1595fdadde9208f2
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years ago[bug] miss a ; in case statement
meimei [Fri, 15 Jul 2016 03:34:00 +0000 (11:34 +0800)]
[bug] miss a ; in case statement

Change-Id: I2e3d99203450d2abc93e5f472c6be701a385a3c6
Signed-off-by: meimei <meimei@huawei.com>
8 years agoEnable domino multi-node test case in Functest
Ulas Kozat [Tue, 12 Jul 2016 22:59:05 +0000 (15:59 -0700)]
Enable domino multi-node test case in Functest

This includes:
 - Add domino.py script
 - Add call to domino.py
 - Add the Domino repo in the Dockerfile
 - Add the test case definition in testcases.yaml

JIRA: FUNCTEST-246

Change-Id: I5a8c3345a14c43fa80e6e86678ae5fb1417b3147
Signed-off-by: Ulas Kozat <ulas.kozat@gmail.com>
8 years agoEnable new SDNVPN test cases in Functest
jose.lausuch [Tue, 12 Jul 2016 15:10:55 +0000 (17:10 +0200)]
Enable new SDNVPN test cases in Functest

JIRA:FUNCTEST-99

The tempest execution has been moved to the sdnvpn
to leverage the logic of execution and collecting
the results from the executor wrapper:
sdnvpn/test/functest/run_tests.py

Change-Id: Ic4763f4ab45dde81f91b241ae91073b9ea47e333
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoUse common way to get the openstack clients
jose.lausuch [Wed, 13 Jul 2016 10:07:21 +0000 (12:07 +0200)]
Use common way to get the openstack clients

Instead of repeating the code in all the scripts and
importing the openstack libraries, the clients shall
be given by openstack utils, which is its purpose

JIRA: FUNCTEST-163

Change-Id: I1ccc05a3af44ee1ab5938ea9e4e01dbe55f4816d
Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
8 years agoSwitch to $SDN_CONTROLLER_IP instead of $neutron_ip
Cédric Ollivier [Thu, 14 Jul 2016 09:51:33 +0000 (11:51 +0200)]
Switch to $SDN_CONTROLLER_IP instead of $neutron_ip

ODL IP is now set to $SDN_CONTROLLER_IP in case of Apex and default.
It follows the next patch [1].

[1] https://gerrit.opnfv.org/gerrit/#/c/16779/

Change-Id: I60238705e45ed31a67da1518ec586e3f2970c13f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
8 years ago[multisite] add kingbird repo and install python lib in docker
meimei [Wed, 13 Jul 2016 09:10:44 +0000 (17:10 +0800)]
[multisite] add kingbird repo and install python lib in docker

JIRA: FUNCTEST-179

Change-Id: Ib88bbbc2d541b2825d768516237a95f68e0418d2
Signed-off-by: meimei <meimei@huawei.com>
8 years ago[multisite] integrate kingbird tempest testcases
meimei [Thu, 14 Jul 2016 04:04:37 +0000 (12:04 +0800)]
[multisite] integrate kingbird tempest testcases

JIRA: FUNCTEST-179

Change-Id: I4a8ae43e2814b3d859c32a6278671f87155aa855
Signed-off-by: meimei <meimei@huawei.com>
8 years agoImprove timeout management with python
boucherv [Wed, 13 Jul 2016 14:48:28 +0000 (16:48 +0200)]
Improve timeout management with python

JIRA: FUNCTEST-368

Change-Id: I9f7a6adc807c63694fc5d9fb8a81ad1d2d9d3225
Signed-off-by: boucherv <valentin.boucher@orange.com>
8 years agoBump to cloudify-manager 3.3.1
boucherv [Wed, 13 Jul 2016 13:54:25 +0000 (15:54 +0200)]
Bump to cloudify-manager 3.3.1

Change-Id: Ica223edb7e2b29ce11634774d8355634b00cada4
Signed-off-by: boucherv <valentin.boucher@orange.com>