dovetail.git
8 years agoMerge "bugfix of wrong testcase name"
hongbo tian [Mon, 15 May 2017 04:26:34 +0000 (04:26 +0000)]
Merge "bugfix of wrong testcase name"

8 years agobugfix of wrong testcase name 25/34625/1
MatthewLi [Thu, 11 May 2017 12:31:21 +0000 (08:31 -0400)]
bugfix of wrong testcase name

JIRA: DOVETAIL-430

this is wrong before(ipv6.tc023 same to ipv6.tc019),
fixed according to https://gerrit.opnfv.org/gerrit/#/c/34577/

Change-Id: I3d996a7c2adffc305b0f8fa6bcd87e4e6e4c5b7e
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoAdd three HA test cases into test suite proposed_tests 85/34585/2
xudan [Thu, 11 May 2017 06:33:29 +0000 (02:33 -0400)]
Add three HA test cases into test suite proposed_tests

JIRA: DOVETAIL-427

There are another three HA test cases which are stable now for adding into
test suite proposed_tests for CI daily run.

Change-Id: I86ed38d2e3a2bcfda29af50164aadd0da89e6aaa
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "store/load offline docker images"
hongbo tian [Thu, 11 May 2017 02:03:33 +0000 (02:03 +0000)]
Merge "store/load offline docker images"

8 years agoMerge "fix ha testcase PASS/FAIL judgement"
hongbo tian [Tue, 9 May 2017 01:24:02 +0000 (01:24 +0000)]
Merge "fix ha testcase PASS/FAIL judgement"

8 years agoMerge "vpn testcases added in proposed_tests"
hongbo tian [Tue, 9 May 2017 01:23:42 +0000 (01:23 +0000)]
Merge "vpn testcases added in proposed_tests"

8 years agostore/load offline docker images 27/34427/2
MatthewLi [Mon, 8 May 2017 09:28:15 +0000 (05:28 -0400)]
store/load offline docker images

JIRA: DOVETAIL-423

usage:
1.cd ${DOVETAIL_HOME}/dovetail/utils/offline
2.edit config.yaml as needed
3.on onsite host, <python download.py>  --> save docker images to defined path
4.on offline host, <python load.py> ---> load docker images to offline env
5.this can be easily extended to other images beside docker images

Change-Id: I97d843e154ecf8d66cafb9ea9594fe73343ee591
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agofix ha testcase PASS/FAIL judgement 37/34337/4
MatthewLi [Fri, 5 May 2017 07:46:12 +0000 (03:46 -0400)]
fix ha testcase PASS/FAIL judgement

JIRA: DOVETAIL-425

Change-Id: Ifee2473956e0a1a3ad9c98beff92d94327251614
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agovpn testcases added in proposed_tests 93/34193/3
MatthewLi [Thu, 4 May 2017 03:12:53 +0000 (23:12 -0400)]
vpn testcases added in proposed_tests

JIRA: DOVETAIL-212

after communicating with Tim/jose, help to add vpn testcase into proposed_tests

reference: https://wiki.opnfv.org/display/sdnvpn/SDNVPN+Testing#SDNVPNTesting-TestcompliancetemplateforDovetail

Change-Id: Id34a2cadba625abbdbc59ef11affeccba24fa10c
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoProposing to move "post conditions" as an optional topic for a test case. 05/34405/1
ChristopherPrice [Sun, 7 May 2017 11:44:39 +0000 (13:44 +0200)]
Proposing to move "post conditions" as an optional topic for a test case.

It is not clear that there are post conditions to be documented for all test cases.
In the case a test cse creates an image and deletes it, there is littöe need for
a post condition to be documented.  If a test case establishes a network to be
used for a subsequent test ina  sequence this information should be clearly dcumented.

Change-Id: Ifb4bb45f240921c180077ab7273988d973417f6f
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
8 years agoMerge "Get results json data from database"
hongbo tian [Fri, 5 May 2017 08:56:30 +0000 (08:56 +0000)]
Merge "Get results json data from database"

8 years agoMerge "Uploading the initial draft of the first VPN test case "
hongbo tian [Fri, 5 May 2017 01:20:31 +0000 (01:20 +0000)]
Merge "Uploading the initial draft of the first VPN test case "

8 years agoMerge "Drafting the first "refstack" version of the test scope document."
hongbo tian [Thu, 4 May 2017 03:41:29 +0000 (03:41 +0000)]
Merge "Drafting the first "refstack" version of the test scope document."

8 years agoGet results json data from database 99/34099/3
xudan [Wed, 3 May 2017 06:11:51 +0000 (06:11 +0000)]
Get results json data from database

JIRA: DOVETAIL-417

1. If report results with files, all results will be stored in local files.
2. If report to database, will get results from database and stored with file results.json

Change-Id: Ic30037bd66cd37042f82b75fac2043a876e21c6f
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "local db support offline mode"
hongbo tian [Wed, 3 May 2017 17:27:38 +0000 (17:27 +0000)]
Merge "local db support offline mode"

8 years agolocal db support offline mode 85/34085/6
Leo Wang [Tue, 2 May 2017 23:50:02 +0000 (16:50 -0700)]
local db support offline mode

JIRA: DOVETAIL-415

The local db also need to support offline, move all the git clone/apt/yum/pip/wget/curl into the Dockerfile

Change-Id: I2392dc9f4a6bd6f6a5a3f4849625a576c51a44f8
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agoDrafting the first "refstack" version of the test scope document. 85/33885/5
Georg Kunz [Wed, 26 Apr 2017 15:31:37 +0000 (17:31 +0200)]
Drafting the first "refstack" version of the test scope document.

To be aligned with the committers and agreed as the "foundation" list of
test scope for moving forward with DoveTail test selection.
Requires a majority of committer approvals prior to merging.

Change-Id: I5fb091d2190a183d24ba9e766d19aba9e6a7c094
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
8 years agoMerge "xipv6 testcase adjustment"
hongbo tian [Wed, 3 May 2017 09:08:36 +0000 (09:08 +0000)]
Merge "xipv6 testcase adjustment"

8 years agoxipv6 testcase adjustment 35/34035/3
MatthewLi [Tue, 2 May 2017 06:34:02 +0000 (02:34 -0400)]
xipv6 testcase adjustment

JIRA: DOVETAIL-418

1) ipv6 tc002-tc005 error fix, delete wrong testcases
2) ipv6 tc018-tc025 os-nosdn-nofeature scenario dependent, make
   it known to the users

Change-Id: I8e63131b795403ee63566ae59bf46782524acd5a
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoUploading the initial draft of the first VPN test case 45/33945/4
ChristopherPrice [Thu, 27 Apr 2017 11:11:59 +0000 (13:11 +0200)]
Uploading the initial draft of the first VPN test case

This should be considered in the danube CVP phase as an
optional test area for OPNFV compliance

Change-Id: I99494cf2246e1cf079b87414ef1bb34d9b26e6b3
Co-Authored-By: Tim Irnich <tim.irnich@ericsson.com>
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
8 years agoTips on update docker to right version, > 1.8 27/34027/1
grkiss [Mon, 1 May 2017 22:50:20 +0000 (15:50 -0700)]
Tips on update docker to right version, > 1.8

JIRA: DOVETAIL-414

Dovetail requires docker at version > 1.8, so add some validation in code, and also versions of client and server should match
with each other, or there will be unpredictable errors.

Gives a notice to user to decide whether to update the docker to the latest version(1.17)

Change-Id: I1331f5c19943a73e8fe25f76bd21628233e556f3
Signed-off-by: grakiss <grakiss.wanglei@huawei.com>
8 years agosupport HA test cases to run with key 63/33963/2
xudan [Thu, 27 Apr 2017 14:16:12 +0000 (14:16 +0000)]
support HA test cases to run with key

JIRA: DOVETAIL-409

1. HA test cases need a file pod.yaml, in this file it can use the password or the key.
2. If use key, Dovetail need to pass the key file $DOVETAIL_HOME/dovetail/userconfig/id_rsa
   to yardstick Container.
3. Add a directory $DOVETAIL_HOME/dovetail/userconfig and a sample_pod.yaml
4. ha.tc001 tc003 tc005 tc006 tc009 are ready.

Change-Id: I391cc211dba2cf0740dd0dcaee8a5c2d712eaee1
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoSetting the document directory structure according to the Plugfest outcome. 65/33865/5
ChristopherPrice [Wed, 26 Apr 2017 13:25:55 +0000 (15:25 +0200)]
Setting the document directory structure according to the Plugfest outcome.

Implemented a directory and file structure that supports the requirements defined
in the plugfest etherpad: https://etherpad.opnfv.org/p/cvp_tooling_sketch

Change-Id: Icb7d2db4b0132f2039c680883e173ff5d7fc0120
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
8 years agoAdding a stub document for a CVP reviewer guide 51/33851/3
Georg Kunz [Wed, 26 Apr 2017 11:50:29 +0000 (13:50 +0200)]
Adding a stub document for a CVP reviewer guide

This document shall describe guidelines and evaluation criteria for
reviewers on how to handle the review process of results.

DOVETAIL-408

Change-Id: I78e669544bc172b74f94443176a48082a4c28a2f
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
8 years agoMerge "skip ha tc002 and tc012"
Dave Neary [Wed, 26 Apr 2017 15:49:28 +0000 (15:49 +0000)]
Merge "skip ha tc002 and tc012"

8 years agoCommitting the test case requirements specification to GIT for committer review. 71/33771/8
ChristopherPrice [Tue, 25 Apr 2017 09:09:26 +0000 (11:09 +0200)]
Committing the test case requirements specification to GIT for committer review.

This doucment needs to be +2'd by the majority of committers prior to merging
to ensure we are all in agreement with regard to the test case requirements.
This is a pre-requisite to test case selection for the CVP.

Updated by dneary 2017-04-25: List formatting, and reorganising requirements for installer and stack composition.

Change-Id: Iabaf11bc9479a20a58f985233d460013e4690d7c
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
8 years agoMerge "JIRA: DOVETAIL-352 Updating the test specification document with iterative...
Christopher Price [Wed, 26 Apr 2017 09:59:05 +0000 (09:59 +0000)]
Merge "JIRA: DOVETAIL-352 Updating the test specification document with iterative inprovements."

8 years agoskip ha tc002 and tc012 39/33839/1
MatthewLi [Wed, 26 Apr 2017 09:56:47 +0000 (05:56 -0400)]
skip ha tc002 and tc012

since these two testcases will kill the host and can't restart
not ready yet, skip.

Change-Id: Icbc7e4f080d32d1ecb59df9fd95cb244d74e4cde
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "Bugfix: All HA testcases are SSH timeout in CI jobs"
hongbo tian [Wed, 26 Apr 2017 08:23:28 +0000 (08:23 +0000)]
Merge "Bugfix: All HA testcases are SSH timeout in CI jobs"

8 years agoJIRA: DOVETAIL-352 Updating the test specification document with iterative inprovements. 99/32499/6
ChristopherPrice [Thu, 30 Mar 2017 13:11:28 +0000 (15:11 +0200)]
JIRA: DOVETAIL-352 Updating the test specification document with iterative inprovements.

JIRA tasks are being created for specific sections of the document.
Refer to the doc for links to relevant JIRA tasks.

Change-Id: I1717d56b8817c38802a227db320f30029f68fbd0
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
8 years agoMerge "AArch64 support for Dovetail docker"
hongbo tian [Tue, 25 Apr 2017 22:49:45 +0000 (22:49 +0000)]
Merge "AArch64 support for Dovetail docker"

8 years agoBugfix: All HA testcases are SSH timeout in CI jobs 79/33779/1
xudan [Tue, 25 Apr 2017 08:58:15 +0000 (08:58 +0000)]
Bugfix: All HA testcases are SSH timeout in CI jobs

JIRA: DOVETAIL-405

1. All HA test cases running in CI are SSH timeout.
2. The reason is that HA test cases need a file pod.yaml providing the info of
   each node of Openstack.
3. This pod.yaml should be put in $DOVETAIL_HOME/dovetail/userconfig by users.
4. Dovetail's CI jobs should also do some adaptions. JIRA: DOVETAIL-407
5. modify file .gitignore to ignore directories results/ and userconfig/.

Change-Id: I5c681445ec83c9d67a0151370f5b56840ffe444c
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoBugfix: KeyError when tempest_custom runs wrong 37/33737/1
xudan [Tue, 25 Apr 2017 03:41:41 +0000 (03:41 +0000)]
Bugfix: KeyError when tempest_custom runs wrong

JIRA: DOVETAIL-406

1. When there are something wrong with the test case tempest_custom,
   it's details json data are "details": {}, So it will throw KeyError
   exception in this situation.
2. Catch this exception and set all sub_testcases FAIL.
3. Disable the exceed_max_retry_times().

Change-Id: I6bdfc5944a535625b2211a3bd8096a13b81b339e
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoAArch64 support for Dovetail docker 89/33689/4
Alexandru Nemes [Mon, 24 Apr 2017 09:00:18 +0000 (12:00 +0300)]
AArch64 support for Dovetail docker

Add aarch64 patch for Dovetail docker file;
The patch should be applied from releng

Move to Ubuntu 16.04 in order, since docker.io package was not
available for arm on Ubuntu 14.04.
Added "pip install -U setuptools", since installation of other
components failed because setuptools was missing.

Change-Id: If00346d89c48be86c6d9b830f75c1792c39d6bb3
Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com>
8 years agoMerge "skip ha and rename tempest.conf"
hongbo tian [Mon, 24 Apr 2017 10:01:08 +0000 (10:01 +0000)]
Merge "skip ha and rename tempest.conf"

8 years agoskip ha and rename tempest.conf 63/33663/1
MatthewLi [Sat, 22 Apr 2017 09:20:39 +0000 (05:20 -0400)]
skip ha and rename tempest.conf

1) ha testcases need node configure file, skip it temporarily
2) tempest.conf-->refstack_tempest.conf, then it will not covered
   by other tempest testcases, can used for users to debug

Change-Id: Ieb7101d8c406e294755629fd11a686f90b6c0a7f
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoBugfix about the ipv6 results checker 51/33651/2
xudan [Sat, 22 Apr 2017 04:00:51 +0000 (04:00 +0000)]
Bugfix about the ipv6 results checker

JIRA: DOVETAIL-404

1. IPV6 sub-testcase's name includes [id], which will affects the regex.
2. Remove the [id] from the name of the sub_testcases.

Change-Id: I51b567e8382fa4e701f8b32ba7ea195d49fd31eb
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "add defcore/ha testcases to proposed_tests"
hongbo tian [Fri, 21 Apr 2017 12:01:13 +0000 (12:01 +0000)]
Merge "add defcore/ha testcases to proposed_tests"

8 years agouse Functest tempest_custom test cases to run all Ipv6 test cases 19/33619/3
xudan [Fri, 21 Apr 2017 06:31:33 +0000 (06:31 +0000)]
use Functest tempest_custom test cases to run all Ipv6 test cases

JIRA: DOVETAIL-401

1. modify all Ipv6 test cases, change tempest_smoke_serial/tempest_full_parallel
   into tempest_custom, add ID of each sub-testcases.
2. Some sub-testcases in ipv6.tc002, ipv6.tc003, ipv6.tc004 and ipv6.tc005 can not
   be found in the full tempest test case list.
2. disable testcases' script_result_acquired to enable tempest_custom run multiple times.
3. use build_tag-testcase_name(such as daily-master-79a6e392-28bb-4c40-9d88-a03bed99da2d-dovetail.ipv6.tc025)
   as the unique ID of each testcase.
4. comment out the progress bar, because it has some problems when print the report text.
5. Functest has changed the TEST_DB_URL from http://<testapi>:<port>/apv/v1 to
   http://<testapi>:<port>/apv/v1/results.
6. Functest has changed the TEST_DB_URL to file://<file_path>/<file_name>.
   see https://gerrit.opnfv.org/gerrit/#/c/33511/

Change-Id: I524f6702377f215f526158214842481eb44565be
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoadd defcore/ha testcases to proposed_tests 23/33523/9
MatthewLi [Wed, 19 Apr 2017 07:27:50 +0000 (03:27 -0400)]
add defcore/ha testcases to proposed_tests

JIRA: DOVETAIL-398
JIRA: DOVETAIL-399

1. this testsuite will run in community CI, see
https://gerrit.opnfv.org/gerrit/#/c/33509/
2. HA testcases please see
https://wiki.opnfv.org/pages/viewpage.action?pageId=8684344
3. vimops testcases deleted, since it's covered in defcore

Change-Id: Ied9c8867e0aa99e33fd213030e6471aa5cfb86e8
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agofix tox error 13/33613/3
MatthewLi [Fri, 21 Apr 2017 04:19:38 +0000 (00:19 -0400)]
fix tox error

see logs

https://build.opnfv.org/ci/job/dovetail-verify-master/560/console

Change-Id: Ifa66087c4c8fcbaae809aae380587a4f03eded6c
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "put dovetail test result back in log and not generte report file"
hongbo tian [Fri, 21 Apr 2017 03:43:55 +0000 (03:43 +0000)]
Merge "put dovetail test result back in log and not generte report file"

8 years agoset dovetail.log level by command line 19/33519/1
xudan [Wed, 19 Apr 2017 07:02:16 +0000 (07:02 +0000)]
set dovetail.log level by command line

JIRA: DOVETAIL-400

1. set both the dovetail.log and screen log level by command --debug.
2. if --report http://testapi_url:port/api/v1/, change it into
      --report http://testapi_url:port/api/v1

Change-Id: I7cf697d4ac9072fc2d302a67ccde0f452d84be8f
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoput dovetail test result back in log and not generte report file 69/33469/2
MatthewLi [Tue, 18 Apr 2017 10:51:25 +0000 (06:51 -0400)]
put dovetail test result back in log and not generte report file

Change-Id: I2da5e7844f5ce034d4c0cfaa4592e6cd47002049
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "fix click and pytz package version"
hongbo tian [Tue, 18 Apr 2017 01:48:27 +0000 (01:48 +0000)]
Merge "fix click and pytz package version"

8 years agofix click and pytz package version 05/33405/1
MatthewLi [Mon, 17 Apr 2017 03:47:50 +0000 (23:47 -0400)]
fix click and pytz package version

JIRA: DOVETAIL-376

Change-Id: Ia269ca7cdd34c71b4531b28e42e40415190ae398
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agodirectory update 71/33371/2
hongbotian [Fri, 14 Apr 2017 06:57:23 +0000 (14:57 +0800)]
directory update

JIRA: DOVETAIL-395

1. since plugfest has its own plan, the files are removed

Change-Id: I4989d1e0e1af483205733b3130e3f1152a8933c7
Signed-off-by: hongbotian <hongbo.tianhongbo@huawei.com>
8 years agoMerge "dovetail tool: bugfix: test cases are erroneously judged"
hongbo tian [Thu, 13 Apr 2017 06:32:29 +0000 (06:32 +0000)]
Merge "dovetail tool: bugfix: test cases are erroneously judged"

8 years agodovetail tool: bugfix: test cases are erroneously judged 33/33033/8
xudan [Thu, 6 Apr 2017 03:31:30 +0000 (03:31 +0000)]
dovetail tool: bugfix: test cases are erroneously judged

JIRA: DOVETAIL-375
JIRA: DOVETAIL-390

1. defcore test cases are erroneously judged because of:
   a) defcore results json data lack some keys
   b) some sub-testcases' names are the same as the prefix of another sub-testcases.
2. this patch removes the extra keys and judge sub-testcases accurately.
3. remove "store_type" in functest_config.yml and yardstick_config.yml.
4. change some log levels from info to debug.

Change-Id: I5f126e70309409cac66ef9a0909f1573d325098b
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "refstack: add manually method to let user can set its tempest.conf and defcore...
hongbo tian [Wed, 12 Apr 2017 08:07:06 +0000 (08:07 +0000)]
Merge "refstack: add manually method to let user can set its tempest.conf and defcore.txt"

8 years agorefstack: add manually method to let user can set its tempest.conf and defcore.txt 29/33229/2
MatthewLi [Tue, 11 Apr 2017 03:15:39 +0000 (23:15 -0400)]
refstack: add manually method to let user can set its tempest.conf and defcore.txt

JIRA: DOVETAIL-394

for refstack, userguide is listed in https://gerrit.opnfv.org/gerrit/#/c/33049/
in CI, use the automate way to generate tempest.conf to let refstack run against SUT, i.e., defcore.tc001.yml
will provide a way to let user can let refstack run against SUT manually, i.e. defcore.tc002.yml
since the tempest.conf autometely generated may not satisfied with SUT, users can adjust its tempest.conf to let the defcore testcases pass.

Change-Id: I5ed212c9c6d864d352d1c497ea039cc553461158
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agodovetail tool: show progress during executing test cases 79/33179/1
xudan [Mon, 10 Apr 2017 08:15:03 +0000 (08:15 +0000)]
dovetail tool: show progress during executing test cases

JIRA: DOVETAIL-159

1. It takes long time to execute test cases that most of users think it hangs there,
   so a progress is needed to show the test is running.
2. add dynamic progress during execution
3. it' s a fake progress only showing the test is alive,
   not to show the percentage of completion.

Change-Id: I0294659f79e8e09d1bcdd36d8f391d933e35b635
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "fix yardstick openrc location"
hongbo tian [Wed, 5 Apr 2017 06:23:17 +0000 (06:23 +0000)]
Merge "fix yardstick openrc location"

8 years agoMerge "bugfix of pbr after updating"
hongbo tian [Wed, 5 Apr 2017 06:21:54 +0000 (06:21 +0000)]
Merge "bugfix of pbr after updating"

8 years agofix yardstick openrc location 65/32865/1
MatthewLi [Sat, 1 Apr 2017 09:37:45 +0000 (05:37 -0400)]
fix yardstick openrc location

since yardstick changes its openrc location
see
https://gerrit.opnfv.org/gerrit/#/c/32861/

Change-Id: Ib85910fe5521fb398c47e389e2baf127342b1c0b
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "refstack result parse"
hongbo tian [Sat, 1 Apr 2017 04:15:41 +0000 (04:15 +0000)]
Merge "refstack result parse"

8 years agorefstack result parse 45/31345/10
MatthewLi [Wed, 22 Mar 2017 09:25:54 +0000 (05:25 -0400)]
refstack result parse

JIRA: DOVETAIL-375

1, together with other testsuite from functest, such as tempest etc.
this should merge after https://gerrit.opnfv.org/gerrit/#/c/31313/
2, delete some unnecessary definition
3, bugfix of docker cp inside container, see
   https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-debug-master/361/console
4, disable tempest testcaes since changed to refstack_defcore

Change-Id: I67503e348af4a3f697f1b4106bfe81c05add8aa0
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "dovetail tool: check push_to_db success or fail"
hongbo tian [Sat, 1 Apr 2017 01:22:09 +0000 (01:22 +0000)]
Merge "dovetail tool: check push_to_db success or fail"

8 years agodovetail tool: change results dir into $DOVETAIL_HOME/results 43/32743/1
xudan [Fri, 31 Mar 2017 08:46:56 +0000 (08:46 +0000)]
dovetail tool: change results dir into $DOVETAIL_HOME/results

JIRA: DOVETAIL-381

Now DoveTail's results dir is defined in $DOVETAIL_HOME/dovetail/conf/dovetail_config.yml,
and its default value is /home/opnfv/dovetail/results.

Since customer's jumphost may not have this dir and they don't want to create it,
Dovetail needs to change this dir into $DOVETAIL_HOME/results.

Change-Id: I616fb2da87bcfaabfd78d8762897e378168f694d
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: check push_to_db success or fail 63/32463/4
xudan [Thu, 30 Mar 2017 10:23:01 +0000 (10:23 +0000)]
dovetail tool: check push_to_db success or fail

JIRA: DOVETAIL-380

If Functest fails to push results to db, there is just an error log in functest.log
Dovetail needs to know the failure.
Dovetail will get the results from db with keys, build_tag and testcase.
If it can get the results data, it means push_to_db success.

Fix the duration time wrong.

Change-Id: If03bdf0ceb2f1a9d422381cdf06e1fd207358bd5
Signed-off-by: xudan <xudan16@huawei.com>
8 years agobugfix of pbr after updating 53/32453/1
MatthewLi [Thu, 30 Mar 2017 10:52:41 +0000 (06:52 -0400)]
bugfix of pbr after updating

JIRA: DOVETAIL-388

Change-Id: I3611624c955fa735aabfd00b456bd7f8455080e5
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "dovetail tool: change function pull_image"
hongbo tian [Thu, 30 Mar 2017 07:45:13 +0000 (07:45 +0000)]
Merge "dovetail tool: change function pull_image"

8 years agoMerge "dovetail tool: set a unique build_tag for each DoveTail test"
hongbo tian [Thu, 30 Mar 2017 07:42:23 +0000 (07:42 +0000)]
Merge "dovetail tool: set a unique build_tag for each DoveTail test"

8 years agoMerge "dovetail tool: support centos 7"
hongbo tian [Thu, 30 Mar 2017 07:41:14 +0000 (07:41 +0000)]
Merge "dovetail tool: support centos 7"

8 years agodovetail tool: refstack testcase integration 13/31313/8
MatthewLi [Wed, 22 Mar 2017 06:39:25 +0000 (02:39 -0400)]
dovetail tool: refstack testcase integration

JIRA: DOVETAIL-370

Change-Id: I60c9f431358f848e24abcb865c4b2ca8c3b2d843
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agodovetail tool: support centos 7 85/32385/1
xudan [Thu, 30 Mar 2017 06:39:18 +0000 (06:39 +0000)]
dovetail tool: support centos 7

JIRA: DOVETAIL-383

The script launch_db.sh can just support ubuntu now.
Make it support centos 7.

Change-Id: Ic6123ac82cb81bacb2d0c2d394619b8d8d7b3296
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: set a unique build_tag for each DoveTail test 95/31995/3
xudan [Tue, 28 Mar 2017 06:50:05 +0000 (06:50 +0000)]
dovetail tool: set a unique build_tag for each DoveTail test

JIRA: DOVETAIL-379

1. create a uuid for each test, and set build_tag=daily-master-<uuid>
2. the reason it starts with "daily_master" is that it's the requirement of
   functest when push data to DB.

Change-Id: I9a59d5695a9f0d83bf4330379f1949a2a04f876c
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: change function pull_image 35/31835/4
xudan [Mon, 27 Mar 2017 03:32:50 +0000 (03:32 +0000)]
dovetail tool: change function pull_image

JIRA: DOVETAIL-378

1. test_runner checks the result of cmd "pull image". If fail to pull,
   then break the running.
2. pull_image will return None when there are errors, otherwise return image name.
3. if there is an old image named the same as the new one, the old one's tag
   will be changed into <none>. In this case, the old image needs to be removed.
4. if there are some containers using this old image, the remove will be skipped
   rather than logging error.

Change-Id: I940cb18911112d5e13f3c14f87ddbdbaced3538a
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoDraft one of the test specification. 11/30811/7
ChristopherPrice [Fri, 17 Mar 2017 13:01:57 +0000 (14:01 +0100)]
Draft one of the test specification.

JIRA: DOVETAIL-352

Needs a lot more work, but the directory tree and layout are according to our
needs in order to have this published as a CVP document.  Will push patches
to restructure the repo according to the new requirements later.

Change-Id: I9d4316407c94bc3096d322f6206eccf236d349da
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
8 years agodovetail tool: offline support 05/31905/3
MatthewLi [Mon, 27 Mar 2017 12:17:55 +0000 (08:17 -0400)]
dovetail tool: offline support

JIRA: DOVETAIL-164

usage:
dovetail run --testsuite <testsuite> --offline
1,if --offline given, can run offline by using local docker images
(if there is no images, will get an error in log, images can be
pulled from dockerhub or downloaded from artifacts.opnfv.org )
2,if --offline not given, will run online, which means
pull docker image from dockerhub

Change-Id: I87ab35c82e59fb13a7a88c38d233e89285efc4d3
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agodovetail tool: Add an option cmd for pushing results into DB 23/31323/4
xudan [Wed, 22 Mar 2017 07:51:37 +0000 (07:51 +0000)]
dovetail tool: Add an option cmd for pushing results into DB

JIRA: DOVETAIL-373

1. add an option --report/-r to choose the report type.
2. "-r file" will store results with files.
3. "-r http://192.168.135.2:8000/api/v1" will push results into DB.
   The DB url can replaced by user's local DB or official DB.
4. Default report type is defined as "report_dest" in dovetail_config.yml.
5. Yardstick can't push results of one testcase into DB now, so both "-r file"
   and "-r http://..." will be regarded as "file".

Change-Id: Ieaa017ab59c5b15235bf306ef57cdc56beb22af8
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoLaunch a local TESTAPI DB instance 55/31455/3
Leo Wang [Thu, 23 Mar 2017 01:40:58 +0000 (21:40 -0400)]
Launch a local TESTAPI DB instance

JIRA: DOVETAIL-374

User may want to dry run the test, and does not intend to upload test results to OPNFV DB.
So dovetail provide a way to launch a local TESTAPI DB instance, then user can upload test results to this local db, and browse test results locally.

Change-Id: I17b6153c9a308bd3cd59a2450c415200496cb855
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agobugfix of package error 91/31491/1
MatthewLi [Thu, 23 Mar 2017 07:34:27 +0000 (03:34 -0400)]
bugfix of package error

JIRA: DOVETAIL-376

Change-Id: I91e0efff01d445e3f74718d730c8ff48c6413b26
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "dovetail tool: Bugfix about the prepare_env of yardstick and get the yardstick...
hongbo tian [Wed, 22 Mar 2017 09:18:57 +0000 (09:18 +0000)]
Merge "dovetail tool: Bugfix about the prepare_env of yardstick and get the yardstick.log"

8 years agodovetail tool: remove installer and CI related variables 95/30795/4
xudan [Fri, 17 Mar 2017 09:23:45 +0000 (09:23 +0000)]
dovetail tool: remove installer and CI related variables

JIRA: DOVETAIL-371

1. remove all the envs in functest_config.yml and yardstick_config.yml
2. remove all relative cmds in cmd_config.yml
3. remove all the envs when running a new functest/yardstick container
4. add some variables just for functest pushing results to db
5. get the external network name for yardstick
6. make --debug/-d just control the log level shown on the screen
7. set requests=2.10.0 since python-openstackclient needs
   'requests!=2.12.2,!=2.13.0,>=2.10.0'

Change-Id: I9f941e09d067fc95a14c3c9be1374f41df4a9a16
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: Bugfix about the prepare_env of yardstick and get the yardstick.log 89/29689/4
xudan [Fri, 3 Mar 2017 03:48:11 +0000 (03:48 +0000)]
dovetail tool: Bugfix about the prepare_env of yardstick and get the yardstick.log

1. For Fuel, yardstick prepare_env.sh will fail when
   pwd != /home/opnfv/repos/yardstick
2. Modify yardstick_config.yml in Dovetail to change pwd to that path.
3. Use log file /tmp/yardstick/yardstick.log created by cmd "yardstick task start"
   rather than redirecting the console output to yardstick.log.

JIRA: DOVETAIL-363

Change-Id: Ie0475cfabc66e342be868f0924a5fccd6fa74b58
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: change name of credentials from creds to openrc 53/30553/3
xudan [Wed, 15 Mar 2017 10:17:08 +0000 (10:17 +0000)]
dovetail tool: change name of credentials from creds to openrc

1. change file name from openstack.creds to openrc.sh
2. change the cmd --creds to --openrc/-o
3. now the path of --openrc can be relative path no restrict to abspath
4. remove the useless "work_dir" in dovetail_config.yml

JIRA: DOVETAIL-369
JIRA: DOVETAIL-367

Change-Id: I2592438eef060f84132ff7fa226aea93ea947362
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "dovetail tool: bugfix for dovetail report and docker build"
hongbo tian [Fri, 24 Feb 2017 07:51:43 +0000 (07:51 +0000)]
Merge "dovetail tool: bugfix for dovetail report and docker build"

8 years agodovetail tool: Bugfix about the creds path in yardstick conf file 05/29105/1
xudan [Tue, 21 Feb 2017 01:57:36 +0000 (01:57 +0000)]
dovetail tool: Bugfix about the creds path in yardstick conf file

JIRA: DOVETAIL-355

Change-Id: I04a74caa080e0fd2ea11648f8085947ef8d556b1
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: bugfix for dovetail report and docker build 45/29045/2
MatthewLi [Mon, 20 Feb 2017 06:28:24 +0000 (01:28 -0500)]
dovetail tool: bugfix for dovetail report and docker build

JIRA: DOVETAIL-353
JIRA: DOVETAIL-354

related patch: https://gerrit.opnfv.org/gerrit/#/c/29049/

Change-Id: Ic871abbda62dead7cf5e23e6117c9885d637c7fb
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "dovetail tool: skip testcase status added"
hongbo tian [Sat, 18 Feb 2017 03:01:05 +0000 (03:01 +0000)]
Merge "dovetail tool: skip testcase status added"

8 years agodovetail tool: change the way to get the functest result file 19/28919/2
xudan [Fri, 17 Feb 2017 09:49:16 +0000 (09:49 +0000)]
dovetail tool: change the way to get the functest result file

Functest has changed it's way to generate the local result file.
see Gerrit: https://gerrit.opnfv.org/gerrit/#/c/28513/
Dovetail needs to do some adaptions to fit this change.

JIRA: DOVETAIL-350

Change-Id: I681f74832f582f26c8874f7d2726c8588f430612
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: skip testcase status added 69/28269/6
MatthewLi [Wed, 8 Feb 2017 07:52:27 +0000 (02:52 -0500)]
dovetail tool: skip testcase status added

JIRA: DOVETAIL-344

1) testcase status, True/False --> PASS/SKIP/FAIL,
   so skipped testcase in tempest can be figured out
   details can be seen in
   https://etherpad.opnfv.org/p/dovetail_report
2) some variable name changed more readable, such as
   passed-->testcase_passed failed--> prepare_failed
3) "dovetail report" minor amend according to 1)

Change-Id: Ibe8e50693fc474d78e7d0fb89257f34703248579
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "dovetail tool: use python new-style classes which inherit from object"
hongbo tian [Wed, 8 Feb 2017 03:11:00 +0000 (03:11 +0000)]
Merge "dovetail tool: use python new-style classes which inherit from object"

8 years agoMerge "bugfix of dovetail tool userguide document"
hongbo tian [Tue, 7 Feb 2017 11:37:33 +0000 (11:37 +0000)]
Merge "bugfix of dovetail tool userguide document"

8 years agoMerge "dovetail tool: report.py bugfix and tempest skipped testcase added"
hongbo tian [Tue, 7 Feb 2017 11:35:53 +0000 (11:35 +0000)]
Merge "dovetail tool: report.py bugfix and tempest skipped testcase added"

8 years agodovetail tool: report.py bugfix and tempest skipped testcase added 23/28023/6
MatthewLi [Sat, 4 Feb 2017 06:32:43 +0000 (01:32 -0500)]
dovetail tool: report.py bugfix and tempest skipped testcase added

JIRA: DOVETAIL-343

1, bugfix
https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-debug-master/lastSuccessfulBuild/artifact/results/dovetail_report.txt
test result error
2, skipped testcase in tempest added
3, two testcase in tempest full testsuite added in debug.yml to debug

Change-Id: I6bf371d3384e3755d6a41d951cfc9d1a55e00ab3
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agobugfix of dovetail tool userguide document 79/28179/1
MatthewLi [Tue, 7 Feb 2017 08:00:46 +0000 (03:00 -0500)]
bugfix of dovetail tool userguide document

Change-Id: I33eea93ca0db018a5f79041152516fa48b94ba46
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agodovetail tool: bugfix: cannot load pre/post condition from config file 67/27967/1
xudan [Fri, 3 Feb 2017 03:52:11 +0000 (03:52 +0000)]
dovetail tool: bugfix: cannot load pre/post condition from config file

JIRA: DOVETAIL-342

Change-Id: If98905b6724c0a990d789a7ce914aeecac72d7c4
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "Change to use "pip install -r requirements.txt" to install deps."
hongbo tian [Thu, 26 Jan 2017 04:42:48 +0000 (04:42 +0000)]
Merge "Change to use "pip install -r requirements.txt" to install deps."

8 years agodovetail tool: functest different testsuite support and yardstick testcase added 15/27215/5
MatthewLi [Thu, 19 Jan 2017 07:07:16 +0000 (02:07 -0500)]
dovetail tool: functest different testsuite support and yardstick testcase added

JIRA: DOVETAIL-186

Change-Id: Id0eec6652c9595c660436dec89a309a77b16bc03
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoChange to use "pip install -r requirements.txt" to install deps. 63/27263/2
Tomofumi Hayashi [Fri, 20 Jan 2017 04:10:50 +0000 (13:10 +0900)]
Change to use "pip install -r requirements.txt" to install deps.

At the change of DOVETAIL-183, dovetail uses requirements.txt to
install python dependency file so this change is following it.

Change-Id: I6cb2420274f385224b517950ddfdf1cf0e764350
Signed-off-by: Tomofumi Hayashi <tohayash@redhat.com>
8 years agoMerge "dovetail tool userguide document"
hongbo tian [Mon, 23 Jan 2017 07:52:38 +0000 (07:52 +0000)]
Merge "dovetail tool userguide document"

8 years agodovetail tool: use python new-style classes which inherit from object 73/27273/2
xudan [Fri, 20 Jan 2017 06:46:57 +0000 (06:46 +0000)]
dovetail tool: use python new-style classes which inherit from object

JIRA: DOVETAIL-187

Change-Id: Id6fa50a3a3cec7a18d27ca8feb383b53836a339b
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: modify function exec_cmd to force it to flush the buffer 89/26989/5
xudan [Sat, 14 Jan 2017 01:27:09 +0000 (01:27 +0000)]
dovetail tool: modify function exec_cmd to force it to flush the buffer

1. force to flush the buffer line by line
2. remove the progress bar since it will stay in the screen if using the cli
   command JIRA: DOVETAIL-173, which will be ugly.

JIRA: DOVETAIL-172

Change-Id: I47b823c7e0bce955a50086613a656c83ab7707e3
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool userguide document 01/27001/11
MatthewLi [Sat, 14 Jan 2017 09:08:13 +0000 (04:08 -0500)]
dovetail tool userguide document

JIRA: DOVETAIL-182

Change-Id: Icbfa347dda3840520484d10b9d539da9c8f1d31c
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agobugfix for delete unnecessary file 95/27095/1
MatthewLi [Tue, 17 Jan 2017 11:49:06 +0000 (06:49 -0500)]
bugfix for delete unnecessary file

Change-Id: I14b1556eb117d2111ebe85e73b8c33820db39a3e
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "dovetail tool: format improvement"
hongbo tian [Tue, 17 Jan 2017 11:44:12 +0000 (11:44 +0000)]
Merge "dovetail tool: format improvement"

8 years agodovetail tool: change to git repo then pbr.version can be get inside docker 91/27091/1
MatthewLi [Tue, 17 Jan 2017 11:12:53 +0000 (06:12 -0500)]
dovetail tool: change to git repo then pbr.version can be get inside docker

JIRA: DOVETAIL-184

Change-Id: I6d466dc34de027faab1aa74f66cc6c8e453111d1
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agodovetail tool: format improvement 41/27041/1
MatthewLi [Mon, 16 Jan 2017 09:32:55 +0000 (04:32 -0500)]
dovetail tool: format improvement

Change-Id: Ibf0bb781c0f0b6be73526bf45231040e68510f76
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>