dovetail.git
8 years ago[dovetail tool] support shell scripts for testcase validation 51/25951/1
Leo Wang [Wed, 14 Dec 2016 09:20:09 +0000 (04:20 -0500)]
[dovetail tool] support shell scripts for testcase validation

JIRA: DOVETAIL-46

1. for now a testcase has two kinds of validation types(functest, yardstick),
and it is not enough to check the complete funcionality
2. add new validation type(shell) for extra validation of the test case to make
result more accurate and more convincing.

Change-Id: I5c049a71f11cca71a7914f8af704c1983aba3dca
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years ago[dovetail tool] support shell scripts for testcase validation 31/25931/1
Leo Wang [Wed, 14 Dec 2016 07:48:16 +0000 (02:48 -0500)]
[dovetail tool] support shell scripts for testcase validation

JIRA: DOVETAIL-46

1. for now a testcase has two kinds of validation types(functest, yardstick),
and it is not enough to check the complete funcionality
2. add new validation type(shell) for extra validation of the test case to make
result more accurate and more convincing.

Change-Id: I60c1b54335b94e0cb150232432adb958b8f5a143
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agoMerge "bugfix of document error"
hongbo tian [Wed, 14 Dec 2016 01:23:19 +0000 (01:23 +0000)]
Merge "bugfix of document error"

8 years agobugfix of document error 85/25785/4
MatthewLi [Mon, 12 Dec 2016 09:19:24 +0000 (04:19 -0500)]
bugfix of document error

JIRA: DOVETAIL-155

errors can refer to
https://build.opnfv.org/ci/job/opnfv-docs-verify-master/3727/console

Change-Id: I19e96239c8183bafca731477723615caff71388b
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years ago[dovetail tool] support shell scripts for testcase validation 85/25085/15
Leo Wang [Tue, 29 Nov 2016 09:21:40 +0000 (04:21 -0500)]
[dovetail tool] support shell scripts for testcase validation

JIRA: DOVETAIL-46

1. for now a testcase has two kinds of validation types(functest, yardstick),
and it is not enough to check the complete funcionality
2. add new validation type(shell) for extra validation of the test case to make
result more accurate and more convincing.

Change-Id: I45dca6b8dbd888757da163189d261f6e4dba5034
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agobugfix for flake8 format 25/25825/1
MatthewLi [Tue, 13 Dec 2016 06:23:11 +0000 (01:23 -0500)]
bugfix for flake8 format

error log see https://build.opnfv.org/ci/view/dovetail/job/dovetail-merge-master/58/console

Change-Id: I7f5a7f290d82e79249561322259aa5e72186e126
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "[WIP] Migrate to tox based environment"
hongbo tian [Tue, 13 Dec 2016 06:14:07 +0000 (06:14 +0000)]
Merge "[WIP] Migrate to tox based environment"

8 years ago[WIP] Migrate to tox based environment 57/23757/12
Leif Madsen [Fri, 28 Oct 2016 09:59:53 +0000 (05:59 -0400)]
[WIP] Migrate to tox based environment

Migrate the custom setup.py to a more formal setup
environment using tox, in the same way that OpenStack
configures their projects.

Adds metadata to tox.ini from the setup.py file, and breaks
out the dependencies from setup.py into requirements.txt
and test-requirements.txt.

Executes flake8 testing for pep8 coverage.

Defaults to using Python 2 for the environment.

Change-Id: Ieaea7d9f3dc227febd8fb1dc9142d478ba197a92
Signed-off-by: Leif Madsen <lmadsen@redhat.com>
Signed-off-by: zshi <zshi@redhat.com>
8 years agoMerge "Add more document and fix dockerfile for CentOS"
hongbo tian [Tue, 13 Dec 2016 01:54:06 +0000 (01:54 +0000)]
Merge "Add more document and fix dockerfile for CentOS"

8 years agoMerge "dovetail tool: add debug option to control the log level showing in screen"
hongbo tian [Tue, 13 Dec 2016 01:53:21 +0000 (01:53 +0000)]
Merge "dovetail tool: add debug option to control the log level showing in screen"

8 years ago[dovetail tool]: move config info from .py to config .yml file 13/25713/2
MatthewLi [Fri, 9 Dec 2016 08:31:28 +0000 (03:31 -0500)]
[dovetail tool]: move config info from .py to config .yml file

JIRA: DOVETAIL-145

Change-Id: Ia9b3c678f1a491e5daf2d280f05904ead3b0a296
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoAdd more document and fix dockerfile for CentOS 35/25635/4
Tomofumi Hayashi [Wed, 7 Dec 2016 15:19:46 +0000 (00:19 +0900)]
Add more document and fix dockerfile for CentOS

Add how to build container from dockerfile and add missing config
for CentOS dockerfile. This fix is verified that functest/yeardstick
are called successfully under debug/proposed_tests testsuite.

Change-Id: Icb31331ecd78b8fccd69ac960c24340b605e54bf
Signed-off-by: Tomofumi Hayashi <tohayash@redhat.com>
8 years ago2 config files description documents 31/24131/3
MatthewLi [Thu, 10 Nov 2016 02:44:51 +0000 (21:44 -0500)]
2 config files description documents

JIRA: DOVETAIL-50

Change-Id: I65c618ce3de78677b9e66a1c2d7f0af705e80cfb
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "[RFC]Add dockerfile to support dovetail in CentOS"
hongbo tian [Wed, 7 Dec 2016 13:58:23 +0000 (13:58 +0000)]
Merge "[RFC]Add dockerfile to support dovetail in CentOS"

8 years agoMerge "Copy the jumphost ssh key into the container"
hongbo tian [Wed, 7 Dec 2016 13:53:00 +0000 (13:53 +0000)]
Merge "Copy the jumphost ssh key into the container"

8 years agoCopy the jumphost ssh key into the container 41/25541/2
Dan Radez [Tue, 6 Dec 2016 14:17:17 +0000 (09:17 -0500)]
Copy the jumphost ssh key into the container

When functest runs against Apex it needs to ssh from the container
to the undercloud to get the keystonerc file so it can authenticate
to the overcloud.

Change-Id: Iea3273d4d1611733f167ffab0b2a75330c413edc
Signed-off-by: Dan Radez <dradez@redhat.com>
8 years ago[RFC]Add dockerfile to support dovetail in CentOS 95/25195/2
Tomofumi Hayashi [Wed, 30 Nov 2016 02:44:44 +0000 (11:44 +0900)]
[RFC]Add dockerfile to support dovetail in CentOS

JIRA: DOVETAIL-151

This diff introduces an optional dovetail dockerfile based on CentOS.
It provides users to have an alternative way to execute dovetail.
It should be just a optional, so we should keep docker/Dockerfile
as official one with dockerhub's image and we do not build docker images
in CI phase for now.

Change-Id: Ia694bcd49f65ff2bc175c1da3b42fbfa19b022a1
Signed-off-by: Tomofumi Hayashi <tohayash@redhat.com>
Signed-off-by: Zenghui Shi <zshi@redhat.com>
8 years agodovetail tool: add unit test into script unittest.sh 69/25469/1
xudan [Mon, 5 Dec 2016 08:28:20 +0000 (08:28 +0000)]
dovetail tool: add unit test into script unittest.sh

1. Since JIRA: DOVETAIL-55 has been merged, unit tests can't pass without
   load_config_files().
2. Fix the bug by adding load_config_files().
3. Add run_tests() into script unittest.sh to include all unit tests in
   tests/unit.

JIRA: DOVETAIL-149

Change-Id: I7bdb0732fefde3a3f3b82441b4cc7e46ece66a1e
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: add debug option to control the log level showing in screen 01/25301/2
xudan [Thu, 1 Dec 2016 06:38:51 +0000 (06:38 +0000)]
dovetail tool: add debug option to control the log level showing in screen

1. add debug option in cmd_config.yml for controling log showing in screen.
2. rename existent DEBUG as CON_DEBUG because it is used for functest/yardstick
   container to control the log level.
3. fix the bug in function update_config_envs(). If one key is not in system's
   envs, os.environ[key] get an error with ErrorKey.

JIRA: DOVETAIL-148

Change-Id: If3d75109f51e2e91743b8f1134a76da42e57383a
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "dovetail tool: reorganize statements in class DovetailConfig"
hongbo tian [Fri, 2 Dec 2016 17:43:00 +0000 (17:43 +0000)]
Merge "dovetail tool: reorganize statements in class DovetailConfig"

8 years agoMerge "dovetail tool: add brackets to print message for consistency"
hongbo tian [Fri, 2 Dec 2016 17:42:46 +0000 (17:42 +0000)]
Merge "dovetail tool: add brackets to print message for consistency"

8 years agoMerge "dovetail tool: fix unittest bugs"
hongbo tian [Fri, 2 Dec 2016 17:42:10 +0000 (17:42 +0000)]
Merge "dovetail tool: fix unittest bugs"

8 years agoMerge "[WIP] dovetail report improvement"
hongbo tian [Fri, 2 Dec 2016 17:41:37 +0000 (17:41 +0000)]
Merge "[WIP] dovetail report improvement"

8 years ago[WIP] dovetail report improvement 93/25393/5
MatthewLi [Fri, 2 Dec 2016 03:29:26 +0000 (22:29 -0500)]
[WIP] dovetail report improvement

JIRA: DOVETAIL-49

1)details please refer to
  https://wiki.opnfv.org/display/dovetail/Dovetail+Reports
2)results please refer to one example
  https://etherpad.opnfv.org/p/dovetail_report

Change-Id: I63a1ba3b6aa1ffe5fa043a11cd3777dd0e8d52d9
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agobugfix: Add BRANCH Argument in the Dockerfile 89/25289/2
MatthewLi [Thu, 1 Dec 2016 01:53:22 +0000 (20:53 -0500)]
bugfix: Add BRANCH Argument in the Dockerfile

JIRA: DOVETAIL-147

1) BRANCH argument added to fix bug in CI
2) docker to use latest version, since BRANCH added not works with old version

Change-Id: Icb0144896389b0cb7ac80e1fb0338825be50d5b1
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agodovetail tool: fix unittest bugs 07/25207/2
zshi [Wed, 30 Nov 2016 07:12:28 +0000 (15:12 +0800)]
dovetail tool: fix unittest bugs

JIRA: DOVETAIL-143

1) fix unittest import module path
2) add create_log in setup

Change-Id: Ic894fa971b4a5091affb36e8e6aac05e0e8623fb
Signed-off-by: zshi <zshi@redhat.com>
8 years agoMerge "dovetail tool: run.py execution always wipes of results files"
hongbo tian [Wed, 30 Nov 2016 03:03:09 +0000 (03:03 +0000)]
Merge "dovetail tool: run.py execution always wipes of results files"

8 years agodovetail: modify process flow in function update_envs() 25/25125/3
wu.zhihui [Tue, 29 Nov 2016 10:02:15 +0000 (18:02 +0800)]
dovetail: modify process flow in function update_envs()

bugfix task DOVETAIL-144
1. If run command "python run.py" without options, e.g. --SUT_TYPE, --SUT_IP,
env values will be updated by os enviromental parameter.
2. If run command "python run.py" with options, no update from os enviromental
parameter.

JIRA: DOVETAIL-144

Change-Id: Id3bde7546722a5f4defe52d4f00609d7049a288b
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
8 years agoMerge "fix flake8 test error for ez_setup.py"
hongbo tian [Wed, 30 Nov 2016 01:23:40 +0000 (01:23 +0000)]
Merge "fix flake8 test error for ez_setup.py"

8 years agodovetail tool: run.py execution always wipes of results files 29/24929/4
xudan [Sat, 26 Nov 2016 08:19:21 +0000 (08:19 +0000)]
dovetail tool: run.py execution always wipes of results files

1. move function clean_results_dir() into run.py
2. call this function at the beginning of function main()
3. this can only work correctly after all log initializations packaged,
   see JIRA: DOVETAIL-56 and gerrit https://gerrit.opnfv.org/gerrit/#/c/24059/

JIRA: DOVETAIL-80

Change-Id: I1cdda8f6d43f21c45c0537868cb7ed10d14ea9df
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: reorganize statements in class DovetailConfig 31/24931/3
xudan [Sat, 26 Nov 2016 08:43:47 +0000 (08:43 +0000)]
dovetail tool: reorganize statements in class DovetailConfig

1. move some statements in class DovetailConfig into function
   load_config_files()
2. call load_config_files() in run.py
3. this can only work correctly after moving clean_results_dir() into run.py,
   see JIRA:DOVETAIL-80 and gerrit https://gerrit.opnfv.org/gerrit/#/c/24929/

JIRA: DOVETAIL-55

Change-Id: I217c1c21dc6ba8849da7c00564f62d3c22cdc391
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: To exec testsuite loading after command line arguments is True 53/25053/2
MatthewLi [Tue, 29 Nov 2016 02:25:35 +0000 (21:25 -0500)]
dovetail tool: To exec testsuite loading after command line arguments is True

JIRA: DOVETAIL-142

Change-Id: I1d8a444580755b4f4d9a0f7098b4b17ea66d6f58
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "dovetail tool: change CI_DEBUG to DEBUG"
hongbo tian [Tue, 29 Nov 2016 02:22:49 +0000 (02:22 +0000)]
Merge "dovetail tool: change CI_DEBUG to DEBUG"

8 years agodovetail tool: package all logger initialization 59/24059/9
xudan [Tue, 8 Nov 2016 11:43:28 +0000 (11:43 +0000)]
dovetail tool: package all logger initialization

1. For container.py, parser.py, report.py and testcase.py, they use functions
   in classes to create loggers. So each class will have its own logger.
2. run.py will create all the loggers by create_logs() function before the use
   of them. So logs will not be initialized until the call of create_logs().
3. Move logger in run.py into function main().

JIRA: DOVETAIL-56

Change-Id: I53de090df270cc7a34762ff99cbc9115cfb09465
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "dovetail tool: testarea argument added"
hongbo tian [Tue, 29 Nov 2016 01:31:04 +0000 (01:31 +0000)]
Merge "dovetail tool: testarea argument added"

8 years agoMerge "Backend rest api mechanism"
hongbo tian [Mon, 28 Nov 2016 08:27:25 +0000 (08:27 +0000)]
Merge "Backend rest api mechanism"

8 years agodovetail tool: add brackets to print message for consistency 79/24979/1
zshi [Mon, 28 Nov 2016 05:26:20 +0000 (13:26 +0800)]
dovetail tool: add brackets to print message for consistency

Change-Id: I74c0aefb7ada960aac55aed15aab6b7b9984154e
Signed-off-by: zshi <zshi@redhat.com>
8 years agofix flake8 test error for ez_setup.py 65/24965/1
zshi [Mon, 28 Nov 2016 04:49:19 +0000 (12:49 +0800)]
fix flake8 test error for ez_setup.py

Change-Id: Ibb764a41bac05599645d283c5fe3dbe409b2b006
Signed-off-by: zshi <zshi@redhat.com>
8 years agodovetail tool: change CI_DEBUG to DEBUG 11/24011/3
zshi [Mon, 28 Nov 2016 03:00:35 +0000 (11:00 +0800)]
dovetail tool: change CI_DEBUG to DEBUG

JIRA: DOVETAIL-42

1) change 'CI_DEBUG' to 'DEBUG' in cmd_config.yml
   and dovetail_logger.py
2) assign 'DEBUG' env value to 'CI_DEBUG' when
   updating functest and yardstick parameter 'envs'

Change-Id: I6d6130e394fddf11d83be619f2c9684bf8c859f9
Signed-off-by: zshi <zshi@redhat.com>
8 years agodovetail tool: testarea argument added 29/24829/7
MatthewLi [Thu, 24 Nov 2016 06:56:41 +0000 (01:56 -0500)]
dovetail tool: testarea argument added

JIRA: DOVETAIL-71

1) argument testarea is added
2) terminology "scenario" change to "testsuite"
3) related documents amended
4) all word "certification" changed to "compliance"

Change-Id: I038a9e04bf83cfdac20f0c59adb1841bf5470584
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "dovetail tool: reorganize dovetail_config.py and modify related files"
hongbo tian [Fri, 25 Nov 2016 08:00:12 +0000 (08:00 +0000)]
Merge "dovetail tool: reorganize dovetail_config.py and modify related files"

8 years ago[Dovetail Tool] Prepare env fail due to dependency on pyyaml 37/24837/2
Leo Wang [Thu, 24 Nov 2016 08:10:14 +0000 (03:10 -0500)]
[Dovetail Tool] Prepare env fail due to dependency on pyyaml

JIRA:DOVETAIL-92

root cause:
1. import dovetail_logger while it depends on dovetail_config
2. dovetail_config import yaml
3. yaml is not installed yet
action:
1. remove dovetail_logger

Change-Id: I51f5e8bab2451639a4a9ebf1978fe8a292df1704
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agoBackend rest api mechanism 63/24463/3
Leo Wang [Thu, 17 Nov 2016 03:44:23 +0000 (22:44 -0500)]
Backend rest api mechanism

JIRA:DOVETAIL-63

provide rest api as the dashboard backend
1. using gunicorn as rest api server
2. using flask as rest api framework
3. using sqlalchemy as mysql database driver
4. implement basic report CRUD operations
5. implement basic session management in  database operations

Change-Id: Ifbd251462396c2cb414b1ae9150cfc1e2e2d00c0
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agodovetail tool: upstream functest env preparation and CLI adaption 55/24455/1
xudan [Thu, 17 Nov 2016 02:11:30 +0000 (02:11 +0000)]
dovetail tool: upstream functest env preparation and CLI adaption

1. Functest has changed it's directory structure, so cmds in
   functest_config.yml cannot run successfully.
2. Use functest CLI to prepare and run test cases instead of running
   prepare_env.py or run_tests.py.

JIRA: DOVETAIL-40

Change-Id: I7907dd70fd671efbed5f4132c0d76d6eb90b63eb
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: reorganize dovetail_config.py and modify related files 93/24093/4
xudan [Wed, 9 Nov 2016 06:26:11 +0000 (06:26 +0000)]
dovetail tool: reorganize dovetail_config.py and modify related files

1. move all sentencs and functions into class DovetailConfig
2. delete dict container_config for it has been replaced by dovetail_config
2. modify files that import dovetail_config, including container.py, parser.py,
   report.py, run.py, dovetail_logger.py and testcase.py

JIRA: DOVETAIL-55

Change-Id: Ib5ca3cadd751075c879f39e8cb2eb2e0adcef6aa
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "Fix option handling in case of no '--tag' option."
hongbo tian [Wed, 16 Nov 2016 08:00:54 +0000 (08:00 +0000)]
Merge "Fix option handling in case of no '--tag' option."

8 years agoMerge "dovetail tool: cleanup the result directory before running"
hongbo tian [Wed, 16 Nov 2016 08:00:31 +0000 (08:00 +0000)]
Merge "dovetail tool: cleanup the result directory before running"

8 years agodovetail tool: cleanup the result directory before running 75/24375/1
xudan [Wed, 16 Nov 2016 02:33:49 +0000 (02:33 +0000)]
dovetail tool: cleanup the result directory before running

1. Jenkins will fail with error
   rm: cannot remove '/home/opnfv/dovetail/results': Device or resource busy
2. The reason is that Jenkins will mount this directory before running
3. The effective solution is removing all contents in this directory rather
   than removing this mounted directory

JIRA: DOVETAIL-38

Change-Id: Ic32ea19341ad2048322ec11a4abc48e13019a7bb
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: replace the hard-coded "Tescase" and modify some classmethod 01/24301/2
Linghui Zeng [Mon, 14 Nov 2016 02:29:21 +0000 (10:29 +0800)]
dovetail tool: replace the hard-coded "Tescase" and modify some classmethod

JIRA: DOVETAIL-45

1. In the testcase.py file, we replace the hard-coded "Tesecase" with "cls" or "self" based on
   the specific contexts.
2. For all the four committed files, we modify some classmethod into staticmethod because each
   method is basically just a function, called syntactically like a method, but without access
   to the object and its' internals (attributes and other methods).

Change-Id: Ieb452f476a2d33ae9aca4c904ae7d2c92b68689e
Signed-off-by: Linghui Zeng <linghui.zeng@huawei.com>
8 years agoFix option handling in case of no '--tag' option. 45/24345/1
Tomofumi Hayashi [Tue, 15 Nov 2016 09:26:29 +0000 (18:26 +0900)]
Fix option handling in case of no '--tag' option.

In case of default, 'None' is in kwargs['tag'] so need to
check it in option handling. The code is to fix it.

Change-Id: Iaadf7e6f0f064dda819970346a810a6a8b47c74c
Signed-off-by: Tomofumi Hayashi <tohayash@redhat.com>
8 years agoMerge "dovetail tool: reorganize prepare_env.py"
hongbo tian [Tue, 15 Nov 2016 08:45:06 +0000 (08:45 +0000)]
Merge "dovetail tool: reorganize prepare_env.py"

8 years agoMerge "generate report data for reuse"
hongbo tian [Tue, 15 Nov 2016 08:44:17 +0000 (08:44 +0000)]
Merge "generate report data for reuse"

8 years agoMerge "Add "docker tag" option for each testcase containers."
hongbo tian [Tue, 15 Nov 2016 08:43:55 +0000 (08:43 +0000)]
Merge "Add "docker tag" option for each testcase containers."

8 years agogenerate report data for reuse 71/24171/8
Leo Wang [Thu, 10 Nov 2016 09:29:07 +0000 (04:29 -0500)]
generate report data for reuse

JIRA: DOVETAIL-22

1. report text is string, can not be reused by other module,
2. generate report data first, then this data can be uploaded
   to database or be translated to report text

Change-Id: Ieecd87c827537c825c87a0db4ff170ae490d75ae
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agodovetail tool: reorganize prepare_env.py 61/24161/3
xudan [Thu, 10 Nov 2016 07:30:15 +0000 (07:30 +0000)]
dovetail tool: reorganize prepare_env.py

1. add main function and move statements into it

JIRA: DOVETAIL-57

Change-Id: I69158fa64ffc10668fc39835de434f0b44b8754a
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoAdd "docker tag" option for each testcase containers. 41/24141/4
Tomofumi Hayashi [Thu, 10 Nov 2016 05:49:46 +0000 (14:49 +0900)]
Add "docker tag" option for each testcase containers.

JIRA: DOVETAIL-69

This commit is to add new option to specify docker tags for
each test containers, such as '--TAG "functest:stable"'.

Change-Id: Ica11e16abfb21649a3eea9c7d7d5cd31502d2e21
Signed-off-by: Tomofumi Hayashi <tohayash@redhat.com>
8 years agodovetail tool: 2 config files, compliance_set and proposed_tests 43/23843/8
MatthewLi [Tue, 1 Nov 2016 12:01:51 +0000 (08:01 -0400)]
dovetail tool: 2 config files, compliance_set and proposed_tests

JIRA: DOVETAIL-50

1)'proposed_tests.yml' is added, which runs as the total tests run.
  The 'compliance_set.yml' will run as dovetail certification tests;
2)ipv6 test cases added accroding to 'docs/testsuites/ipv6/ipv6_all_testcases',
  tc026 missed temporarily,yardstick ipv6tc026 only supports 2 installers by now

according to https://wiki.opnfv.org/display/dovetail/Dovetail+Test+Cases
3)vimops001-006 added for VIM images operation test cases
4)nfvi001-002 added for vping ssh/userdata in fucntest

5)more test cases can be added in future in proposed_tests.yml,
 besides, the test cases can be adjusted in future in proposed_tests.yml

Change-Id: I8a8634583549a802f6f64fbea651e7be4efa361c
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "dovetail tool: ensure sub_testcase_list and testcase are not None"
hongbo tian [Wed, 9 Nov 2016 07:16:29 +0000 (07:16 +0000)]
Merge "dovetail tool: ensure sub_testcase_list and testcase are not None"

8 years agoMerge "dovetail tool: os name bugfix"
hongbo tian [Wed, 9 Nov 2016 07:15:50 +0000 (07:15 +0000)]
Merge "dovetail tool: os name bugfix"

8 years agoMerge "dovetail tool: cleanup result directory before running"
hongbo tian [Wed, 9 Nov 2016 07:15:18 +0000 (07:15 +0000)]
Merge "dovetail tool: cleanup result directory before running"

8 years agodovetail tool: os name bugfix 13/24013/3
MatthewLi [Mon, 7 Nov 2016 08:03:06 +0000 (03:03 -0500)]
dovetail tool: os name bugfix

JIRA: DOVETAIL-54

bug: in some envs, return value platform_os is Ubuntu rather than ubuntu.

Change-Id: I6a2aa964d00eff9f336f8204071a05a6b5f11134
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agodovetail tool: ensure sub_testcase_list and testcase are not None 19/23919/7
zshi [Thu, 3 Nov 2016 08:46:19 +0000 (16:46 +0800)]
dovetail tool: ensure sub_testcase_list and testcase are not None

JIRA: DOVETAIL-52

1) save return value of testcase.sub_testcase() in sub_testcase_list
for multiple calls later
2) ensure sub_testcase_list is not called with None value
3) ensure testcase is not called with None value
4) identify None testcase with 'Undefined' in dovetail_report.txt

Change-Id: Ifcf0c0846dc788575310f29932a4757752f1e3a3
Signed-off-by: zshi <zshi@redhat.com>
8 years agoMerge "dovetail tool: Add cli mechanism for changing envs at run-time"
hongbo tian [Thu, 3 Nov 2016 07:40:28 +0000 (07:40 +0000)]
Merge "dovetail tool: Add cli mechanism for changing envs at run-time"

8 years agoMerge "dovetail tool: rename report_file_path variable"
hongbo tian [Thu, 3 Nov 2016 07:40:15 +0000 (07:40 +0000)]
Merge "dovetail tool: rename report_file_path variable"

8 years agoMerge "dovetail tool: Cleanup functest_conf.yml"
hongbo tian [Thu, 3 Nov 2016 07:40:04 +0000 (07:40 +0000)]
Merge "dovetail tool: Cleanup functest_conf.yml"

8 years agoMerge "Add more distro support when preparing environment"
hongbo tian [Thu, 3 Nov 2016 07:39:52 +0000 (07:39 +0000)]
Merge "Add more distro support when preparing environment"

8 years agoMerge "dovetail tool: ignore file amended"
hongbo tian [Thu, 3 Nov 2016 07:39:35 +0000 (07:39 +0000)]
Merge "dovetail tool: ignore file amended"

8 years agodovetail tool: cleanup result directory before running 77/23677/3
xudan [Wed, 26 Oct 2016 02:27:12 +0000 (02:27 +0000)]
dovetail tool: cleanup result directory before running

JIRA: DOVETAIL-38

1. Do cleaning at the beginning of each run.
2. Since some files and folders have the 'root root' privilege,
   no command except cmd 'sudo rm -rf <results_dir>' can work.

Change-Id: If8cd5c8c9c35434af2a5fa38575c06f83f0afc4d
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: Add cli mechanism for changing envs at run-time 95/23495/6
xudan [Fri, 21 Oct 2016 08:03:43 +0000 (08:03 +0000)]
dovetail tool: Add cli mechanism for changing envs at run-time

JIRA: DOVETAIL-23

1. Add file cmd_config.yml for defining arguments and options.
2. Using python click module to add and parse cmds in cmd_config.yml
3. Using envs options to update envs in yardstick/functest config file

Change-Id: Ie513c0f0a1ed76f29408419a4bcbbcf21334bb3f
Signed-off-by: xudan <xudan16@huawei.com>
8 years agodovetail tool: ignore file amended 01/23401/6
MatthewLi [Thu, 20 Oct 2016 03:49:49 +0000 (23:49 -0400)]
dovetail tool: ignore file amended

JIRA: DOVETAIL-37

1)as much as possible to stay the same
2)*.mo deleted since no C used in this repo
3)unittest-log.txt added, it can be produced by using unittest.sh -f
4)document related added, including *.sw?, CI related /docs_build/ /docs_ouput/, results dir
5)other personal habit will not included in this

Change-Id: I1e3c95d74f99c212838a987f1705af3902f6f58d
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agodovetail tool: Add unit tests for 'parser' module 63/23763/1
mathsyouth [Sat, 29 Oct 2016 08:53:47 +0000 (16:53 +0800)]
dovetail tool: Add unit tests for 'parser' module

JIRA: DOVETAIL-45

1. Add two '__init__.py' files for packages
2. Add two test functions in 'test_parser.py' to verify the function
   of the method 'parser_cmd'
3. 'test_testcase.yaml' is used in 'test_parser.py'

Change-Id: Ic3e2e6e67bcac3276fe47ec5584c0d9632b5286a
Signed-off-by: mathsyouth <linghui.zeng@huawei.com>
8 years agodovetail tool: Cleanup functest_conf.yml 39/23739/2
zshi [Fri, 28 Oct 2016 04:57:46 +0000 (12:57 +0800)]
dovetail tool: Cleanup functest_conf.yml

JIRA: DOVETAIL-43
JIRA: DOVETAIL-44

1) remove result_dir
2) remove pre_cmd, exec_cmd, post_cmd

Change-Id: Iea8173026b1f467011a40c6954581f25cd5e1912
Signed-off-by: zshi <zshi@redhat.com>
8 years agoMerge "dovetail tool: developer userguide document"
hongbo tian [Fri, 28 Oct 2016 05:54:43 +0000 (05:54 +0000)]
Merge "dovetail tool: developer userguide document"

8 years agoAdd more distro support when preparing environment 03/23403/2
zshi [Thu, 20 Oct 2016 05:02:38 +0000 (13:02 +0800)]
Add more distro support when preparing environment

Change-Id: I3ff4419e38872bc28e59699a11080777ea596dc3
Signed-off-by: zshi <zshi@redhat.com>
8 years agodovetail tool: developer userguide document 99/23399/4
MatthewLi [Thu, 20 Oct 2016 03:14:09 +0000 (23:14 -0400)]
dovetail tool: developer userguide document

JIRA: DOVETAIL-28

1) documents are reorganized, all dovetail tool related documents are moved to /docs/dovetailtool
2) some sections are rewritten to make it more clear

Change-Id: Ib0c23b8c1771eb6de7d6cc264c101e8e26ed3d1f
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agodovetail tool: rename report_file_path variable 07/23507/1
zshi [Fri, 21 Oct 2016 11:40:05 +0000 (19:40 +0800)]
dovetail tool: rename report_file_path variable

the value assigned to report_file_path is a file
name, not a path.

Change-Id: I338ae2bb5e836d26c711bfa9144f16d06c08deb7
Signed-off-by: zshi <zshi@redhat.com>
8 years agoSimplify getting pre_condition cmds 07/23407/3
zshi [Thu, 20 Oct 2016 07:53:37 +0000 (15:53 +0800)]
Simplify getting pre_condition cmds

Change-Id: I8ca04cbdc4236e9ff754bad816bb7282673b5b55
Signed-off-by: zshi <zshi@redhat.com>
8 years agodovetail tool : add @classmethod to post_condition_cls 05/23405/2
zshi [Thu, 20 Oct 2016 07:42:35 +0000 (15:42 +0800)]
dovetail tool : add @classmethod to post_condition_cls

JIRA: DOVETAIL-39

Turn post_condition_cls to classmethod function which will
be consistent with pre_condition_cls

Change-Id: If8457035493f2ed93bdae5be5f22a57255b2c474
Signed-off-by: zshi <zshi@redhat.com>
8 years agoupdate the committer email address 69/23269/4
hongbotian [Tue, 18 Oct 2016 06:45:47 +0000 (23:45 -0700)]
update the committer email address

JIRA: DOVETAIL-36

Update the email addrees for Wenjing Chu

Change-Id: I61e378bba703806cef0cb829b855b822671ddbbe
Signed-off-by: hongbotian <hongbo.tianhongbo@huawei.com>
8 years agoGet envs from environment variables to test multi-SUT 65/23065/5
zshi [Mon, 17 Oct 2016 06:03:14 +0000 (14:03 +0800)]
Get envs from environment variables to test multi-SUT

JIRA: DOVETAIL-32

currently envs variable is hard-coded to specific values
which are not suitable for multi-SUT platform, fix it by
reading it from environment variable. this commit is for
apex installer testing purpose.

Change-Id: I2163b96208d42a91f2795de6a55b276b803dc5e8
Signed-off-by: zshi <zshi@redhat.com>
8 years agoMerge "[dovetail_tool] Fix the python style scanned by flake8"
hongbo tian [Mon, 17 Oct 2016 03:28:37 +0000 (03:28 +0000)]
Merge "[dovetail_tool] Fix the python style scanned by flake8"

8 years ago[dovetail_tool] Fix the python style scanned by flake8 51/23151/5
xudan [Fri, 14 Oct 2016 10:21:11 +0000 (10:21 +0000)]
[dovetail_tool] Fix the python style scanned by flake8

JIRA: DOVETAIL-34

Change-Id: Ib9a6dd254f1d77b0cbbfa66e8e1e44df1ce14a4b
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "Add documentation for dovetail tool"
hongbo tian [Fri, 14 Oct 2016 09:25:23 +0000 (09:25 +0000)]
Merge "Add documentation for dovetail tool"

8 years agodovetail tool: setup.py added and path error amend 59/23059/4
MatthewLi [Thu, 13 Oct 2016 03:51:50 +0000 (23:51 -0400)]
dovetail tool: setup.py added and path error amend

JIRA: DOVETAIL-26

1)ez_setup.py setup.py is added
2)flake8 is there, then format check can be easily achived in CI later
3)path error amend see details in DOVETAIL-33

Change-Id: Ic19dbac852694adaeecf5cdf6c2ae7c9ea9dbb33
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agopreparation for setup.py 15/23015/1
MatthewLi [Wed, 12 Oct 2016 09:17:29 +0000 (05:17 -0400)]
preparation for setup.py

JIRA: DOVETAIL-26

1)change the directory scripts to dovetail, to make it unified in OPNFV and for setup.py
2)change the paths accordingly
3)this should be first merged, since docker is changed, then to add setup.py and run the local test

Change-Id: I65974bde17f310ebbfe3c025532b05dae8752945
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
8 years agoMerge "Add yardstick crawler and checker for the results of testcase"
hongbo tian [Wed, 12 Oct 2016 09:08:19 +0000 (09:08 +0000)]
Merge "Add yardstick crawler and checker for the results of testcase"

8 years agoAdd documentation for dovetail tool 93/22893/8
Leo Wang [Mon, 10 Oct 2016 01:53:00 +0000 (21:53 -0400)]
Add documentation for dovetail tool

JIRA:DOVETAIL-28

Change-Id: I2478655d951a65a11f39e9ceba10969e56274cc3
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agoAdd yardstick crawler and checker for the results of testcase 95/22895/1
xudan [Mon, 10 Oct 2016 01:50:57 +0000 (01:50 +0000)]
Add yardstick crawler and checker for the results of testcase

JIRA: DOVETAIL-18

Change-Id: Idaad0ee4f8e6304f18fa478f1cded6e256cb12fd
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoRemove scripts in dovetail and use yarstick scripts to prepare environment 35/22835/4
xudan [Sat, 8 Oct 2016 08:42:56 +0000 (08:42 +0000)]
Remove scripts in dovetail and use yarstick scripts to prepare environment

JIRA: DOVETAIL-20

Change-Id: I46a095d5714245a9399e3a33c72f5f4bc77a18bf
Signed-off-by: xudan <xudan16@huawei.com>
8 years agoMerge "Use template to unify commands in functest/yardstick"
hongbo tian [Sat, 8 Oct 2016 02:43:57 +0000 (02:43 +0000)]
Merge "Use template to unify commands in functest/yardstick"

8 years agoUse template to unify commands in functest/yardstick 65/22365/12
Leo Wang [Fri, 23 Sep 2016 01:56:54 +0000 (21:56 -0400)]
Use template to unify commands in functest/yardstick

JIRA: DOVETAIL-19

1. use jinja2 to unify commands in config files
2. it simplify the process of test execution, put the dissimilarity in config
3. add precondition/postcondition for functest/yardstick config

Change-Id: Ib996b11ea065b61910b34b78191bb7b1ffd92e59
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agoMerge "IPv6 test cases: Test case specification document example"
hongbo tian [Wed, 28 Sep 2016 01:16:28 +0000 (01:16 +0000)]
Merge "IPv6 test cases: Test case specification document example"

8 years agoIPv6 test cases: Test case specification document example 19/20719/8
Bin Hu [Thu, 8 Sep 2016 22:27:26 +0000 (15:27 -0700)]
IPv6 test cases: Test case specification document example

JIRA: DOVETAIL-9

1. We are keeping the test methodology simple and easy.
   We are reusing existing tests as much as we can from
   IPv6 projects and Tempest API and Scenario tests, which
   are run in FuncTest and Yardstick.

2. We modified "testplan.rst" to briefly describe the test plan,
   i.e. current IPv6 test cases for platform compliance, which
   have passed FuncTest and Yardstick test in Arno and Brahmaputra
   releases. We also leave room for additional IPv6 test cases if
   there are more contributions down the road

3. Original "designspecification.rst", "testprocedure.rst" and
   "testspecification.rst" are not applicable any more here. The
   test design, setup, and execution procedure related specification
   are covered by each test case.

4. Thus "index.rst" only covers "testplan.rst" and the list of test cases.

5. Two examples of test case specification documents are provided
   "ipv6.tc001.specification.rst"
   "ipv6.tc026.specification.rst"

6. A complete set of current IPv6 test cases are listed in
   "ipv6_all_testcases.rst". Once template is agreed, we will converted
   it into specific test case document.

For more background of IPv6, and its features and technical details, please see:
   https://wiki.opnfv.org/display/ipv6/IPv6+Home
   http://artifacts.opnfv.org/ipv6/docs/installationprocedure/index.html

Change-Id: I7e47c73c0f02a407df278c93a3da2142b5cc86e5
Signed-off-by: Bin Hu <bh526r@att.com>
8 years agoMove copyfile to container and remove hard-code 51/22151/2
xudan2189 [Wed, 21 Sep 2016 08:40:44 +0000 (04:40 -0400)]
Move copyfile to container and remove hard-code

JIRA: DOVETAIL-17

Change-Id: Id4ad126169cf83c6c72a7b599f3309f0f930e461
Signed-off-by: xudan <xudan16@huawei.com>
8 years agosave dovetail report to disk 11/21811/3
Leo Wang [Tue, 20 Sep 2016 11:51:11 +0000 (07:51 -0400)]
save dovetail report to disk

JIRA: DOVETAIL-16

Change-Id: I8aab0b34425b698fee556ef97d3b0942b1b149d2
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agochange workdir of docker container for ci 09/21809/1
Leo Wang [Tue, 20 Sep 2016 11:26:40 +0000 (07:26 -0400)]
change workdir of docker container for ci

JIRA: DOVETAIL-15
 - change workdir to /home/opnfv/dovetail/scripts for ci

Change-Id: Ib289fb3e2e7af09a4a3a1fb7b124876155d811e7
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agoMerge "add prototype of dovetail tool"
Christopher Price [Tue, 20 Sep 2016 08:43:07 +0000 (08:43 +0000)]
Merge "add prototype of dovetail tool"

8 years agoadd prototype of dovetail tool 45/21345/12
Leo Wang [Sun, 18 Sep 2016 03:15:18 +0000 (23:15 -0400)]
add prototype of dovetail tool

JIRA: DOVETAIL-12

Change-Id: I1ee120ed8199589e1e7cbce9cbb55036e9e5f7ea
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agoadd dockerfile of dovetail tool 41/21341/3
Leo Wang [Wed, 14 Sep 2016 08:11:07 +0000 (04:11 -0400)]
add dockerfile of dovetail tool

JIRA:DOVETAIL-13
  - add dockerfile

Change-Id: I82d74013fef9b9203b531a1ba944e014d77a25cd
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
8 years agoProposing a Dovetail test suite structure for IPv6 51/18151/5
ChristopherPrice [Mon, 8 Aug 2016 14:07:05 +0000 (16:07 +0200)]
Proposing a Dovetail test suite structure for IPv6

Adding the first draft of the IPv6 test suite.
This patch attempts to start the documentation according to the
proposal on https://wiki.opnfv.org/pages/viewpage.action?pageId=6827269
The text is neither complete nor accurate at this stage, this patch
is intended to establish a concrete dialog for how we structure our
test suites and reference test cases.

Change-Id: I156ddafa3e8b7630390523864490426fc742eb3a
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>