From 3d012db86bbdfeb51ed5c6da52523fc2647dffb0 Mon Sep 17 00:00:00 2001 From: Morgan Richomme Date: Mon, 5 Sep 2016 11:44:41 +0200 Subject: [PATCH] Doc for Xreview by other test projects Change-Id: I75da3530685b74f628fc689b2ef8908919df147e Signed-off-by: Morgan Richomme (cherry picked from commit e7e579f3f03d2bdce513e6bd17da3aecfbca8612) --- docs/configguide/configguide.rst | 4 ++-- docs/devguide/index.rst | 30 ++++++++++++++++++++---------- docs/release-notes/functest-release.rst | 6 +++--- docs/userguide/index.rst | 4 ++-- docs/userguide/introduction.rst | 4 ++-- 5 files changed, 29 insertions(+), 19 deletions(-) diff --git a/docs/configguide/configguide.rst b/docs/configguide/configguide.rst index d79494399..4ef673c94 100644 --- a/docs/configguide/configguide.rst +++ b/docs/configguide/configguide.rst @@ -31,7 +31,7 @@ following docker command:: [functester@jumphost ~]$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE opnfv/functest latest 8cd6683c32ae 2 weeks ago 1.611 GB - opnfv/functest brahmaputra.3.0 94b78faa94f7 4 weeks ago 874.9 MB + opnfv/functest colorado.1.0 94b78faa94f7 4 weeks ago 874.9 MB hello-world latest 94df4f0ce8a4 7 weeks ago 967 B The Functest docker container environment can -in principle- be also @@ -119,7 +119,7 @@ recommended parameters for invoking docker container The credentials file needs to exist in the Docker container under the path: '/home/opnfv/functest/conf/openstack.creds'. - WARNING: If you are using the Joid installer, you must pass the + **WARNING:** If you are using the Joid installer, you must pass the credentials using the **-v** option: -v /var/lib/jenkins/admin-openrc:/home/opnfv/functest/conf/openstack.creds. See the section `Accessing the Openstack credentials`_ above. diff --git a/docs/devguide/index.rst b/docs/devguide/index.rst index 6b98437c6..cab9ec33c 100644 --- a/docs/devguide/index.rst +++ b/docs/devguide/index.rst @@ -144,6 +144,7 @@ the test results from CI: http://testresults.opnfv.org/test/swagger/spec.html + Authentication: opnfv/api@opnfv Any test project running on any lab integrated in CI can push the @@ -249,12 +250,16 @@ Results:: "build_tag": , "scenario": , "criteria": , - "trust_indicator": <0~1> + "trust_indicator": { + "current": 0, + "histories": [] + } } The API can described as follows. For detailed information, please go to http://testresults.opnfv.org/test/swagger/spec.html + Authentication: opnfv/api@opnfv Please notes that POST/DELETE/PUT operations for test or study purpose via @@ -475,6 +480,9 @@ Dashboard url: http://testresults.opnfv.org/kibana_dashboards/ Credentials for a guest account: opnfvuser/kibana +A script has been developped to build elasticsearch data set. This +script can be found in `[16]`_. + Automatic reporting =================== @@ -511,8 +519,14 @@ corresponding to this scenario. +---------------------+---------+---------+---------+---------+ | promise | | | X | X | +---------------------+---------+---------+---------+---------+ + | doctor | X | | X | | + +---------------------+---------+---------+---------+---------+ | security_scan | X | | | | +---------------------+---------+---------+---------+---------+ + | parser | | | X | | + +---------------------+---------+---------+---------+---------+ + | moon | | X | | | + +---------------------+---------+---------+---------+---------+ | copper | X | | | X | +---------------------+---------+---------+---------+---------+ @@ -520,7 +534,7 @@ All the testcases listed in the table are runnable on os-odl_l2-nofeature scenarios. If no result is available or if all the results are failed, the test case get 0 point. -If it was succesfull at least once but no anymore during the 4 runs, +If it was succesfull at least once but not anymore during the 4 runs, the case get 1 point (it worked once). If at least 3 of the last 4 runs were successful, the case get 2 points. If the last 4 runs of the test are successful, the test get 3 points. @@ -649,8 +663,6 @@ bgpvpn scenarios:: scenario: '(ocl)|(nosdn)|^(os-odl)((?!bgpvpn).)*$' - - How to write and check constaint regex? ======================================= @@ -914,15 +926,13 @@ _`[14]`: https://git.opnfv.org/cgit/releng/tree/jjb/functest/functest-daily.sh _`[15]`: https://git.opnfv.org/cgit/releng/tree/utils/test/result_collection_api/README.rst -OPNFV main site: opnfvmain_. +_`[16]`: https://git.opnfv.org/cgit/releng/tree/utils/test/scripts/mongo_to_elasticsearch.py -OPNFV functional test page: opnfvfunctest_. +OPNFV main site: http://www.opnfv.org -IRC support chan: #opnfv-functest +OPNFV functional test page: https://wiki.opnfv.org/opnfv_functional_testing -_opnfvmain: http://www.opnfv.org - -_opnfvfunctest: https://wiki.opnfv.org/opnfv_functional_testing +IRC support chan: #opnfv-functest _`OpenRC`: http://docs.openstack.org/user-guide/common/cli_set_environment_variables_using_openstack_rc.html diff --git a/docs/release-notes/functest-release.rst b/docs/release-notes/functest-release.rst index c05cc15cf..47efa917a 100644 --- a/docs/release-notes/functest-release.rst +++ b/docs/release-notes/functest-release.rst @@ -1,6 +1,6 @@ -============================================== -OPNFV Brahmaputra3.0 release note for Functest -============================================== +=========================================== +OPNFV Colorado1.0 release note for Functest +=========================================== Abstract ======== diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst index 6140f606d..30bee8dc0 100644 --- a/docs/userguide/index.rst +++ b/docs/userguide/index.rst @@ -16,7 +16,7 @@ The goal of this document is to describe the OPNFV Functest test cases and to provide a procedure to execute them. In the OPNFV Colorado system release, a Functest CLI utility is introduced for easier execution of test procedures. -A overview presentation has been created for the first OPNFV Summit `[4]`_. +An overview presentation has been created for the first OPNFV Summit `[4]`_. This document is a continuation of the OPNFV Functest Configuration Guide `[1]`_. @@ -360,7 +360,7 @@ The current work flow is as follows: * Jenkins Build Initiated * security_scan.py script is called, and a config file is passed to the script as an argument. - * The IP addresses of each NFVi node (compute / control), is gathered. + * The IP addresses of each NFVi node (compute / control) are gathered * A scan profile is matched to the node type. * The OpenSCAP application is remotely installed onto each target node gathered on step 3, using upstream packaging (rpm and .deb). diff --git a/docs/userguide/introduction.rst b/docs/userguide/introduction.rst index fc8e6680e..627f907f3 100644 --- a/docs/userguide/introduction.rst +++ b/docs/userguide/introduction.rst @@ -195,14 +195,14 @@ offers a lot of potential combinations: Most of the tests are runnable by any combination, but some tests might have restrictions imposed by the utilized installers or due to the available deployed features. The system uses the environment variables (INSTALLER_IP and -DEPLOY_SCENARIO) to automatically determine the valid test cases; for each given +DEPLOY_SCENARIO) to automatically determine the valid test cases, for each given environment. In the Colorado OPNFV System release a convenience Functest CLI utility is also introduced to simplify setting up the Functest evironment, management of the OpenStack environment (e.g. resource clean-up) and for executing tests. The Functest CLI organised the testcase into logical Tiers, which contain in -turn one or more testcases. The CLI allow execution of a single specified +turn one or more testcases. The CLI allows execution of a single specified testcase, all test cases in a specified Tier, or the special case of execution of **ALL** testcases. The Functest CLI is introduced in more detail in the section `Executing the functest suites`_ of this document. -- 2.16.6