Some developer documents are now old and suggested to be removed.
Change-Id: Ifa26a1535d57a560a0525eafa300ae087feb3191
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
+++ /dev/null
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
-
-================================
-Dovetail Command-line Interface
-================================
-
-
-Command Line Introduction
-==========================
-
-The Dovetail command-line interface provides a method for interacting with
-Dovetail from the console. For help on the ``dovetail`` command, enter:
-
-::
-
- dovetail --help
-
-**dovetail** optional arguments:
-
-::
-
- --version
- show program's version number and exit
- list <testsuite_name>
- list the testsuite details
- show <testcase_name>
- show the testcase details
- run <arguments>
- run the testcases
-
-For **dovetail list**, if the ``<testsuite_name>`` is omitted,
-all the testsuites defined under ``/dovetail/compliance`` directory
-and related testcases will be listed, otherwise,
-the test area and test cases defined in ``<testsuite_name>`` are listed.
-
-For **dovetail show**, the ``<testcase_name>`` is required, the contents defined
-in ``<testcase_name>.yml`` is shown.
-
-For **dovetail run**, by running ``dovetail run --help``, the ``dovetail run``
-usage is shown as:
-
-+------------------------+-----------------------------------------------------+
-|Options | |
-+========================+=====================================================+
-| -t, --SUT_TYPE |Installer type of the system under test (SUT). |
-+------------------------+-----------------------------------------------------+
-| --creds |Openstack credential file location |
-+------------------------+-----------------------------------------------------+
-| -i, --SUT_IP |IP of the system under test (SUT). |
-+------------------------+-----------------------------------------------------+
-| -d, --debug |True for showing debug log on screen. |
-+------------------------+-----------------------------------------------------+
-| -f, --func_tag |Overwrite tag for functest docker container (e.g. |
-| |stable or latest) |
-+------------------------+-----------------------------------------------------+
-| -y, --yard_tag |Overwrite tag for yardstick docker container (e.g. |
-| |stable or latest) |
-+------------------------+-----------------------------------------------------+
-| --testarea |compliance testarea within testsuite |
-+------------------------+-----------------------------------------------------+
-| --testsuite |compliance testsuite. |
-+------------------------+-----------------------------------------------------+
-| -h, --help |Show this message and exit. |
-+------------------------+-----------------------------------------------------+
-
-
-If no arguments are given, the default testsuite will be performed, i.e., the ``compliance_set``
-testsuite with default configurations.
-
-For more information about **dovetail** command-line interface, please refer to the wiki page [3]_
-
-Parameters definition with config file
-======================================
-
-The default **dovetail run** parameters can be modified in
-``dovetail/dovetail/conf/cmd_config.yml``, which is shown as:
-
-::
-
- cli:
- arguments:
- config:
- # This is a simple example of arguments.
- # Dovetail has no need of this kind of parameters currently.
- # The arguments must be given orderly at the run-time.
- #
- # docker_tag:
- # flags: 'docker_tag'
- # path:
- # - 'functest/docker_tag'
- # - 'yardstick/docker_tag'
- control:
-
- options:
- config:
- SUT_TYPE:
- flags:
- - '--SUT_TYPE'
- - '-t'
- path:
- - 'functest/envs'
- - 'yardstick/envs'
- help: 'Installer type of the system under test (SUT).'
- yard_tag:
- flags:
- - '--yard_tag'
- - '-y'
- path:
- - 'yardstick/docker_tag'
- help: 'Overwrite tag for yardstick docker container (e.g. stable or latest)'
- func_tag:
- flags:
- - '--func_tag'
- - '-f'
- path:
- - 'functest/docker_tag'
- help: 'Overwrite tag for functest docker container (e.g. stable or latest)'
- control:
- testsuite:
- flags:
- - '--testsuite'
- default: 'compliance_set'
- help: 'compliance testsuite.'
- testarea:
- flags:
- - '--testarea'
- default: 'full'
- help: 'compliance testarea within testsuite'
-
-The Click module is used to parse parameters defined in the above config file,
-two subsections are included in this file, ``arguments`` and ``options``,
-which corresponds to two types of parameters in Click.
-
-Arguments and Options
-+++++++++++++++++++++
-Only ``options`` is used currently, which means parameters can be given (or not) without
-sequence restriction.
-
-Config and control
-++++++++++++++++++
-
-All options/arguments are divided into two parts: **config** and **control**.
-The config ones are used for updating functest or yardstick config files according
-to the **path** given. For example, functest's config file is
-``dovetail/dovetail/conf/functest_config.yml``, following is a simple example:
-
-::
-
- docker_tag: latest
- envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
- -e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
- -e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal'
-
-If running with the command ``python run.py --SUT_TYPE fuel -f stable``, then
-the configs will be changed into
-
-::
-
- docker_tag: stable
- envs: '-e INSTALLER_TYPE=fuel -e INSTALLER_IP=192.168.200.2
- -e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
- -e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal'
-
-The config options/arguments can be added or deleted by modifying
-``cmd_config.yml`` rather than changing the source code. However, for control
-command, besides adding it into ``cmd_config.yml``, some other operations about
-the source code are also needed.
-
-
-. [3] https://wiki.opnfv.org/display/dovetail/Dovetail+Command+Line
+++ /dev/null
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
-
-===========================
-Dovetail Tool Installation
-===========================
-
-Abstract
-########
-
-The Dovetail tool supports installation either directly on the Linux host, or within a Docker container.
-The detailed installation procedure on the Linux host or via the Docker container are shown
-in the following sections.
-
-There is a need to config the following parameters before Dovetail tool
-running on the SUT (System Under Test):
-
-::
-
- SUT_TYPE
- SUT type, e.g., apex, compass, fuel, joid, etc
- SUT_IP
- SUT external network IP, e.g., 192.168.200.2
- NODE_NAME
- this can be shown in the test result for users to see which pod the dovetail tool runs
- DEPLOY_SCENARIO
- deployment scenario, e.g., os-nosdn-nofeature-ha
- BUILD_TAG
- this can be shown in the test result for users to identify logs
- CI_DEBUG
- true for debug information printed and false for not printed
- DEPLOY_TYPE
- baremetal or virtual
-
-The above configuration can be achieved by
-
-- modifying the environment variables in files which live under ``/dovetail/conf/`` directory
-- set and use Linux environment variables using ``export`` command
-- set and use these variables when using ``dovetail run`` command line, for details see the
- `Dovetail Command-line Interface`_ section
-- enable the OpenStack credential file, which can be achieved by using
- ``dovetail run --creds </path/creds>``
-
-
-Dovetail tool installation on local Linux host environment
-##########################################################
-
-In order to install Dovetail into a local Linux host environment, the following steps should
-be performed:
-
-Downloading Dovetail tool
---------------------------
-
-Source code downloading:
-
-::
-
- git clone https://git.opnfv.org/dovetail
-
-Environment preparation
------------------------
-
-Dovetail configuration:
-
-::
-
- cd {dovetail_path}/dovetail/conf
- vim functest_config.yml
- vim yardstick_config.yml
-
-To modify the ``envs`` parameters, e.g., SUT_TYPE, SUT_IP, NODE_NAME,
-DEPLOY_SCENARIO, BUILD_TAG, CI_DEBUG, DEPLOY_TYPE, in the yml files.
-
-System dependencies and project configuration information are listed in
-``requirements.txt``, ``setup.py`` and ``setup.cfg``, in Ubuntu environment,
-to run
-
-::
-
- apt-get -y install python-pip
- pip install -e .
-
-in ``centos`` and ``redhat`` environment, to run
-
-::
-
- yum -y install python-pip
- pip install -e .
-
-in ``fedora`` environment, to run
-
-
-::
-
- dnf -y install python-pip
- pip install -e .
-
-then the setup requirements are installed and **dovetail** commmand-line can be used,
-by using ``dovetail --help``, **dovetail** command-line details are shown.
-
-The tool ``tox`` is used, which configuration information is defined in
-``tox.ini``, unitttest, flake8-checking, coverage check, etc, can be done
-by running:
-
-::
-
- pip install tox
- tox
-
-Compliance and certification test cases
-----------------------------------------
-
-The compliance and certification test cases can be defined under the ``/dovetail/compliance``
-directory, which is defined in yaml format.
-A sample file named ``compliance_set.yml`` is provided as follows:
-
-::
-
- compliance_set:
- name: compliance_set
- testcases_list:
- - dovetail.ipv6.tc001
-
-The testcase listed here is dovetail.ipv6.tc001, defined within ``dovetail/testcase``.
-
-Running Dovetail tool
----------------------
-
-After environment preparation is complete and test cases added, the Dovetail tool can be run with
-
-::
-
- dovetail run --testsuite compliance_set
-
-The value ``compliance_set`` passed to the ``testsuite`` flag can be replaced
-with the testsuite yaml file name which want to be run.
-If no argument is given, the compliance_set testsuite will be run as the default.
-
-Moreover, the testcases in given testarea can be run with ``testarea`` command line argument, such as
-testarea ``ipv6`` in ``compliance_set``
-
-::
-
- dovetail run --testsuite compliance_set --testarea ipv6
-
-Dovetail provides some sets, ``debug``, ``proposed_tests`` and ``compliance_set``,
-``debug`` is used for locally and Continuous Integration(CI) developing purpose,
-which provides typical testcase examples, feel free to edit it when develops locally, such as
-only to run a testcase which only takes minutes. ``proposed_tests`` is the testcase
-candidate which mainly comes from the wiki link [1]_.
-``compliance_set`` is used for compliance. Moreover, dovetail tool can be easily
-extended to support more complicated compliance requirements,
-such as feature set based or scenario based compliance.
-
-If you want to run the ``debug`` testsuite, just run with
-
-::
-
- dovetail run --testsuite debug
-
-Running Dovetail in a Docker container
-########################################
-
-The Dovetail tool can be run in a Docker container by utilizing the following steps:
-
-Pull Dovetail Docker image from public Dockerhub
-------------------------------------------------
-
-::
-
- sudo docker pull opnfv/dovetail:<Tag>
-
-<Tag> here is the version, 'latest' is used for the master branch.
-
-(Optional) Create Docker image from Dockerfile
------------------------------------------------
-Instead of an official Docker image, you can build your own Docker image from
-Dockerfile(s) located under the ``docker/`` directory. The ``Dockerfile``
-is based on Ubuntu and the ``Dockerfile.centos7`` file is for a CentOS based
-Docker image.
-
-::
-
- cd {dovetail_path}/dovetail/docker
- docker build --no-cache -t opnfv/dovetail:<Tag> --build-arg BRANCH=master .
-
-Dovetail Docker container creation
-----------------------------------
-
-Next, create the ``dovetail-docker-env`` file to define the environment parameters ::
-
- INSTALLER_TYPE=compass
- INSTALLER_IP=192.168.200.2
- DEPLOY_TYPE=baremetal
- DEPLOY_SCENARIO=ha-nosdn
- CI_DEBUG=true
-
-or if an OpenStack credential file is provided.
-
-Then to instantiate the Dovetail Docker container, execute::
-
- sudo docker run --privileged=true --rm -t \
- --env-file dovetail-docker-env OR </path/creds> \
- -v /home/opnfv/dovetail/results:/home/opnfv/dovetail/results \
- -v /var/run/docker.sock:/var/run/docker.sock \
- --name <Dovetail_Container_Name> (optional) \
- opnfv/dovetail:<Tag> /bin/bash
-
-To attach dovetail container and Running test cases
-----------------------------------------------------
-
-Before connecting to the container, you can check the container status by running ::
-
- sudo docker ps -a
-
-Attach to the container by starting it and obtaining a bash prompt with ::
-
- sudo docker exec -it <Dovetail_Container_Name>/<Container_Id> bash
-
-Inside the container the following commands can be executed to trigger the testing ::
-
- dovetail run --testsuite compliance_set
-
-Offline Support
-################
-
-There are some SUTs that are isolated from the public internet,
-so offline support is needed. The idea is to provide all of the packages of dovetail
-release in http://artifacts.opnfv.org, then the user can download and transfer to their inner
-development environment.
-
-The packages are shown in [2]_
-
-TO DO: to introduce more when it is mature enough.
-
-Results Output
-###############
-
-The running log is stored in ``/home/opnfv/dovetail/results/dovetail.log``.
-The compliance report is stored in ``/home/opnfv/dovetail/results/dovetail_report.txt``.
-
-Dovetail Version and Release
-############################
-
-Dovetail version information is defined in ``setup.cfg``.
-At the time of release, it is the dovetail team's responsibility to set
-the ``version`` value in ``setup.cfg``.
-
-
-.. [1] https://wiki.opnfv.org/display/dovetail/Dovetail+Test+Areas+and+Test+Cases.
-.. [2] http://artifacts.opnfv.org/dovetail.html.
+++ /dev/null
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
-
-==============
-Dovetail Tool
-==============
-
-What is Dovetail tool
-#####################
-
-A toolset for providing an evaluation of a deployed cloud environment for OPNFV compliance.
-
-Overview
-########
-
-Dovetail tool intends on being THE source of tools to provide evaluation of the use of OPNFV trademarks.
-
-Dovetail tool provides users with a wrapper which leverage other test projects, such as Functest,
-Yardstick, etc, that helps to provide evaluation of the use of OPNFV trademarks.
-It does so by validating any NFV(Network Function Virtualization) implementation against the test cases in Dovetail.
-
-Get involved!
-#############
-- Wiki: https://wiki.opnfv.org/display/dovetail/Dovetail+Home
-- Git Repo: https://git.opnfv.org/dovetail
+++ /dev/null
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
-
-==================
-Template Syntax
-==================
-
-Testcase Template Syntax
-=========================
-
-The testcases used for compliance and certification are defined in the
-``dovetail/testcase`` directory, which are written in yaml format.
-Take the testcase ``ipv6.tc001.yml`` as an example. It is shown as:
-
-::
-
- ---
- dovetail.ipv6.tc001:
- name: dovetail.ipv6.tc001
- objective: Bulk creation and deletion of IPv6 networks, ports and subnets
- validate:
- type: functest
- testcase: tempest_smoke_serial
- pre_condition:
- - 'echo test for precondition in testcase'
- cmds:
- - 'functest env prepare'
- - 'functest testcase run {{validate_testcase}}'
- post_condition:
- - 'echo test for precondition in testcase'
- report:
- sub_testcase_list:
- - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
- - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
- - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
-
-
-- At least three sections named ``name``, ``objective``, ``validate``,
- ``report`` must be included
-- Section ``name`` distinguishes different test cases used for compliance,
- it is composed of 3 parts, ``dovetail.``, belongs to which test area,
- and the serial number
-- Section ``objective`` briefly describes what this testcase does
-- Section ``validate`` defines the scripts and configurations for the
- validation of the test case. ``type`` defines which method is used to validate,
- 3 ways, i.e., functest, yardstick and shell is supported currently.
- ``testcase`` represents the testcases in slicing/tier.
-- Section ``report`` defines the sub_testcases to be run.
- For ``yardstick``, since it is not sliced by now,
- ``sub_testcase_list`` is not needed, only to edit the ``testcase`` part in
- section ``validate``, such as ``yardstick_tc027``
-
-
-Config Template Syntax
-=======================
-
-For Dovetail tool, the config files are located in ``dovetail/dovetail/conf``, which are written
-in yaml format. As both functest and yardstick are utilized by Dovetail, their configuration files
-should be configured as follows, within the ``functest_config.yml`` and ``yardstick_config.yml`` files,
-respectively.
-
-Functest config template syntax
--------------------------------
-
-An example of ``functest`` configuration is shown as follows:
-
-::
-
- functest:
- image_name: opnfv/functest
- docker_tag: latest
- envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
- -e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
- -e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal'
- opts: '-id --privileged=true'
- pre_condition:
- cmds:
- - 'echo test for precondition'
- testcase:
- cmds:
- - 'python /home/opnfv/repos/functest/ci/prepare_env.py start'
- - 'python /home/opnfv/repos/functest/ci/run_tests.py -t {{script_testcase}} -r'
- post_condition:
- cmds:
- - ''
- result:
- dir: '/home/opnfv/functest/results'
- store_type: 'file'
- file_path: 'tempest/tempest.log'
- db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
-
-- ``image_name`` and ``docker_tag`` sections define the docker image pulled from the public dockerhub.
-- ``envs`` should be correctly configed according to the SUT(System Under Test).
-- ``pre_condition`` represents some cleanups or preparations.
- ``testcase`` represents the testcases running cmds.
- ``post_condition`` represents some cleanups needed after all testcases finished.
-- ``result`` section gives the directory of the dovetail tool test result.
- ``db_url`` gives the database URL of the dovetail results to be stored.
-
-Yardstick config template syntax
----------------------------------
-
-The configuration in ``yardstick_config.yml`` is similiar to ``functest_config.yml``,
-and an example is shown as follows:
-
-::
-
- yardstick:
- image_name: opnfv/yardstick
- docker_tag: latest
- envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
- -e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
- -e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal
- -e EXTERNAL_NETWORK=ext-net'
- opts: '-id --privileged=true'
- pre_condition:
- cmds:
- - 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh &&
- source /home/opnfv/repos/yardstick/tests/ci/clean_images.sh && cleanup'
- - 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh &&
- cd /home/opnfv/repos/yardstick && source tests/ci/load_images.sh'
- testcase:
- cmds:
- - 'mkdir -p /home/opnfv/yardstick/results/'
- - 'cd /home/opnfv/repos/yardstick && source tests/ci/prepare_env.sh &&
- yardstick task start tests/opnfv/test_cases/{{script_testcase}}.yaml
- --output-file /home/opnfv/yardstick/results/{{script_testcase}}.out &>
- /home/opnfv/yardstick/results/yardstick.log'
- post_condition:
- cmds:
- - ''
- result:
- dir: '/home/opnfv/yardstick/results'
- store_type: 'file'
- file_path: 'yardstick.log'
- db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
-
-The main differences between ``yardstick_config.yml`` and ``functest_config.yml``
-are the ``cmds`` subsection.
-
-Jinja2 template syntax
-----------------------
-
-Jinja2 module can be used to config the ``{{script_testcase}}``. The
-parameters are defined in ``dovetail/dovetail/conf/dovetail_config.yml``:
-
-::
-
- parameters:
- - name: testcase
- path: '("name",)'
- - name: script_testcase
- path: '("scripts", "testcase")'
-
-Here ``path`` is the path defined in the testcase configuration files.
-Take ``script_testcase`` as an example. For testcase ``dovetail.ipv6.tc001``:
-
-::
-
- dovetail.ipv6.tc001:
- name: dovetail.ipv6.tc001
- objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation
- scripts:
- type: functest
- testcase: tempest_smoke_serial
- sub_testcase_list:
- - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
- - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
- - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
-
-The path ("scripts", "testcase") means 'testcase' is the subsection of 'scripts' section. From above,
-by following the path ("scripts", "testcase") we can get the value of ``{{script_testcase}}`` is 'tempest_smoke_serial'.
+++ /dev/null
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
-
-==================
-Dovetail Overview
-==================
-
-.. toctree::
- :maxdepth: 2
-
- dovetail.tool.overview.rst
- dovetail.tool.installation.rst
- dovetail.tool.template.rst
- dovetail.tool.cli.rst
Code Review / dovetail.git / commitdiff