X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=blobdiff_plain;f=docs%2Ftesting%2Fuser%2Fconfigguide%2Fconfigguide.rst;h=07f0a16810dbda17785f62f44261ab45570eca7e;hb=cf4e1c1b506adb0461ca308b4ad53ec98e8d631f;hp=9adf79e143fcf3e03f2ec5103b36de978e538692;hpb=fb85975fee7c3c67d3b10df58e4b742034fbd2ab;p=functest.git diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index 9adf79e14..07f0a1681 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -1,4 +1,3 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. .. SPDX-License-Identifier: CC-BY-4.0 Installation and configuration @@ -10,37 +9,27 @@ the refactoring on dependency management should allow the creation of light and fully customized docker images. -Functest Dockers ----------------- +Functest Dockers for OpenStack deployment +----------------------------------------- Docker images are available on the dockerhub: * opnfv/functest-core * opnfv/functest-healthcheck * opnfv/functest-smoke + * opnfv/functest-benchmarking * opnfv/functest-features * opnfv/functest-components * opnfv/functest-vnf - * opnfv/functest-parser - * opnfv/functest-restapi - -The tag "opnfv-5.0.0" is the official release image in Euphrates, but you can also pull "euphrates" -tag as it is being maintained by Functest team and might include bugfixes. - -The Functest docker container environment can -in principle- be also -used with non-OPNFV official installers (e.g. 'devstack'), with the -**disclaimer** that support for such environments is outside of the -scope and responsibility of the OPNFV project. Preparing your environment --------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^ cat env:: - INSTALLER_TYPE=XXX - INSTALLER_IP=XXX EXTERNAL_NETWORK=XXX - DEPLOY_SCENARIO=XXX + DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario + NAMESERVER=XXX # if not 8.8.8.8 See section on environment variables for details. @@ -50,37 +39,31 @@ cat env_file:: export OS_USER_DOMAIN_NAME=XXX export OS_PROJECT_DOMAIN_NAME=XXX export OS_USERNAME=XXX - export OS_TENANT_NAME=XXX export OS_PROJECT_NAME=XXX export OS_PASSWORD=XXX - export OS_VOLUME_API_VERSION=XXX - export OS_IDENTITY_API_VERSION=XXX - export OS_IMAGE_API_VERSION=XXX + export OS_IDENTITY_API_VERSION=3 See section on OpenStack credentials for details. Create a directory for the different images (attached as a Docker volume):: - mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/euphrates | bash -s -- images && ls -1 images/* + mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/fraser | bash -s -- images && ls -1 images/* images/CentOS-7-aarch64-GenericCloud.qcow2 images/CentOS-7-aarch64-GenericCloud.qcow2.xz images/CentOS-7-x86_64-GenericCloud.qcow2 images/cirros-0.4.0-x86_64-disk.img images/cirros-0.4.0-x86_64-lxc.tar.gz - images/cirros-d161201-aarch64-disk.img - images/cirros-d161201-aarch64-initramfs - images/cirros-d161201-aarch64-kernel images/cloudify-manager-premium-4.0.1.qcow2 - images/img - images/trusty-server-cloudimg-amd64-disk1.img + images/shaker-image-arm64.qcow2 + images/shaker-image.qcow images/ubuntu-14.04-server-cloudimg-amd64-disk1.img images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img images/ubuntu-16.04-server-cloudimg-amd64-disk1.img images/vyos-1.1.7.img Testing healthcheck suite --------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^ Run healthcheck suite:: @@ -94,14 +77,25 @@ Results shall be displayed as follows:: +----------------------------+------------------+---------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +----------------------------+------------------+---------------------+------------------+----------------+ - | connection_check | functest | healthcheck | 00:06 | PASS | - | api_check | functest | healthcheck | 06:52 | PASS | - | snaps_health_check | functest | healthcheck | 00:42 | PASS | + | connection_check | functest | healthcheck | 00:09 | PASS | + | tenantnetwork1 | functest | healthcheck | 00:14 | PASS | + | tenantnetwork2 | functest | healthcheck | 00:11 | PASS | + | vmready1 | functest | healthcheck | 00:19 | PASS | + | vmready2 | functest | healthcheck | 00:16 | PASS | + | singlevm1 | functest | healthcheck | 00:41 | PASS | + | singlevm2 | functest | healthcheck | 00:36 | PASS | + | vping_ssh | functest | healthcheck | 00:46 | PASS | + | vping_userdata | functest | healthcheck | 00:41 | PASS | + | cinder_test | functest | healthcheck | 01:18 | PASS | + | api_check | functest | healthcheck | 10:33 | PASS | + | snaps_health_check | functest | healthcheck | 00:44 | PASS | + | odl | functest | healthcheck | 00:00 | SKIP | +----------------------------+------------------+---------------------+------------------+----------------+ - NOTE: the duration is a reference and it might vary depending on your SUT. + +NOTE: the duration is a reference and it might vary depending on your SUT. Testing smoke suite -------------------- +^^^^^^^^^^^^^^^^^^^ Run smoke suite:: @@ -112,23 +106,48 @@ Run smoke suite:: Results shall be displayed as follows:: - +------------------------------+------------------+---------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +------------------------------+------------------+---------------+------------------+----------------+ - | vping_ssh | functest | smoke | 00:45 | PASS | - | vping_userdata | functest | smoke | 00:36 | PASS | - | tempest_smoke_serial | functest | smoke | 14:04 | PASS | - | rally_sanity | functest | smoke | 23:59 | PASS | - | refstack_defcore | functest | smoke | 14:37 | FAIL | - | snaps_smoke | functest | smoke | 42:09 | PASS | - | odl | functest | smoke | 00:00 | SKIP | - | odl_netvirt | functest | smoke | 00:00 | SKIP | - +------------------------------+------------------+---------------+------------------+----------------+ - Note: if the scenario does not support some tests, they are indicated as SKIP. - See User guide for details. + +------------------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +------------------------------------+------------------+---------------+------------------+----------------+ + | tempest_smoke | functest | smoke | 04:24 | PASS | + | neutron-tempest-plugin-api | functest | smoke | 10:43 | PASS | + | rally_sanity | functest | smoke | 23:16 | PASS | + | refstack_defcore | functest | smoke | 06:30 | PASS | + | patrole | functest | smoke | 02:08 | PASS | + | snaps_smoke | functest | smoke | 00:00 | SKIP | + | neutron_trunk | functest | smoke | 02:09 | PASS | + | networking-bgpvpn | functest | smoke | 00:00 | SKIP | + | networking-sfc | functest | smoke | 00:00 | SKIP | + | barbican | functest | smoke | 00:00 | SKIP | + +------------------------------------+------------------+---------------+------------------+----------------+ + +Note: if the scenario does not support some tests, they are indicated as SKIP. +See User guide for details. + +Testing benchmarking suite +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run benchmarking suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-benchmarking + +Results shall be displayed as follows:: + + +-------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------+------------------+----------------------+------------------+----------------+ + | vmtp | functest | benchmarking | 18:43 | PASS | + | shaker | functest | benchmarking | 29:45 | PASS | + +-------------------+------------------+----------------------+------------------+----------------+ + +Note: if the scenario does not support some tests, they are indicated as SKIP. +See User guide for details. Testing features suite ----------------------- +^^^^^^^^^^^^^^^^^^^^^^ Run features suite:: @@ -147,12 +166,15 @@ Results shall be displayed as follows:: | functest-odl-sfc | sfc | features | 00:00 | SKIP | | barometercollectd | barometer | features | 00:00 | SKIP | | fds | fastdatastacks | features | 00:00 | SKIP | + | vgpu | functest | features | 00:00 | SKIP | + | stor4nfv_os | stor4nfv | features | 00:00 | SKIP | +-----------------------------+------------------------+------------------+------------------+----------------+ - Note: if the scenario does not support some tests, they are indicated as SKIP. - See User guide for details. + +Note: if the scenario does not support some tests, they are indicated as SKIP. +See User guide for details. Testing components suite ------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^ Run components suite:: @@ -163,47 +185,121 @@ Run components suite:: Results shall be displayed as follows:: - +-------------------------------+------------------+--------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +-------------------------------+------------------+--------------------+------------------+----------------+ - | tempest_full_parallel | functest | components | 52:42 | PASS | - | rally_full | functest | components | 114:22 | PASS | - +-------------------------------+------------------+--------------------+------------------+----------------+ + +--------------------------+------------------+--------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------------+------------------+--------------------+------------------+----------------+ + | tempest_full | functest | components | 53:40 | FAIL | + | tempest_scenario | functest | components | 18:50 | PASS | + | rally_full | functest | components | 167:13 | PASS | + +--------------------------+------------------+--------------------+------------------+----------------+ Testing vnf suite ------------------ +^^^^^^^^^^^^^^^^^ Run vnf suite:: -sudo docker run --env-file env \ - -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ - -v $(pwd)/images:/home/opnfv/functest/images \ - opnfv/functest-vnf + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-vnf Results shall be displayed as follows:: - +---------------------------------+-------------------+--------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +---------------------------------+-------------------+--------------+------------------+----------------+ - | cloudify_ims | functest | vnf | 28:49 | FAIL | - | vyos_vrouter | functest | vnf | 27:57 | FAIL | - | juju_epc | functest | vnf | 55:03 | PASS | - | orchestra_openims | orchestra | vnf | 00:00 | SKIP | - | orchestra_clearwaterims | orchestra | vnf | 00:00 | SKIP | - +---------------------------------+-------------------+--------------+------------------+----------------+ + +----------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +----------------------+------------------+--------------+------------------+----------------+ + | cloudify | functest | vnf | 04:05 | PASS | + | cloudify_ims | functest | vnf | 24:07 | PASS | + | heat_ims | functest | vnf | 18:15 | PASS | + | vyos_vrouter | functest | vnf | 15:48 | PASS | + | juju_epc | functest | vnf | 29:38 | PASS | + +----------------------+------------------+--------------+------------------+----------------+ + +Functest Dockers for Kubernetes deployment +------------------------------------------ +Docker images are available on the dockerhub: + + * opnfv/functest-kubernetes-core + * opnfv/functest-kubernetest-healthcheck + * opnfv/functest-kubernetest-smoke + * opnfv/functest-kubernetest-features + +Preparing your environment +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +cat env:: + + DEPLOY_SCENARIO=k8s-XXX + +Testing healthcheck suite +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run healthcheck suite:: + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-healthcheck + +A config file in the current dir 'config' is also required, which should be +volume mapped to ~/.kube/config inside kubernetes container. + +Results shall be displayed as follows:: + + +-------------------+------------------+---------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------+------------------+---------------------+------------------+----------------+ + | k8s_smoke | functest | healthcheck | 02:27 | PASS | + +-------------------+------------------+---------------------+------------------+----------------+ + +Testing smoke suite +^^^^^^^^^^^^^^^^^^^ + +Run smoke suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-smoke + +Results shall be displayed as follows:: + + +-------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------+------------------+---------------+------------------+----------------+ + | k8s_conformance | functest | smoke | 57:14 | PASS | + +-------------------------+------------------+---------------+------------------+----------------+ + +Testing features suite +^^^^^^^^^^^^^^^^^^^^^^ + +Run features suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-features + +Results shall be displayed as follows:: + + +----------------------+------------------+------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +----------------------+------------------+------------------+------------------+----------------+ + | stor4nfv_k8s | stor4nfv | stor4nfv | 00:00 | SKIP | + | clover_k8s | clover | clover | 00:00 | SKIP | + +----------------------+------------------+------------------+------------------+----------------+ Environment variables ===================== Several environement variables may be specified: - * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid) + * INSTALLER_IP= * DEPLOY_SCENARIO=--- + * NAMESERVER=XXX # if not 8.8.8.8 + * VOLUME_DEVICE_NAME=XXX # if not vdb + * EXTERNAL_NETWORK=XXX # if not first network with router:external=True + * NEW_USER_ROLE=XXX # if not member - -INSTALLER IP may be required by some test cases like SFC or Barometer in order -to access the installer node and the deployment. +INSTALLER_IP is required by Barometer in order to access the installer node and +the deployment. The format for the DEPLOY_SCENARIO env variable can be described as follows: * vim: (os|k8s) = OpenStack or Kubernetes @@ -220,8 +316,8 @@ release note. **NOTE:** The scenario name is mainly used to automatically detect if a test suite is runnable or not (e.g. it will prevent ODL test suite to be -run on 'nosdn' scenarios). If not set, Functest will try to run the default test -cases that might not include SDN controller or a specific feature +run on 'nosdn' scenarios). If not set, Functest will try to run the default +test cases that might not include SDN controller or a specific feature. **NOTE:** An HA scenario means that 3 OpenStack controller nodes are deployed. It does not necessarily mean that the whole system is HA. See @@ -233,8 +329,9 @@ to the Functest Docker Container, using the -e only relevant to Jenkins CI invoked testing and **should not be used** when performing manual test scenarios: - * NODE_NAME = - * BUILD_TAG = + * INSTALLER_TYPE=(apex|compass|daisy|fuel) + * NODE_NAME= + * BUILD_TAG= where: @@ -310,194 +407,6 @@ be:: export OS_INSECURE=true -Functest docker container directory structure -============================================= -Inside the Functest docker container, the following directory structure -should now be in place:: - - `-- - |- home - | |-- opnfv - | | `- functest - | | |-- conf - | | `-- results - | `-- repos - | `-- vnfs - |- src - | |-- tempest - | |-- vims-test - | |-- odl_test - | `-- fds - `- usr - `- lib - `- python2.7 - `- site-packages - `- functest - |-- ... - -Underneath the '/home/opnfv/functest' directory, the Functest docker container -includes two main directories: - - * The **conf** directory stores configuration files (e.g. the - OpenStack creds are stored in path '/home/opnfv/functest/conf/env_file'), - * the **results** directory stores some temporary result log files - -src and repos directories are used to host third party code used for the tests. - -The structure of functest repo can be described as follows:: - - |-- INFO - |-- LICENSE - |-- api - | `-- apidoc - |-- build.sh - |-- commons - |-- docker - | |-- components - | |-- core - | |-- features - | |-- healthcheck - | |-- smoke - | |-- vnf - | |-- parser - | |-- restapi - | |-- thirdparty-requirements.txt - |-- docs - | |-- com - | |-- images - | |-- release - | | `-- release-notes - | |-- results - | | testing - | | |-- developer - | | `-- user - | | |-- configguide - | | `-- userguide - `-- functest - |-- api - | |-- base.py - | |-- server.py - | |-- urls.py - | |-- common - | | |-- api_utils.py - | | |-- thread.py - | `-- resources - | `-- v1 - | |-- creds.py - | |-- envs.py - | |-- testcases.py - | |-- tiers.py - | |-- tasks.py - | `-- database - | |-- db.py - | `-- v1 - | |-- handlers.py - | |-- models.py - | `-- swagger - |-- ci - │ |-- check_deployment.py - │ |-- config_aarch64_patch.yaml - │ |-- config_functest.yaml - │ |-- config_patch.yaml - │ |-- download_images.sh - │ |-- logging.ini - │ |-- rally_aarch64_patch.conf - │ |-- run_tests.py - │ |-- testcases.yaml - │ |-- tier_builder.py - │ |-- tier_handler.py - |-- cli - │ |-- cli_base.py - │ |-- commands - │ │ |-- cli_env.py - │ │ |-- cli_os.py - │ │ |-- cli_testcase.py - │ │ |-- cli_tier.py - │ |-- functest-complete.sh - |-- core - │ |-- feature.py - │ |-- robotframework.py - │ |-- testcase.py - │ |-- unit.py - │ |-- vnf.py - |-- energy - │ |-- energy.py - |-- opnfv_tests - │ `-- openstack - │ |-- rally - │ |-- refstack_client - │ |-- snaps - │ |-- tempest - │ |-- vping - │ `-- sdn - │ │ `-- odl - │ `-- vnf - │ |-- ims - │ `-- router - |-- tests - │ `-- unit - │ |-- ci - │ |-- cli - │ |-- core - │ |-- energy - │ |-- features - │ |-- odl - │ |-- openstack - │ |-- opnfv_tests - │ |-- test_utils.py - │ |-- utils - │ `-- vnf - |-- utils - | |-- config.py - | |-- constants.py - | |-- decorators.py - | |-- env.py - | |-- functest_utils.py - | |-- functest_vacation.py - | |-- openstack_clean.py - | |-- openstack_tacker.py - | `-- openstack_utils.py - |-- requirements.txt - |-- setup.cfg - |-- setup.py - |-- test-requirements.txt - |-- tox.ini - |-- upper-constraints.txt - - (Note: All *.pyc files removed from above list for brevity...) - -We may distinguish several directories, the first level has 5 directories: - -* **api**: This directory is dedicated to the API (framework) documentations. -* **commons**: This directory is dedicated for storage of traffic profile or - any other test inputs that could be reused by any test project. -* **docker**: This directory includes the needed files and tools to - build the Functest Docker images. -* **docs**: This directory includes documentation: Release Notes, - User Guide, Configuration Guide and Developer Guide. -* **functest**: This directory contains all the code needed to run - functest internal cases and OPNFV onboarded feature or VNF test cases. - -Functest directory has 7 sub-directories, which is located under -/usr/lib/python2.7/site-packages/functest: - * **api**: This directory is dedicated for the internal Functest API. - * **ci**: This directory contains test structure definition files - (e.g .yaml) and bash shell/python scripts used to - configure and execute Functional tests. The test execution script - can be executed under the control of Jenkins CI jobs. - * **cli**: This directory holds the python based Functest CLI utility - source code, which is based on the Python 'click' framework. - * **core**: This directory holds the python based Functest core - source code. Three abstraction classes have been created to ease - the integration of internal, feature or vnf cases. - * **opnfv_tests**: This directory includes the scripts required by - Functest internal test cases and other feature projects test cases. - * **tests**: This directory includes the functest unit tests. - * **utils**: this directory holds Python source code for some general - purpose helper utilities, which testers can also re-use in their - own test code. See for an example the Openstack helper utility: - 'openstack_utils.py'. - Logs ==== @@ -505,7 +414,7 @@ By default all the logs are put un /home/opnfv/functest/results/functest.log. If you want to have more logs in console, you may edit the logging.ini file manually. Connect on the docker then edit the file located in -/usr/lib/python2.7/site-packages/functest/ci/logging.ini +/usr/lib/python2.7/site-packages/xtesting/ci/logging.ini Change wconsole to console in the desired module to get more traces. @@ -515,6 +424,7 @@ Configuration You may also directly modify the python code or the configuration file (e.g. testcases.yaml used to declare test constraints) under +/usr/lib/python2.7/site-packages/xtesting and /usr/lib/python2.7/site-packages/functest @@ -607,6 +517,8 @@ If the OpenStack command still does not show anything or complains about connectivity issues, it could be due to an incorrect url given to the OS_AUTH_URL environment variable. Check the deployment settings. +.. _`Proxy support`: + Proxy support ------------- If your Jumphost node is operating behind a http proxy, then there are @@ -616,7 +528,7 @@ succeed: #. Initial installation of docker engine First, try following the official Docker documentation for Proxy settings. Some issues were experienced on CentOS 7 based Jumphost. Some tips are documented - in section: `Docker Installation on CentOS behind http proxy`_ + in section: :ref:`Docker Installation on CentOS behind http proxy` below. If that is the case, make sure the resolv.conf and the needed @@ -679,6 +591,8 @@ commands might not work. You can use the **curl** command instead. (Ignore the content. If command returns a valid HTML page, it proves the connection.) +.. _`Docker Installation on CentOS behind http proxy`: + Docker Installation on CentOS behind http proxy ----------------------------------------------- This section is applicable for CentOS family OS on Jumphost which