X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=blobdiff_plain;f=docs%2Ftesting%2Fuser%2Fconfigguide%2Fconfigguide.rst;h=25aa7cd2694aceb8d2beb8e86fdd9f664de37f61;hb=37251af62ca89b55f09dfd6ddce15ee179c580f3;hp=c4d9a6d7741888ff7e72b68a7df61ff2ae2daa46;hpb=e3299f119c93a57e00d1aaeeb1f7073a1adf7b87;p=functest.git diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index c4d9a6d77..25aa7cd26 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -1,4 +1,3 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. .. SPDX-License-Identifier: CC-BY-4.0 Installation and configuration @@ -10,36 +9,29 @@ the refactoring on dependency management should allow the creation of light and fully customized docker images. -Functest Dockers ----------------- +Functest Dockers for OpenStack deployment +----------------------------------------- Docker images are available on the dockerhub: * opnfv/functest-core * opnfv/functest-healthcheck * opnfv/functest-smoke - * opnfv/functest-features - * opnfv/functest-components + * opnfv/functest-benchmarking * opnfv/functest-vnf - * opnfv/functest-parser - * opnfv/functest-restapi - -The tag "opnfv-5.0.0" is the official release image in Euphrates, but you can also pull "euphrates" -tag as it is being maintained by Functest team and might include bugfixes. - -The Functest docker container environment can -in principle- be also -used with non-OPNFV official installers (e.g. 'devstack'), with the -**disclaimer** that support for such environments is outside of the -scope and responsibility of the OPNFV project. Preparing your environment --------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^ cat env:: - INSTALLER_TYPE=XXX - EXTERNAL_NETWORK=XXX - DEPLOY_SCENARIO=XXX + DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario + NAMESERVER=XXX # if not 8.8.8.8 + EXTERNAL_NETWORK=XXX # if not first network with router:external=True + NEW_USER_ROLE=XXX # if not member + SDN_CONTROLLER_IP=XXX # if odl scenario + VOLUME_DEVICE_NAME=XXX # if not vdb + FLAVOR_EXTRA_SPECS=hw:mem_page_size:large # if fdio scenarios See section on environment variables for details. @@ -49,37 +41,28 @@ cat env_file:: export OS_USER_DOMAIN_NAME=XXX export OS_PROJECT_DOMAIN_NAME=XXX export OS_USERNAME=XXX - export OS_TENANT_NAME=XXX export OS_PROJECT_NAME=XXX export OS_PASSWORD=XXX - export OS_VOLUME_API_VERSION=XXX - export OS_IDENTITY_API_VERSION=XXX - export OS_IMAGE_API_VERSION=XXX + export OS_IDENTITY_API_VERSION=3 + export OS_REGION_NAME=XXX See section on OpenStack credentials for details. Create a directory for the different images (attached as a Docker volume):: - mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/euphrates | bash -s -- images && ls -1 images/* + mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/* - images/CentOS-7-aarch64-GenericCloud.qcow2 - images/CentOS-7-aarch64-GenericCloud.qcow2.xz - images/CentOS-7-x86_64-GenericCloud.qcow2 + images/cirros-0.4.0-aarch64-disk.img images/cirros-0.4.0-x86_64-disk.img - images/cirros-0.4.0-x86_64-lxc.tar.gz - images/cirros-d161201-aarch64-disk.img - images/cirros-d161201-aarch64-initramfs - images/cirros-d161201-aarch64-kernel - images/cloudify-manager-premium-4.0.1.qcow2 - images/img - images/trusty-server-cloudimg-amd64-disk1.img + images/cloudify-docker-manager-community-19.01.24.tar + images/shaker-image-1.3.0+stretch.qcow2 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img images/ubuntu-16.04-server-cloudimg-amd64-disk1.img - images/vyos-1.1.7.img + images/vyos-1.1.8-amd64.qcow2 Testing healthcheck suite --------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^ Run healthcheck suite:: @@ -90,17 +73,27 @@ Run healthcheck suite:: Results shall be displayed as follows:: - +----------------------------+------------------+---------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +----------------------------+------------------+---------------------+------------------+----------------+ - | connection_check | functest | healthcheck | 00:06 | PASS | - | api_check | functest | healthcheck | 06:52 | PASS | - | snaps_health_check | functest | healthcheck | 00:42 | PASS | - +----------------------------+------------------+---------------------+------------------+----------------+ - NOTE: the duration is a reference and it might vary depending on your SUT. + +--------------------------+------------------+---------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------------+------------------+---------------------+------------------+----------------+ + | connection_check | functest | healthcheck | 00:02 | PASS | + | tenantnetwork1 | functest | healthcheck | 00:06 | PASS | + | tenantnetwork2 | functest | healthcheck | 00:06 | PASS | + | vmready1 | functest | healthcheck | 00:06 | PASS | + | vmready2 | functest | healthcheck | 00:08 | PASS | + | singlevm1 | functest | healthcheck | 00:28 | PASS | + | singlevm2 | functest | healthcheck | 00:25 | PASS | + | vping_ssh | functest | healthcheck | 00:36 | PASS | + | vping_userdata | functest | healthcheck | 00:34 | PASS | + | cinder_test | functest | healthcheck | 01:03 | PASS | + | tempest_smoke | functest | healthcheck | 05:13 | PASS | + | odl | functest | healthcheck | 00:00 | SKIP | + +--------------------------+------------------+---------------------+------------------+----------------+ + +NOTE: the duration is a reference and it might vary depending on your SUT. Testing smoke suite -------------------- +^^^^^^^^^^^^^^^^^^^ Run smoke suite:: @@ -111,94 +104,141 @@ Run smoke suite:: Results shall be displayed as follows:: - +------------------------------+------------------+---------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +------------------------------+------------------+---------------+------------------+----------------+ - | vping_ssh | functest | smoke | 00:45 | PASS | - | vping_userdata | functest | smoke | 00:36 | PASS | - | tempest_smoke_serial | functest | smoke | 14:04 | PASS | - | rally_sanity | functest | smoke | 23:59 | PASS | - | refstack_defcore | functest | smoke | 14:37 | FAIL | - | snaps_smoke | functest | smoke | 42:09 | PASS | - | odl | functest | smoke | 00:00 | SKIP | - | odl_netvirt | functest | smoke | 00:00 | SKIP | - +------------------------------+------------------+---------------+------------------+----------------+ - Note: if the scenario does not support some tests, they are indicated as SKIP. - See User guide for details. - -Testing features suite ----------------------- - -Run features suite:: + +------------------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +------------------------------------+------------------+---------------+------------------+----------------+ + | neutron-tempest-plugin-api | functest | smoke | 09:12 | PASS | + | rally_sanity | functest | smoke | 16:29 | PASS | + | refstack_compute | functest | smoke | 06:25 | PASS | + | refstack_object | functest | smoke | 01:54 | PASS | + | refstack_platform | functest | smoke | 06:52 | PASS | + | tempest_full | functest | smoke | 30:26 | PASS | + | tempest_scenario | functest | smoke | 09:23 | PASS | + | tempest_slow | functest | smoke | 24:42 | PASS | + | patrole | functest | smoke | 02:36 | PASS | + | barbican | functest | smoke | 02:13 | PASS | + | neutron_trunk | functest | smoke | 00:00 | SKIP | + | networking-bgpvpn | functest | smoke | 00:00 | SKIP | + | networking-sfc | functest | smoke | 00:00 | SKIP | + | octavia | functest | smoke | 00:00 | SKIP | + +------------------------------------+------------------+---------------+------------------+----------------+ + +Note: if the scenario does not support some tests, they are indicated as SKIP. +See User guide for details. + +Testing benchmarking suite +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run benchmarking suite:: sudo docker run --env-file env \ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ -v $(pwd)/images:/home/opnfv/functest/images \ - opnfv/functest-features + opnfv/functest-benchmarking Results shall be displayed as follows:: - +-----------------------------+------------------------+------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +-----------------------------+------------------------+------------------+------------------+----------------+ - | doctor-notification | doctor | features | 00:00 | SKIP | - | bgpvpn | sdnvpn | features | 00:00 | SKIP | - | functest-odl-sfc | sfc | features | 00:00 | SKIP | - | barometercollectd | barometer | features | 00:00 | SKIP | - | fds | fastdatastacks | features | 00:00 | SKIP | - +-----------------------------+------------------------+------------------+------------------+----------------+ - Note: if the scenario does not support some tests, they are indicated as SKIP. - See User guide for details. + +--------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------+------------------+----------------------+------------------+----------------+ + | rally_full | functest | benchmarking | 92:16 | PASS | + | rally_jobs | functest | benchmarking | 18:49 | PASS | + | vmtp | functest | benchmarking | 15:28 | PASS | + | shaker | functest | benchmarking | 24:04 | PASS | + +--------------------+------------------+----------------------+------------------+----------------+ -Testing components suite ------------------------- +Note: if the scenario does not support some tests, they are indicated as SKIP. +See User guide for details. -Run components suite:: +Testing vnf suite +^^^^^^^^^^^^^^^^^ + +Run vnf suite:: sudo docker run --env-file env \ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ -v $(pwd)/images:/home/opnfv/functest/images \ - opnfv/functest-components + opnfv/functest-vnf Results shall be displayed as follows:: - +-------------------------------+------------------+--------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +-------------------------------+------------------+--------------------+------------------+----------------+ - | tempest_full_parallel | functest | components | 52:42 | PASS | - | rally_full | functest | components | 114:22 | PASS | - +-------------------------------+------------------+--------------------+------------------+----------------+ + +----------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +----------------------+------------------+--------------+------------------+----------------+ + | cloudify | functest | vnf | 03:49 | PASS | + | cloudify_ims | functest | vnf | 24:20 | PASS | + | heat_ims | functest | vnf | 32:13 | PASS | + | vyos_vrouter | functest | vnf | 14:55 | PASS | + | juju_epc | functest | vnf | 41:24 | PASS | + +----------------------+------------------+--------------+------------------+----------------+ + +Functest Dockers for Kubernetes deployment +------------------------------------------ +Docker images are available on the dockerhub: -Testing vnf suite ------------------ + * opnfv/functest-kubernetes-core + * opnfv/functest-kubernetest-healthcheck + * opnfv/functest-kubernetest-smoke -Run vnf suite:: +Preparing your environment +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +cat env:: + + DEPLOY_SCENARIO=k8s-XXX + +Testing healthcheck suite +^^^^^^^^^^^^^^^^^^^^^^^^^ -sudo docker run --env-file env \ - -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ - -v $(pwd)/images:/home/opnfv/functest/images \ - opnfv/functest-vnf +Run healthcheck suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-healthcheck + +A config file in the current dir 'config' is also required, which should be +volume mapped to ~/.kube/config inside kubernetes container. Results shall be displayed as follows:: - +---------------------------------+-------------------+--------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +---------------------------------+-------------------+--------------+------------------+----------------+ - | cloudify_ims | functest | vnf | 28:49 | FAIL | - | vyos_vrouter | functest | vnf | 27:57 | FAIL | - | juju_epc | functest | vnf | 55:03 | PASS | - | orchestra_openims | orchestra | vnf | 00:00 | SKIP | - | orchestra_clearwaterims | orchestra | vnf | 00:00 | SKIP | - +---------------------------------+-------------------+--------------+------------------+----------------+ + +-------------------+------------------+---------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------+------------------+---------------------+------------------+----------------+ + | k8s_smoke | functest | healthcheck | 01:09 | PASS | + +-------------------+------------------+---------------------+------------------+----------------+ + +Testing smoke suite +^^^^^^^^^^^^^^^^^^^ + +Run smoke suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-smoke + +Results shall be displayed as follows:: + +---------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+------------------+---------------+------------------+----------------+ + | xrally_kubernetes | functest | smoke | 22:04 | PASS | + | k8s_conformance | functest | smoke | 173:48 | PASS | + +---------------------------+------------------+---------------+------------------+----------------+ Environment variables ===================== Several environement variables may be specified: - * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid) + + * INSTALLER_IP= * DEPLOY_SCENARIO=--- + * NAMESERVER=XXX # if not 8.8.8.8 + * VOLUME_DEVICE_NAME=XXX # if not vdb + * EXTERNAL_NETWORK=XXX # if not first network with router:external=True + * NEW_USER_ROLE=XXX # if not member +INSTALLER_IP is required by Barometer in order to access the installer node and +the deployment. The format for the DEPLOY_SCENARIO env variable can be described as follows: * vim: (os|k8s) = OpenStack or Kubernetes @@ -215,8 +255,8 @@ release note. **NOTE:** The scenario name is mainly used to automatically detect if a test suite is runnable or not (e.g. it will prevent ODL test suite to be -run on 'nosdn' scenarios). If not set, Functest will try to run the default test -cases that might not include SDN controller or a specific feature +run on 'nosdn' scenarios). If not set, Functest will try to run the default +test cases that might not include SDN controller or a specific feature. **NOTE:** An HA scenario means that 3 OpenStack controller nodes are deployed. It does not necessarily mean that the whole system is HA. See @@ -228,8 +268,9 @@ to the Functest Docker Container, using the -e only relevant to Jenkins CI invoked testing and **should not be used** when performing manual test scenarios: - * NODE_NAME = - * BUILD_TAG = + * INSTALLER_TYPE=(apex|compass|daisy|fuel) + * NODE_NAME= + * BUILD_TAG= where: @@ -305,194 +346,6 @@ be:: export OS_INSECURE=true -Functest docker container directory structure -============================================= -Inside the Functest docker container, the following directory structure -should now be in place:: - - `-- - |- home - | |-- opnfv - | | `- functest - | | |-- conf - | | `-- results - | `-- repos - | `-- vnfs - |- src - | |-- tempest - | |-- vims-test - | |-- odl_test - | `-- fds - `- usr - `- lib - `- python2.7 - `- site-packages - `- functest - |-- ... - -Underneath the '/home/opnfv/functest' directory, the Functest docker container -includes two main directories: - - * The **conf** directory stores configuration files (e.g. the - OpenStack creds are stored in path '/home/opnfv/functest/conf/env_file'), - * the **results** directory stores some temporary result log files - -src and repos directories are used to host third party code used for the tests. - -The structure of functest repo can be described as follows:: - - |-- INFO - |-- LICENSE - |-- api - | `-- apidoc - |-- build.sh - |-- commons - |-- docker - | |-- components - | |-- core - | |-- features - | |-- healthcheck - | |-- smoke - | |-- vnf - | |-- parser - | |-- restapi - | |-- thirdparty-requirements.txt - |-- docs - | |-- com - | |-- images - | |-- release - | | `-- release-notes - | |-- results - | | testing - | | |-- developer - | | `-- user - | | |-- configguide - | | `-- userguide - `-- functest - |-- api - | |-- base.py - | |-- server.py - | |-- urls.py - | |-- common - | | |-- api_utils.py - | | |-- thread.py - | `-- resources - | `-- v1 - | |-- creds.py - | |-- envs.py - | |-- testcases.py - | |-- tiers.py - | |-- tasks.py - | `-- database - | |-- db.py - | `-- v1 - | |-- handlers.py - | |-- models.py - | `-- swagger - |-- ci - │ |-- check_deployment.py - │ |-- config_aarch64_patch.yaml - │ |-- config_functest.yaml - │ |-- config_patch.yaml - │ |-- download_images.sh - │ |-- logging.ini - │ |-- rally_aarch64_patch.conf - │ |-- run_tests.py - │ |-- testcases.yaml - │ |-- tier_builder.py - │ |-- tier_handler.py - |-- cli - │ |-- cli_base.py - │ |-- commands - │ │ |-- cli_env.py - │ │ |-- cli_os.py - │ │ |-- cli_testcase.py - │ │ |-- cli_tier.py - │ |-- functest-complete.sh - |-- core - │ |-- feature.py - │ |-- robotframework.py - │ |-- testcase.py - │ |-- unit.py - │ |-- vnf.py - |-- energy - │ |-- energy.py - |-- opnfv_tests - │ `-- openstack - │ |-- rally - │ |-- refstack_client - │ |-- snaps - │ |-- tempest - │ |-- vping - │ `-- sdn - │ │ `-- odl - │ `-- vnf - │ |-- ims - │ `-- router - |-- tests - │ `-- unit - │ |-- ci - │ |-- cli - │ |-- core - │ |-- energy - │ |-- features - │ |-- odl - │ |-- openstack - │ |-- opnfv_tests - │ |-- test_utils.py - │ |-- utils - │ `-- vnf - |-- utils - | |-- config.py - | |-- constants.py - | |-- decorators.py - | |-- env.py - | |-- functest_utils.py - | |-- functest_vacation.py - | |-- openstack_clean.py - | |-- openstack_tacker.py - | `-- openstack_utils.py - |-- requirements.txt - |-- setup.cfg - |-- setup.py - |-- test-requirements.txt - |-- tox.ini - |-- upper-constraints.txt - - (Note: All *.pyc files removed from above list for brevity...) - -We may distinguish several directories, the first level has 5 directories: - -* **api**: This directory is dedicated to the API (framework) documentations. -* **commons**: This directory is dedicated for storage of traffic profile or - any other test inputs that could be reused by any test project. -* **docker**: This directory includes the needed files and tools to - build the Functest Docker images. -* **docs**: This directory includes documentation: Release Notes, - User Guide, Configuration Guide and Developer Guide. -* **functest**: This directory contains all the code needed to run - functest internal cases and OPNFV onboarded feature or VNF test cases. - -Functest directory has 7 sub-directories, which is located under -/usr/lib/python2.7/site-packages/functest: - * **api**: This directory is dedicated for the internal Functest API. - * **ci**: This directory contains test structure definition files - (e.g .yaml) and bash shell/python scripts used to - configure and execute Functional tests. The test execution script - can be executed under the control of Jenkins CI jobs. - * **cli**: This directory holds the python based Functest CLI utility - source code, which is based on the Python 'click' framework. - * **core**: This directory holds the python based Functest core - source code. Three abstraction classes have been created to ease - the integration of internal, feature or vnf cases. - * **opnfv_tests**: This directory includes the scripts required by - Functest internal test cases and other feature projects test cases. - * **tests**: This directory includes the functest unit tests. - * **utils**: this directory holds Python source code for some general - purpose helper utilities, which testers can also re-use in their - own test code. See for an example the Openstack helper utility: - 'openstack_utils.py'. - Logs ==== @@ -500,7 +353,7 @@ By default all the logs are put un /home/opnfv/functest/results/functest.log. If you want to have more logs in console, you may edit the logging.ini file manually. Connect on the docker then edit the file located in -/usr/lib/python2.7/site-packages/functest/ci/logging.ini +/usr/lib/python3.6/site-packages/xtesting/ci/logging.ini Change wconsole to console in the desired module to get more traces. @@ -510,7 +363,8 @@ Configuration You may also directly modify the python code or the configuration file (e.g. testcases.yaml used to declare test constraints) under -/usr/lib/python2.7/site-packages/functest +/usr/lib/python3.6/site-packages/xtesting and +/usr/lib/python3.6/site-packages/functest Tips @@ -602,6 +456,8 @@ If the OpenStack command still does not show anything or complains about connectivity issues, it could be due to an incorrect url given to the OS_AUTH_URL environment variable. Check the deployment settings. +.. _`Proxy support`: + Proxy support ------------- If your Jumphost node is operating behind a http proxy, then there are @@ -611,7 +467,7 @@ succeed: #. Initial installation of docker engine First, try following the official Docker documentation for Proxy settings. Some issues were experienced on CentOS 7 based Jumphost. Some tips are documented - in section: `Docker Installation on CentOS behind http proxy`_ + in section: :ref:`Docker Installation on CentOS behind http proxy` below. If that is the case, make sure the resolv.conf and the needed @@ -657,7 +513,7 @@ docker container:: Note: In a Jumphost node based on the CentOS family OS, the **nc** commands might not work. You can use the **curl** command instead. - curl http://www.opnfv.org:80 + curl https://www.opnfv.org/