1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. SPDX-License-Identifier: CC-BY-4.0
4 Installation and configuration
5 ==============================
7 Alpine containers have been introduced in Euphrates.
8 Alpine allows Functest testing in several very light containers and thanks to
9 the refactoring on dependency management should allow the creation of light and
10 fully customized docker images.
15 Docker images are available on the dockerhub:
18 * opnfv/functest-healthcheck
19 * opnfv/functest-smoke
20 * opnfv/functest-features
21 * opnfv/functest-components
23 * opnfv/functest-parser
24 * opnfv/functest-restapi
26 The tag "opnfv-5.0.0" is the official release image in Euphrates, but you can also pull "euphrates"
27 tag as it is being maintained by Functest team and might include bugfixes.
29 The Functest docker container environment can -in principle- be also
30 used with non-OPNFV official installers (e.g. 'devstack'), with the
31 **disclaimer** that support for such environments is outside of the
32 scope and responsibility of the OPNFV project.
35 Preparing your environment
36 --------------------------
45 See section on environment variables for details.
49 export OS_AUTH_URL=XXX
50 export OS_USER_DOMAIN_NAME=XXX
51 export OS_PROJECT_DOMAIN_NAME=XXX
52 export OS_USERNAME=XXX
53 export OS_TENANT_NAME=XXX
54 export OS_PROJECT_NAME=XXX
55 export OS_PASSWORD=XXX
56 export OS_VOLUME_API_VERSION=XXX
57 export OS_IDENTITY_API_VERSION=XXX
58 export OS_IMAGE_API_VERSION=XXX
60 See section on OpenStack credentials for details.
62 Create a directory for the different images (attached as a Docker volume)::
64 mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/euphrates | bash -s -- images && ls -1 images/*
66 images/CentOS-7-aarch64-GenericCloud.qcow2
67 images/CentOS-7-aarch64-GenericCloud.qcow2.xz
68 images/CentOS-7-x86_64-GenericCloud.qcow2
69 images/cirros-0.3.5-x86_64-disk.img
70 images/cirros-0.3.5-x86_64-lxc.tar.gz
71 images/cirros-d161201-aarch64-disk.img
72 images/cirros-d161201-aarch64-initramfs
73 images/cirros-d161201-aarch64-kernel
74 images/cloudify-manager-premium-4.0.1.qcow2
76 images/trusty-server-cloudimg-amd64-disk1.img
77 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
78 images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
79 images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
83 Testing healthcheck suite
84 --------------------------
86 Run healthcheck suite::
88 sudo docker run --env-file env \
89 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
90 -v $(pwd)/images:/home/opnfv/functest/images \
91 opnfv/functest-healthcheck
93 Results shall be displayed as follows::
95 +----------------------------+------------------+---------------------+------------------+----------------+
96 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
97 +----------------------------+------------------+---------------------+------------------+----------------+
98 | connection_check | functest | healthcheck | 00:02 | PASS |
99 | api_check | functest | healthcheck | 04:57 | PASS |
100 | snaps_health_check | functest | healthcheck | 00:51 | PASS |
101 +----------------------------+------------------+---------------------+------------------+----------------+
102 NOTE: the duration is a reference and it might vary depending on your SUT.
109 sudo docker run --env-file env \
110 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
111 -v $(pwd)/images:/home/opnfv/functest/images \
114 Results shall be displayed as follows::
116 +------------------------------+------------------+---------------+------------------+----------------+
117 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
118 +------------------------------+------------------+---------------+------------------+----------------+
119 | vping_ssh | functest | smoke | 01:19 | PASS |
120 | vping_userdata | functest | smoke | 01:56 | PASS |
121 | tempest_smoke_serial | functest | smoke | 26:30 | PASS |
122 | rally_sanity | functest | smoke | 19:42 | PASS |
123 | refstack_defcore | functest | smoke | 22:00 | PASS |
124 | snaps_smoke | functest | smoke | 41:14 | PASS |
125 | odl | functest | smoke | 00:16 | PASS |
126 | odl_netvirt | functest | smoke | 00:00 | SKIP |
127 | fds | functest | smoke | 00:00 | SKIP |
128 +------------------------------+------------------+---------------+------------------+----------------+
129 Note: if the scenario does not support some tests, they are indicated as SKIP.
130 See User guide for details.
132 Testing features suite
133 ----------------------
137 sudo docker run --env-file env \
138 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
139 -v $(pwd)/images:/home/opnfv/functest/images \
140 opnfv/functest-features
142 Results shall be displayed as follows::
144 +---------------------------+--------------------------+------------------+------------------+----------------+
145 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
146 +---------------------------+--------------------------+------------------+------------------+----------------+
147 | promise | promise | features | 00:00 | SKIP |
148 | bgpvpn | sdnvpn | features | 00:00 | SKIP |
149 | security_scan | securityscanning | features | 00:00 | SKIP |
150 | functest-odl-sfc | sfc | features | 00:00 | SKIP |
151 | domino-multinode | domino | features | 00:00 | SKIP |
152 | barometercollectd | barometer | features | 00:00 | SKIP |
153 +---------------------------+--------------------------+------------------+------------------+----------------+
154 Note: if the scenario does not support some tests, they are indicated as SKIP.
155 See User guide for details.
157 Testing components suite
158 ------------------------
160 Run components suite::
162 sudo docker run --env-file env \
163 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
164 -v $(pwd)/images:/home/opnfv/functest/images \
165 opnfv/functest-components
167 Results shall be displayed as follows::
169 +-------------------------------+------------------+--------------------+------------------+----------------+
170 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
171 +-------------------------------+------------------+--------------------+------------------+----------------+
172 | tempest_full_parallel | functest | components | 102:48 | PASS |
173 | rally_full | functest | components | 160:58 | PASS |
174 +-------------------------------+------------------+--------------------+------------------+----------------+
181 sudo docker run --env-file env \
182 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
183 -v $(pwd)/images:/home/opnfv/functest/images \
186 Results shall be displayed as follows::
188 +---------------------------------+------------------+--------------+------------------+----------------+
189 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
190 +---------------------------------+------------------+--------------+------------------+----------------+
191 | cloudify_ims | functest | vnf | 21:25 | PASS |
192 | orchestra_openims | functest | vnf | 11:02 | FAIL |
193 | orchestra_clearwaterims | functest | vnf | 09:13 | FAIL |
194 +---------------------------------+------------------+--------------+------------------+----------------+
197 Environment variables
198 =====================
200 Several environement variables may be specified:
201 * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid)
202 * INSTALLER_IP=<Specific IP Address>
203 * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
206 INSTALLER IP may be required by some test cases like SFC or Barometer in order
207 to access the installer node and the deployment.
209 The format for the DEPLOY_SCENARIO env variable can be described as follows:
210 * vim: (os|k8s) = OpenStack or Kubernetes
211 * controller is one of ( nosdn | odl )
212 * nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
213 * ha_mode (high availability) is one of ( ha | noha )
215 If several features are pertinent then use the underscore character '_' to
216 separate each feature (e.g. ovs_kvm). 'nofeature' indicates that no OPNFV
219 The list of supported scenarios per release/installer is indicated in the
222 **NOTE:** The scenario name is mainly used to automatically detect
223 if a test suite is runnable or not (e.g. it will prevent ODL test suite to be
224 run on 'nosdn' scenarios). If not set, Functest will try to run the default test
225 cases that might not include SDN controller or a specific feature
227 **NOTE:** An HA scenario means that 3 OpenStack controller nodes are
228 deployed. It does not necessarily mean that the whole system is HA. See
229 installer release notes for details.
231 Finally, three additional environment variables can also be passed in
232 to the Functest Docker Container, using the -e
233 "<EnvironmentVariable>=<Value>" mechanism. The first two parameters are
234 only relevant to Jenkins CI invoked testing and **should not be used**
235 when performing manual test scenarios:
237 * NODE_NAME = <Test POD Name>
238 * BUILD_TAG = <Jenkins Build Tag>
239 * CI_DEBUG = <DebugTraceValue>
243 * <Test POD Name> = Symbolic name of the POD where the tests are run.
244 Visible in test results files, which are stored
245 to the database. This option is only used when
246 tests are activated under Jenkins CI control.
247 It indicates the POD/hardware where the test has
248 been run. If not specified, then the POD name is
249 defined as "Unknown" by default.
250 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
251 * <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
252 Visible in test results files, which are stored
253 to the database. This option is only set when
254 tests are activated under Jenkins CI control.
255 It enables the correlation of test results,
256 which are independently pushed to the results database
257 from different Jenkins jobs.
258 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
259 * <DebugTraceValue> = "true" or "false"
260 Default = "false", if not specified
261 If "true" is specified, then additional debug trace
262 text can be sent to the test results file / log files
263 and also to the standard console output.
266 Openstack credentials
267 =====================
268 OpenStack credentials are mandatory and must be provided to Functest.
269 When running the command "functest env prepare", the framework will
270 automatically look for the Openstack credentials file
271 "/home/opnfv/functest/conf/openstack.creds" and will exit with
272 error if it is not present or is empty.
274 There are 2 ways to provide that file:
276 * by using a Docker volume with -v option when creating the Docker container.
277 This is referred to in docker documentation as "Bind Mounting".
278 See the usage of this parameter in the following chapter.
279 * or creating manually the file '/home/opnfv/functest/conf/openstack.creds'
280 inside the running container and pasting the credentials in it. Consult
281 your installer guide for further details. This is however not
282 instructed in this document.
284 In proxified environment you may need to change the credentials file.
285 There are some tips in chapter: `Proxy support`_
289 If you need to connect to a server that is TLS-enabled (the auth URL
290 begins with "https") and it uses a certificate from a private CA or a
291 self-signed certificate, then you will need to specify the path to an
292 appropriate CA certificate to use, to validate the server certificate
293 with the environment variable OS_CACERT::
296 /etc/ssl/certs/ca.crt
298 However, this certificate does not exist in the container by default.
299 It has to be copied manually from the OpenStack deployment. This can be
302 #. Create manually that file and copy the contents from the OpenStack
304 #. (Recommended) Add the file using a Docker volume when starting the
307 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
309 You might need to export OS_CACERT environment variable inside the
312 export OS_CACERT=/etc/ssl/certs/ca.crt
314 Certificate verification can be turned off using OS_INSECURE=true. For
315 example, Fuel uses self-signed cacerts by default, so an pre step would
318 export OS_INSECURE=true
320 Functest docker container directory structure
321 =============================================
322 Inside the Functest docker container, the following directory structure
323 should now be in place::
345 Underneath the '/home/opnfv/functest' directory, the Functest docker container
346 includes two main directories:
348 * The **conf** directory stores configuration files (e.g. the
349 OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
350 * the **results** directory stores some temporary result log files
352 src and repos directories are used to host third party code used for the tests.
354 The structure of functest repo can be described as follows::
371 | |-- thirdparty-requirements.txt
376 | | `-- release-notes
405 │  |-- check_deployment.py
406 │  |-- config_aarch64_patch.yaml
407 │  |-- config_functest.yaml
408 │  |-- config_patch.yaml
409 │  |-- download_images.sh
410 │  |-- logging.ini
411 │  |-- prepare_env.py
412 │  |-- rally_aarch64_patch.conf
413 │  |-- run_tests.py
414 │  |-- testcases.yaml
415 │  |-- tier_builder.py
416 │  |-- tier_handler.py
418 │  |-- cli_base.py
420 │  │  |-- cli_env.py
421 │  │  |-- cli_os.py
422 │  │  |-- cli_testcase.py
423 │  │  |-- cli_tier.py
424 │  |-- functest-complete.sh
426 │  |-- feature.py
427 │  |-- testcase.py
435 │    |-- refstack_client
437 │    |-- tempest
440 │  │  `-- odl
453 │  |-- opnfv_tests
454 │  |-- test_utils.py
462 | |-- functest_utils.py
463 | |-- functest_vacation.py
464 | |-- openstack_clean.py
465 | |-- openstack_snapshot.py
466 | |-- openstack_tacker.py
467 | `-- openstack_utils.py
471 |-- test-requirements.txt
473 |-- upper-constraints.txt
475 (Note: All *.pyc files removed from above list for brevity...)
477 We may distinguish several directories, the first level has 5 directories:
479 * **api**: This directory is dedicated to the API (framework) documentations.
480 * **commons**: This directory is dedicated for storage of traffic profile or
481 any other test inputs that could be reused by any test project.
482 * **docker**: This directory includes the needed files and tools to
483 build the Functest Docker images.
484 * **docs**: This directory includes documentation: Release Notes,
485 User Guide, Configuration Guide and Developer Guide.
486 * **functest**: This directory contains all the code needed to run
487 functest internal cases and OPNFV onboarded feature or VNF test cases.
489 Functest directory has 7 sub-directories, which is located under
490 /usr/lib/python2.7/site-packages/functest:
491 * **api**: This directory is dedicated for the internal Functest API.
492 * **ci**: This directory contains test structure definition files
493 (e.g <filename>.yaml) and bash shell/python scripts used to
494 configure and execute Functional tests. The test execution script
495 can be executed under the control of Jenkins CI jobs.
496 * **cli**: This directory holds the python based Functest CLI utility
497 source code, which is based on the Python 'click' framework.
498 * **core**: This directory holds the python based Functest core
499 source code. Three abstraction classes have been created to ease
500 the integration of internal, feature or vnf cases.
501 * **opnfv_tests**: This directory includes the scripts required by
502 Functest internal test cases and other feature projects test cases.
503 * **tests**: This directory includes the functest unit tests.
504 * **utils**: this directory holds Python source code for some general
505 purpose helper utilities, which testers can also re-use in their
506 own test code. See for an example the Openstack helper utility:
507 'openstack_utils.py'.
512 By default all the logs are put un /home/opnfv/functest/results/functest.log.
513 If you want to have more logs in console, you may edit the logging.ini file
515 Connect on the docker then edit the file located in
516 /usr/lib/python2.7/site-packages/functest/ci/logging.ini
518 Change wconsole to console in the desired module to get more traces.
524 You may also directly modify the python code or the configuration file (e.g.
525 testcases.yaml used to declare test constraints) under
526 /usr/lib/python2.7/site-packages/functest
534 When typing **exit** in the container prompt, this will cause exiting
535 the container and probably stopping it. When stopping a running Docker
536 container all the changes will be lost, there is a keyboard shortcut
537 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
538 reconnect to the running container **DO NOT** use the *run* command
539 again (since it will create a new container), use the *exec* or *attach*
542 docker ps # <check the container ID from the output>
543 docker exec -ti <CONTAINER_ID> /bin/bash
545 There are other useful Docker commands that might be needed to manage possible
546 issues with the containers.
548 List the running containers::
552 List all the containers including the stopped ones::
556 Start a stopped container named "FunTest"::
560 Attach to a running container named "StrikeTwo"::
562 docker attach StrikeTwo
564 It is useful sometimes to remove a container if there are some problems::
566 docker rm <CONTAINER_ID>
568 Use the *-f* option if the container is still running, it will force to
571 docker rm -f <CONTAINER_ID>
573 Check the Docker documentation [`dockerdocs`_] for more information.
576 Checking Openstack and credentials
577 ----------------------------------
578 It is recommended and fairly straightforward to check that Openstack
579 and credentials are working as expected.
581 Once the credentials are there inside the container, they should be
582 sourced before running any Openstack commands::
584 source /home/opnfv/functest/conf/openstack.creds
586 After this, try to run any OpenStack command to see if you get any
587 output, for instance::
591 This will return a list of the actual users in the OpenStack
592 deployment. In any other case, check that the credentials are sourced::
596 This command must show a set of environment variables starting with
599 OS_REGION_NAME=RegionOne
600 OS_USER_DOMAIN_NAME=Default
601 OS_PROJECT_NAME=admin
603 OS_IDENTITY_API_VERSION=3
604 OS_PASSWORD=da54c27ae0d10dfae5297e6f0d6be54ebdb9f58d0f9dfc
605 OS_AUTH_URL=http://10.1.0.9:5000/v3
608 OS_ENDPOINT_TYPE=internalURL
609 OS_INTERFACE=internalURL
611 OS_PROJECT_DOMAIN_NAME=Default
614 If the OpenStack command still does not show anything or complains
615 about connectivity issues, it could be due to an incorrect url given to
616 the OS_AUTH_URL environment variable. Check the deployment settings.
620 If your Jumphost node is operating behind a http proxy, then there are
621 2 places where some special actions may be needed to make operations
624 #. Initial installation of docker engine First, try following the
625 official Docker documentation for Proxy settings. Some issues were
626 experienced on CentOS 7 based Jumphost. Some tips are documented
627 in section: `Docker Installation on CentOS behind http proxy`_
630 If that is the case, make sure the resolv.conf and the needed
631 http_proxy and https_proxy environment variables, as well as the
632 'no_proxy' environment variable are set correctly::
634 # Make double sure that the 'no_proxy=...' line in the
635 # 'openstack.creds' file is commented out first. Otherwise, the
636 # values set into the 'no_proxy' environment variable below will
637 # be ovewrwritten, each time the command
638 # 'source ~/functest/conf/openstack.creds' is issued.
641 sed -i 's/export no_proxy/#export no_proxy/' openstack.creds
642 source ./openstack.creds
644 # Next calculate some IP addresses for which http_proxy
645 # usage should be excluded:
647 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
649 adminURL_IP=$(openstack catalog show identity | \
650 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
652 export http_proxy="<your http proxy settings>"
653 export https_proxy="<your https proxy settings>"
654 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
656 # Ensure that "git" uses the http_proxy
657 # This may be needed if your firewall forbids SSL based git fetch
658 git config --global http.sslVerify True
659 git config --global http.proxy <Your http proxy settings>
661 For example, try to use the **nc** command from inside the functest
665 Connection to opnfv.org 80 port [tcp/http] succeeded!
668 Connection to opnfv.org 443 port [tcp/https] succeeded!
670 Note: In a Jumphost node based on the CentOS family OS, the **nc**
671 commands might not work. You can use the **curl** command instead.
673 curl http://www.opnfv.org:80
675 <HTML><HEAD><meta http-equiv="content-type"
680 curl https://www.opnfv.org:443
682 <HTML><HEAD><meta http-equiv="content-type"
687 (Ignore the content. If command returns a valid HTML page, it proves
690 Docker Installation on CentOS behind http proxy
691 -----------------------------------------------
692 This section is applicable for CentOS family OS on Jumphost which
693 itself is behind a proxy server. In that case, the instructions below
694 should be followed **before** installing the docker engine::
696 1) # Make a directory '/etc/systemd/system/docker.service.d'
697 # if it does not exist
698 sudo mkdir /etc/systemd/system/docker.service.d
700 2) # Create a file called 'env.conf' in that directory with
701 # the following contents:
703 EnvironmentFile=-/etc/sysconfig/docker
705 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
706 # with the following contents:
707 HTTP_PROXY="<Your http proxy settings>"
708 HTTPS_PROXY="<Your https proxy settings>"
709 http_proxy="${HTTP_PROXY}"
710 https_proxy="${HTTPS_PROXY}"
712 4) # Reload the daemon
713 systemctl daemon-reload
715 5) # Sanity check - check the following docker settings:
716 systemctl show docker | grep -i env
720 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
721 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
723 Now follow the instructions in [`Install Docker on CentOS`_] to download
724 and install the **docker-engine**. The instructions conclude with a
725 "test pull" of a sample "Hello World" docker container. This should now
726 work with the above pre-requisite actions.
729 .. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
730 .. _`dockerdocs`: https://docs.docker.com/
731 .. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy
732 .. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
733 .. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
734 .. _`images/CentOS-7-x86_64-GenericCloud.qcow2` http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
735 .. _`images/cirros-0.3.5-x86_64-disk.img` https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img
736 .. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img` https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2