1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. SPDX-License-Identifier: CC-BY-4.0
4 Installation and configuration
5 ==============================
7 Alpine containers have been introduced in Euphrates.
8 Alpine allows Functest testing in several very light containers and thanks to
9 the refactoring on dependency management should allow the creation of light and
10 fully customized docker images.
12 It is still possible to use the monolithic Ubuntu image opnfv/functest especially
13 for tests on Aarch64 architecture.
17 Docker images are available on the dockerhub:
20 * opnfv/functest-healthcheck
21 * opnfv/functest-smoke
22 * opnfv/functest-features
23 * opnfv/functest-components
25 * opnfv/functest-parser
26 * opnfv/functest-restapi
28 The tag "opnfv-5.0.0" is the official release image in Euphrates, but you can also pull "euphrates"
29 tag as it is being maintained by Functest team and might include bugfixes.
31 The Functest docker container environment can -in principle- be also
32 used with non-OPNFV official installers (e.g. 'devstack'), with the
33 **disclaimer** that support for such environments is outside of the
34 scope and responsibility of the OPNFV project.
37 Preparing your environment
38 --------------------------
47 See section on environment variables for details.
51 export OS_AUTH_URL=XXX
52 export OS_USER_DOMAIN_NAME=XXX
53 export OS_PROJECT_DOMAIN_NAME=XXX
54 export OS_USERNAME=XXX
55 export OS_TENANT_NAME=XXX
56 export OS_PROJECT_NAME=XXX
57 export OS_PASSWORD=XXX
58 export OS_VOLUME_API_VERSION=XXX
59 export OS_IDENTITY_API_VERSION=XXX
60 export OS_IMAGE_API_VERSION=XXX
62 See section on OpenStack credentials for details.
64 Create a directory for the different images (attached as a Docker volume)::
66 mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/euphrates | bash -s -- images && ls -1 images/*
68 images/CentOS-7-aarch64-GenericCloud.qcow2
69 images/CentOS-7-aarch64-GenericCloud.qcow2.xz
70 images/CentOS-7-x86_64-GenericCloud.qcow2
71 images/cirros-0.3.5-x86_64-disk.img
72 images/cirros-0.3.5-x86_64-lxc.tar.gz
73 images/cirros-d161201-aarch64-disk.img
74 images/cirros-d161201-aarch64-initramfs
75 images/cirros-d161201-aarch64-kernel
76 images/cloudify-manager-premium-4.0.1.qcow2
78 images/trusty-server-cloudimg-amd64-disk1.img
79 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
80 images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
81 images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
85 Testing healthcheck suite
86 --------------------------
88 Run healthcheck suite::
90 sudo docker run --env-file env \
91 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
92 -v $(pwd)/images:/home/opnfv/functest/images \
93 opnfv/functest-healthcheck
95 Results shall be displayed as follows::
97 +----------------------------+------------------+---------------------+------------------+----------------+
98 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
99 +----------------------------+------------------+---------------------+------------------+----------------+
100 | connection_check | functest | healthcheck | 00:02 | PASS |
101 | api_check | functest | healthcheck | 04:57 | PASS |
102 | snaps_health_check | functest | healthcheck | 00:51 | PASS |
103 +----------------------------+------------------+---------------------+------------------+----------------+
104 NOTE: the duration is a reference and it might vary depending on your SUT.
111 sudo docker run --env-file env \
112 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
113 -v $(pwd)/images:/home/opnfv/functest/images \
116 Results shall be displayed as follows::
118 +------------------------------+------------------+---------------+------------------+----------------+
119 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
120 +------------------------------+------------------+---------------+------------------+----------------+
121 | vping_ssh | functest | smoke | 01:19 | PASS |
122 | vping_userdata | functest | smoke | 01:56 | PASS |
123 | tempest_smoke_serial | functest | smoke | 26:30 | PASS |
124 | rally_sanity | functest | smoke | 19:42 | PASS |
125 | refstack_defcore | functest | smoke | 22:00 | PASS |
126 | snaps_smoke | functest | smoke | 41:14 | PASS |
127 | odl | functest | smoke | 00:16 | PASS |
128 | odl_netvirt | functest | smoke | 00:00 | SKIP |
129 | fds | functest | smoke | 00:00 | SKIP |
130 +------------------------------+------------------+---------------+------------------+----------------+
131 Note: if the scenario does not support some tests, they are indicated as SKIP.
132 See User guide for details.
134 Testing features suite
135 ----------------------
139 sudo docker run --env-file env \
140 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
141 -v $(pwd)/images:/home/opnfv/functest/images \
142 opnfv/functest-features
144 Results shall be displayed as follows::
146 +---------------------------+--------------------------+------------------+------------------+----------------+
147 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
148 +---------------------------+--------------------------+------------------+------------------+----------------+
149 | promise | promise | features | 00:00 | SKIP |
150 | bgpvpn | sdnvpn | features | 00:00 | SKIP |
151 | security_scan | securityscanning | features | 00:00 | SKIP |
152 | functest-odl-sfc | sfc | features | 00:00 | SKIP |
153 | domino-multinode | domino | features | 00:00 | SKIP |
154 | barometercollectd | barometer | features | 00:00 | SKIP |
155 +---------------------------+--------------------------+------------------+------------------+----------------+
156 Note: if the scenario does not support some tests, they are indicated as SKIP.
157 See User guide for details.
159 Testing components suite
160 ------------------------
162 Run components suite::
164 sudo docker run --env-file env \
165 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
166 -v $(pwd)/images:/home/opnfv/functest/images \
167 opnfv/functest-components
169 Results shall be displayed as follows::
171 +-------------------------------+------------------+--------------------+------------------+----------------+
172 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
173 +-------------------------------+------------------+--------------------+------------------+----------------+
174 | tempest_full_parallel | functest | components | 102:48 | PASS |
175 | rally_full | functest | components | 160:58 | PASS |
176 +-------------------------------+------------------+--------------------+------------------+----------------+
183 sudo docker run --env-file env \
184 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
185 -v $(pwd)/images:/home/opnfv/functest/images \
188 Results shall be displayed as follows::
190 +---------------------------------+------------------+--------------+------------------+----------------+
191 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
192 +---------------------------------+------------------+--------------+------------------+----------------+
193 | cloudify_ims | functest | vnf | 21:25 | PASS |
194 | orchestra_openims | functest | vnf | 11:02 | FAIL |
195 | orchestra_clearwaterims | functest | vnf | 09:13 | FAIL |
196 +---------------------------------+------------------+--------------+------------------+----------------+
199 Environment variables
200 =====================
202 Several environement variables may be specified:
203 * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid)
204 * INSTALLER_IP=<Specific IP Address>
205 * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
208 INSTALLER IP may be required by some test cases like SFC or Barometer in order
209 to access the installer node and the deployment.
211 The format for the DEPLOY_SCENARIO env variable can be described as follows:
212 * vim: (os|k8s) = OpenStack or Kubernetes
213 * controller is one of ( nosdn | odl )
214 * nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
215 * ha_mode (high availability) is one of ( ha | noha )
217 If several features are pertinent then use the underscore character '_' to
218 separate each feature (e.g. ovs_kvm). 'nofeature' indicates that no OPNFV
221 The list of supported scenarios per release/installer is indicated in the
224 **NOTE:** The scenario name is mainly used to automatically detect
225 if a test suite is runnable or not (e.g. it will prevent ODL test suite to be
226 run on 'nosdn' scenarios). If not set, Functest will try to run the default test
227 cases that might not include SDN controller or a specific feature
229 **NOTE:** An HA scenario means that 3 OpenStack controller nodes are
230 deployed. It does not necessarily mean that the whole system is HA. See
231 installer release notes for details.
233 Finally, three additional environment variables can also be passed in
234 to the Functest Docker Container, using the -e
235 "<EnvironmentVariable>=<Value>" mechanism. The first two parameters are
236 only relevant to Jenkins CI invoked testing and **should not be used**
237 when performing manual test scenarios:
239 * NODE_NAME = <Test POD Name>
240 * BUILD_TAG = <Jenkins Build Tag>
241 * CI_DEBUG = <DebugTraceValue>
245 * <Test POD Name> = Symbolic name of the POD where the tests are run.
246 Visible in test results files, which are stored
247 to the database. This option is only used when
248 tests are activated under Jenkins CI control.
249 It indicates the POD/hardware where the test has
250 been run. If not specified, then the POD name is
251 defined as "Unknown" by default.
252 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
253 * <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
254 Visible in test results files, which are stored
255 to the database. This option is only set when
256 tests are activated under Jenkins CI control.
257 It enables the correlation of test results,
258 which are independently pushed to the results database
259 from different Jenkins jobs.
260 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
261 * <DebugTraceValue> = "true" or "false"
262 Default = "false", if not specified
263 If "true" is specified, then additional debug trace
264 text can be sent to the test results file / log files
265 and also to the standard console output.
268 Openstack credentials
269 =====================
270 OpenStack credentials are mandatory and must be provided to Functest.
271 When running the command "functest env prepare", the framework will
272 automatically look for the Openstack credentials file
273 "/home/opnfv/functest/conf/openstack.creds" and will exit with
274 error if it is not present or is empty.
276 There are 2 ways to provide that file:
278 * by using a Docker volume with -v option when creating the Docker container.
279 This is referred to in docker documentation as "Bind Mounting".
280 See the usage of this parameter in the following chapter.
281 * or creating manually the file '/home/opnfv/functest/conf/openstack.creds'
282 inside the running container and pasting the credentials in it. Consult
283 your installer guide for further details. This is however not
284 instructed in this document.
286 In proxified environment you may need to change the credentials file.
287 There are some tips in chapter: `Proxy support`_
291 If you need to connect to a server that is TLS-enabled (the auth URL
292 begins with "https") and it uses a certificate from a private CA or a
293 self-signed certificate, then you will need to specify the path to an
294 appropriate CA certificate to use, to validate the server certificate
295 with the environment variable OS_CACERT::
298 /etc/ssl/certs/ca.crt
300 However, this certificate does not exist in the container by default.
301 It has to be copied manually from the OpenStack deployment. This can be
304 #. Create manually that file and copy the contents from the OpenStack
306 #. (Recommended) Add the file using a Docker volume when starting the
309 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
311 You might need to export OS_CACERT environment variable inside the
314 export OS_CACERT=/etc/ssl/certs/ca.crt
316 Certificate verification can be turned off using OS_INSECURE=true. For
317 example, Fuel uses self-signed cacerts by default, so an pre step would
320 export OS_INSECURE=true
322 Functest docker container directory structure
323 =============================================
324 Inside the Functest docker container, the following directory structure
325 should now be in place::
347 Underneath the '/home/opnfv/functest' directory, the Functest docker container
348 includes two main directories:
350 * The **conf** directory stores configuration files (e.g. the
351 OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
352 * the **results** directory stores some temporary result log files
354 src and repos directories are used to host third party code used for the tests.
356 The functest code is under /usr/lib/python2.7/site-packages/functest
357 The structure can be described as follows::
367 | |-- Dockerfile.aarch64.patch
369 | |-- config_install_env.sh
371 | |-- docker_remote_api
376 | `-- thirdparty-requirements.txt
381 | | `-- release-notes
403 │  |-- check_deployment.py
404 │  |-- config_aarch64_patch.yaml
405 │  |-- config_functest.yaml
406 │  |-- config_patch.yaml
407 │  |-- download_images.sh
408 │  |-- installer_params.yaml
409 │  |-- logging.ini
410 │  |-- prepare_env.py
411 │  |-- rally_aarch64_patch.conf
412 │  |-- run_tests.py
413 │  |-- testcases.yaml
414 │  |-- tier_builder.py
415 │  `-- tier_handler.py
417 │  |-- cli_base.py
419 │  │  |-- cli_env.py
420 │  │  |-- cli_os.py
421 │  │  |-- cli_testcase.py
422 │  │  `-- cli_tier.py
423 │  |-- functest-complete.sh
425 │  |-- feature.py
426 │  |-- testcase.py
431 │  `-- energy.pyc
434 │  │  |-- orchestra.py
436 │  │  |-- rally
437 │  │  |-- refstack_client
438 │  │  |-- snaps
439 │  │  |-- tempest
440 │  │  `-- vping
442 │  │  `-- odl
456 │  |-- opnfv_tests
457 │  |-- test_utils.py
465 | |-- functest_utils.py
466 | |-- functest_vacation.py
467 | |-- openstack_clean.py
468 | |-- openstack_snapshot.py
469 | |-- openstack_tacker.py
470 | `-- openstack_utils.py
474 |-- test-requirements.txt
476 |-- upper-constraints.txt
478 (Note: All *.pyc files removed from above list for brevity...)
480 We may distinguish several directories, the first level has 5 directories:
482 * **api**: This directory is dedicated to the API (framework) documentations.
483 * **commons**: This directory is dedicated for storage of traffic
484 profile or any other test inputs that could be reused by any test
486 * **docker**: This directory includes the needed files and tools to
487 build the Functest Docker image.
488 * **docs**: This directory includes documentation: Release Notes,
489 User Guide, Configuration Guide and Developer Guide.
490 * **functest**: This directory contains all the code needed to run
491 functest internal cases and OPNFV onboarded feature or VNF test cases.
493 Functest directory has 7 sub-directories:
494 * **api**: This directory is dedicated for the internal Functest API and the
495 API (framework) documentations.
496 * **ci**: This directory contains test structure definition files
497 (e.g <filename>.yaml) and bash shell/python scripts used to
498 configure and execute Functional tests. The test execution script
499 can be executed under the control of Jenkins CI jobs.
500 * **cli**: This directory holds the python based Functest CLI utility
501 source code, which is based on the Python 'click' framework.
502 * **core**: This directory holds the python based Functest core
503 source code. Three abstraction classes have been created to ease
504 the integration of internal, feature or vnf cases.
505 * **opnfv_tests**: This directory includes the scripts required by
506 Functest internal test cases and other feature projects test cases.
507 * **tests**: This directory includes the functest unit tests
508 * **utils**: this directory holds Python source code for some general
509 purpose helper utilities, which testers can also re-use in their
510 own test code. See for an example the Openstack helper utility:
511 'openstack_utils.py'.
516 By default all the logs are put un /home/opnfv/functest/results/functest.log.
517 If you want to have more logs in console, you may edit the logging.ini file
519 Connect on the docker then edit the file located in
520 /usr/lib/python2.7/site-packages/functest/ci/logging.ini
522 Change wconsole to console in the desired module to get more traces.
528 You may also directly modify the python code or the configuration file (e.g.
529 testcases.yaml used to declare test constraints) under
530 /usr/lib/python2.7/site-packages/functest
538 When typing **exit** in the container prompt, this will cause exiting
539 the container and probably stopping it. When stopping a running Docker
540 container all the changes will be lost, there is a keyboard shortcut
541 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
542 reconnect to the running container **DO NOT** use the *run* command
543 again (since it will create a new container), use the *exec* or *attach*
546 docker ps # <check the container ID from the output>
547 docker exec -ti <CONTAINER_ID> /bin/bash
549 There are other useful Docker commands that might be needed to manage possible
550 issues with the containers.
552 List the running containers::
556 List all the containers including the stopped ones::
560 Start a stopped container named "FunTest"::
564 Attach to a running container named "StrikeTwo"::
566 docker attach StrikeTwo
568 It is useful sometimes to remove a container if there are some problems::
570 docker rm <CONTAINER_ID>
572 Use the *-f* option if the container is still running, it will force to
575 docker rm -f <CONTAINER_ID>
577 Check the Docker documentation [`dockerdocs`_] for more information.
580 Checking Openstack and credentials
581 ----------------------------------
582 It is recommended and fairly straightforward to check that Openstack
583 and credentials are working as expected.
585 Once the credentials are there inside the container, they should be
586 sourced before running any Openstack commands::
588 source /home/opnfv/functest/conf/openstack.creds
590 After this, try to run any OpenStack command to see if you get any
591 output, for instance::
595 This will return a list of the actual users in the OpenStack
596 deployment. In any other case, check that the credentials are sourced::
600 This command must show a set of environment variables starting with
603 OS_REGION_NAME=RegionOne
604 OS_USER_DOMAIN_NAME=Default
605 OS_PROJECT_NAME=admin
607 OS_IDENTITY_API_VERSION=3
608 OS_PASSWORD=da54c27ae0d10dfae5297e6f0d6be54ebdb9f58d0f9dfc
609 OS_AUTH_URL=http://10.1.0.9:5000/v3
612 OS_ENDPOINT_TYPE=internalURL
613 OS_INTERFACE=internalURL
615 OS_PROJECT_DOMAIN_NAME=Default
618 If the OpenStack command still does not show anything or complains
619 about connectivity issues, it could be due to an incorrect url given to
620 the OS_AUTH_URL environment variable. Check the deployment settings.
624 If your Jumphost node is operating behind a http proxy, then there are
625 2 places where some special actions may be needed to make operations
628 #. Initial installation of docker engine First, try following the
629 official Docker documentation for Proxy settings. Some issues were
630 experienced on CentOS 7 based Jumphost. Some tips are documented
631 in section: `Docker Installation on CentOS behind http proxy`_
634 If that is the case, make sure the resolv.conf and the needed
635 http_proxy and https_proxy environment variables, as well as the
636 'no_proxy' environment variable are set correctly::
638 # Make double sure that the 'no_proxy=...' line in the
639 # 'openstack.creds' file is commented out first. Otherwise, the
640 # values set into the 'no_proxy' environment variable below will
641 # be ovewrwritten, each time the command
642 # 'source ~/functest/conf/openstack.creds' is issued.
645 sed -i 's/export no_proxy/#export no_proxy/' openstack.creds
646 source ./openstack.creds
648 # Next calculate some IP addresses for which http_proxy
649 # usage should be excluded:
651 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
653 adminURL_IP=$(openstack catalog show identity | \
654 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
656 export http_proxy="<your http proxy settings>"
657 export https_proxy="<your https proxy settings>"
658 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
660 # Ensure that "git" uses the http_proxy
661 # This may be needed if your firewall forbids SSL based git fetch
662 git config --global http.sslVerify True
663 git config --global http.proxy <Your http proxy settings>
665 For example, try to use the **nc** command from inside the functest
669 Connection to opnfv.org 80 port [tcp/http] succeeded!
672 Connection to opnfv.org 443 port [tcp/https] succeeded!
674 Note: In a Jumphost node based on the CentOS family OS, the **nc**
675 commands might not work. You can use the **curl** command instead.
677 curl http://www.opnfv.org:80
679 <HTML><HEAD><meta http-equiv="content-type"
684 curl https://www.opnfv.org:443
686 <HTML><HEAD><meta http-equiv="content-type"
691 (Ignore the content. If command returns a valid HTML page, it proves
694 Docker Installation on CentOS behind http proxy
695 -----------------------------------------------
696 This section is applicable for CentOS family OS on Jumphost which
697 itself is behind a proxy server. In that case, the instructions below
698 should be followed **before** installing the docker engine::
700 1) # Make a directory '/etc/systemd/system/docker.service.d'
701 # if it does not exist
702 sudo mkdir /etc/systemd/system/docker.service.d
704 2) # Create a file called 'env.conf' in that directory with
705 # the following contents:
707 EnvironmentFile=-/etc/sysconfig/docker
709 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
710 # with the following contents:
711 HTTP_PROXY="<Your http proxy settings>"
712 HTTPS_PROXY="<Your https proxy settings>"
713 http_proxy="${HTTP_PROXY}"
714 https_proxy="${HTTPS_PROXY}"
716 4) # Reload the daemon
717 systemctl daemon-reload
719 5) # Sanity check - check the following docker settings:
720 systemctl show docker | grep -i env
724 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
725 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
727 Now follow the instructions in [`Install Docker on CentOS`_] to download
728 and install the **docker-engine**. The instructions conclude with a
729 "test pull" of a sample "Hello World" docker container. This should now
730 work with the above pre-requisite actions.
733 .. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
734 .. _`dockerdocs`: https://docs.docker.com/
735 .. _`dockerhub`: https://hub.docker.com/r/opnfv/functest/
736 .. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy
737 .. _`FunctestDockerTags`: https://hub.docker.com/r/opnfv/functest/tags/
738 .. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
739 .. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
740 .. _`images/CentOS-7-x86_64-GenericCloud.qcow2` http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
741 .. _`images/cirros-0.3.5-x86_64-disk.img` https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img
742 .. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img` https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2