1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. SPDX-License-Identifier: CC-BY-4.0
4 Installation and configuration
5 ==============================
7 Alpine containers have been introduced in Euphrates.
8 Alpine allows Functest testing in several very light containers and thanks to
9 the refactoring on dependency management should allow the creation of light and
10 fully customized docker files.
12 It is still possible to use the monolithic functest opnfv/functest especially
13 for tests on Aarch64 architecture.
17 Docker files are available on the dockerhub:
20 * opnfv/functest-healthcheck
21 * opnfv/functest-smoke
22 * opnfv/functest-features
23 * opnfv/functest-components
25 * opnfv/functest-parser
26 * opnfv/functest-restapi
28 By default, we use the docker tag latest, but you may pull a tagged docker
31 The Functest docker container environment can -in principle- be also
32 used with non-OPNFV official installers (e.g. 'devstack'), with the
33 **disclaimer** that support for such environments is outside of the
34 scope and responsibility of the OPNFV project.
37 Preparing your environment
38 --------------------------
47 See section on environment variables for details.
51 export OS_AUTH_URL=XXX
52 export OS_USER_DOMAIN_NAME=XXX
53 export OS_PROJECT_DOMAIN_NAME=XXX
54 export OS_USERNAME=XXX
55 export OS_TENANT_NAME=XXX
56 export OS_PROJECT_NAME=XXX
57 export OS_PASSWORD=XXX
58 export OS_VOLUME_API_VERSION=XXX
59 export OS_IDENTITY_API_VERSION=XXX
60 export OS_IMAGE_API_VERSION=XXX
62 See section on OpenStack credentials for details.
64 Create a directory for the different images (included as volume)::
66 mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/*
68 images/CentOS-7-aarch64-GenericCloud.qcow2
69 images/CentOS-7-aarch64-GenericCloud.qcow2.xz
70 images/CentOS-7-x86_64-GenericCloud.qcow2
71 images/cirros-0.3.5-x86_64-disk.img
72 images/cirros-0.3.5-x86_64-lxc.tar.gz
73 images/cirros-d161201-aarch64-disk.img
74 images/cirros-d161201-aarch64-initramfs
75 images/cirros-d161201-aarch64-kernel
76 images/cloudify-manager-premium-4.0.1.qcow2
78 images/trusty-server-cloudimg-amd64-disk1.img
79 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
80 images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
81 images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
85 Testing healthcheck suite
86 --------------------------
88 Run healthcheck suite::
90 sudo docker run --env-file env \
91 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
92 -v $(pwd)/images:/home/opnfv/functest/images \
93 opnfv/functest-healthcheck
95 Results shall be displayed as follows::
97 +----------------------------+------------------+---------------------+------------------+----------------+
98 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
99 +----------------------------+------------------+---------------------+------------------+----------------+
100 | connection_check | functest | healthcheck | 00:02 | PASS |
101 | api_check | functest | healthcheck | 04:57 | PASS |
102 | snaps_health_check | functest | healthcheck | 00:51 | PASS |
103 +----------------------------+------------------+---------------------+------------------+----------------+
110 sudo docker run --env-file env \
111 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
112 -v $(pwd)/images:/home/opnfv/functest/images \
115 Results shall be displayed as follows::
117 +------------------------------+------------------+---------------+------------------+----------------+
118 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
119 +------------------------------+------------------+---------------+------------------+----------------+
120 | vping_ssh | functest | smoke | 01:19 | PASS |
121 | vping_userdata | functest | smoke | 01:56 | PASS |
122 | tempest_smoke_serial | functest | smoke | 26:30 | PASS |
123 | rally_sanity | functest | smoke | 19:42 | PASS |
124 | refstack_defcore | functest | smoke | 22:00 | PASS |
125 | snaps_smoke | functest | smoke | 41:14 | PASS |
126 | odl | functest | smoke | 00:16 | PASS |
127 | odl_netvirt | functest | smoke | 00:00 | SKIP |
128 | fds | functest | smoke | 00:00 | SKIP |
129 +------------------------------+------------------+---------------+------------------+----------------+
130 Note: if the scenario does not support some tests, they are indicated as SKIP.
131 See User guide for details.
133 Testing features suite
134 ----------------------
138 sudo docker run --env-file env \
139 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
140 -v $(pwd)/images:/home/opnfv/functest/images \
141 opnfv/functest-features
143 Results shall be displayed as follows::
145 +---------------------------+--------------------------+------------------+------------------+----------------+
146 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
147 +---------------------------+--------------------------+------------------+------------------+----------------+
148 | promise | promise | features | 00:00 | SKIP |
149 | bgpvpn | sdnvpn | features | 00:00 | SKIP |
150 | security_scan | securityscanning | features | 00:00 | SKIP |
151 | functest-odl-sfc | sfc | features | 00:00 | SKIP |
152 | domino-multinode | domino | features | 00:00 | SKIP |
153 | barometercollectd | barometer | features | 00:00 | SKIP |
154 +---------------------------+--------------------------+------------------+------------------+----------------+
155 Note: if the scenario does not support some tests, they are indicated as SKIP.
156 See User guide for details.
158 Testing components suite
159 ------------------------
161 Run components suite::
163 sudo docker run --env-file env \
164 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
165 -v $(pwd)/images:/home/opnfv/functest/images \
166 opnfv/functest-components
168 Results shall be displayed as follows::
170 +-------------------------------+------------------+--------------------+------------------+----------------+
171 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
172 +-------------------------------+------------------+--------------------+------------------+----------------+
173 | tempest_full_parallel | functest | components | 102:48 | PASS |
174 | rally_full | functest | components | 160:58 | PASS |
175 | tempest_custom | functest | components | 00:00 | SKIP |
176 +-------------------------------+------------------+--------------------+------------------+----------------+
183 sudo docker run --env-file env \
184 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
185 -v $(pwd)/images:/home/opnfv/functest/images \
188 Results shall be displayed as follows::
190 +---------------------------------+------------------+--------------+------------------+----------------+
191 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
192 +---------------------------------+------------------+--------------+------------------+----------------+
193 | cloudify_ims | functest | vnf | 21:25 | PASS |
194 | orchestra_openims | functest | vnf | 11:02 | FAIL |
195 | orchestra_clearwaterims | functest | vnf | 09:13 | FAIL |
196 | vyos_vrouter | functest | vnf | 00:00 | SKIP |
197 +---------------------------------+------------------+--------------+------------------+----------------+
200 Environment variables
201 =====================
203 Several environement variables may be specified:
204 * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid|osa)
205 * INSTALLER_IP=<Specific IP Address>
206 * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
209 INSTALLER IP may be required by some test cases like SFC or Barometer in order
210 to access the installer node and the deployment.
212 The format for the DEPLOY_SCENARIO env variable can be described as follows:
213 * vim: (os|k8s) = OpenStack or Kubernetes
214 * controller is one of ( nosdn | odl )
215 * nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
216 * ha_mode (high availability) is one of ( ha | noha )
218 If several features are pertinent then use the underscore character '_' to
219 separate each feature (e.g. ovs_kvm) 'nofeature' indicates no OPNFV feature is
222 The list of supported scenarios per release/installer is indicated in the
225 **NOTE:** The scenario name is mainly used to automatically detect
226 if a test suite is runnable or not (e.g. it will prevent ONOS test suite to be
227 run on ODL scenarios). If not set, Functest will try to run the default test
228 cases that might not include SDN controller or a specific
231 **NOTE:** A HA scenario means that 3 OpenStack controller nodes are
232 deployed. It does not necessarily mean that the whole system is HA. See
233 installer release notes for details.
235 Finally, three additional environment variables can also be passed in
236 to the Functest Docker Container, using the -e
237 "<EnvironmentVariable>=<Value>" mechanism. The first two parameters are
238 only relevant to Jenkins CI invoked testing and **should not be used**
239 when performing manual test scenarios:
241 * NODE_NAME = <Test POD Name>
242 * BUILD_TAG = <Jenkins Build Tag>
243 * CI_DEBUG = <DebugTraceValue>
247 * <Test POD Name> = Symbolic name of the POD where the tests are run.
248 Visible in test results files, which are stored
249 to the database. This option is only used when
250 tests are activated under Jenkins CI control.
251 It indicates the POD/hardware where the test has
252 been run. If not specified, then the POD name is
253 defined as "Unknown" by default.
254 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
255 * <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
256 Visible in test results files, which are stored
257 to the database. This option is only set when
258 tests are activated under Jenkins CI control.
259 It enables the correlation of test results,
261 are independently pushed to the results database
262 from different Jenkins jobs.
263 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
264 * <DebugTraceValue> = "true" or "false"
265 Default = "false", if not specified
266 If "true" is specified, then additional debug trace
267 text can be sent to the test results file / log files
268 and also to the standard console output.
271 Openstack credentials
272 =====================
273 OpenStack credentials are mandatory and must be provided to Functest.
274 When running the command "functest env prepare", the framework will
275 automatically look for the Openstack credentials file
276 "/home/opnfv/functest/conf/openstack.creds" and will exit with
277 error if it is not present or is empty.
279 There are 2 ways to provide that file:
281 * by using a Docker volume with -v option when creating the Docker container.
282 This is referred to in docker documentation as "Bind Mounting".
283 See the usage of this parameter in the following chapter.
284 * or creating manually the file '/home/opnfv/functest/conf/openstack.creds'
285 inside the running container and pasting the credentials in it. Consult
286 your installer guide for further details. This is however not
287 instructed in this document.
289 There is a default environment variable in the Functest container **$creds**
290 that points to the credentials absolute path to help the user with this task.
292 In proxified environment you may need to change the credentials file.
293 There are some tips in chapter: `Proxy support`_
297 If you need to connect to a server that is TLS-enabled (the auth URL
298 begins with "https") and it uses a certificate from a private CA or a
299 self-signed certificate, then you will need to specify the path to an
300 appropriate CA certificate to use, to validate the server certificate
301 with the environment variable OS_CACERT::
304 /etc/ssl/certs/ca.crt
306 However, this certificate does not exist in the container by default.
307 It has to be copied manually from the OpenStack deployment. This can be
310 #. Create manually that file and copy the contents from the OpenStack
312 #. (Recommended) Add the file using a Docker volume when starting the
315 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
317 You might need to export OS_CACERT environment variable inside the
320 export OS_CACERT=/etc/ssl/certs/ca.crt
322 Certificate verification can be turned off using OS_INSECURE=true. For
323 example, Fuel uses self-signed cacerts by default, so an pre step would
326 export OS_INSECURE=true
328 Functest docker container directory structure
329 =============================================
330 Inside the Functest docker container, the following directory structure
331 should now be in place::
353 Underneath the '/home/opnfv/functest' directory, the Functest docker container
354 includes two main directories:
356 * The **conf** directory stores configuration files (e.g. the
357 OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
358 * the **results** directory stores some temporary result log files
360 src and repos directories are used to host third party code used for the tests.
362 The functest code is under /usr/lib/python2.7/site-packages/functest
363 The structure can be described as follows::
373 | |-- Dockerfile.aarch64.patch
375 | |-- config_install_env.sh
377 | |-- docker_remote_api
382 | `-- thirdparty-requirements.txt
387 | | `-- release-notes
409 │  |-- check_deployment.py
410 │  |-- config_aarch64_patch.yaml
411 │  |-- config_functest.yaml
412 │  |-- config_patch.yaml
413 │  |-- download_images.sh
414 │  |-- installer_params.yaml
415 │  |-- logging.ini
416 │  |-- prepare_env.py
417 │  |-- rally_aarch64_patch.conf
418 │  |-- run_tests.py
419 │  |-- testcases.yaml
420 │  |-- tier_builder.py
421 │  `-- tier_handler.py
423 │  |-- cli_base.py
425 │  │  |-- cli_env.py
426 │  │  |-- cli_os.py
427 │  │  |-- cli_testcase.py
428 │  │  `-- cli_tier.py
429 │  |-- functest-complete.sh
431 │  |-- feature.py
432 │  |-- testcase.py
437 │  `-- energy.pyc
440 │  │  |-- orchestra.py
442 │  │  |-- rally
443 │  │  |-- refstack_client
444 │  │  |-- snaps
445 │  │  |-- tempest
446 │  │  `-- vping
448 │  │  `-- odl
462 │  |-- opnfv_tests
463 │  |-- test_utils.py
471 | |-- functest_utils.py
472 | |-- functest_vacation.py
473 | |-- openstack_clean.py
474 | |-- openstack_snapshot.py
475 | |-- openstack_tacker.py
476 | `-- openstack_utils.py
480 |-- test-requirements.txt
482 |-- upper-constraints.txt
484 (Note: All *.pyc files removed from above list for brevity...)
486 We may distinguish several directories, the first level has 5 directories:
488 * **api**: This directory is dedicated to the API (framework) documentations.
489 * **commons**: This directory is dedicated for storage of traffic
490 profile or any other test inputs that could be reused by any test
492 * **docker**: This directory includes the needed files and tools to
493 build the Functest Docker image.
494 * **docs**: This directory includes documentation: Release Notes,
495 User Guide, Configuration Guide and Developer Guide.
496 * **functest**: This directory contains all the code needed to run
497 functest internal cases and OPNFV onboarded feature or VNF test cases.
499 Functest directory has 7 sub-directories:
500 * **api**: This directory is dedicated for the internal Functest API and the
501 API (framework) documentations.
502 * **ci**: This directory contains test structure definition files
503 (e.g <filename>.yaml) and bash shell/python scripts used to
504 configure and execute Functional tests. The test execution script
505 can be executed under the control of Jenkins CI jobs.
506 * **cli**: This directory holds the python based Functest CLI utility
507 source code, which is based on the Python 'click' framework.
508 * **core**: This directory holds the python based Functest core
509 source code. Three abstraction classes have been created to ease
510 the integration of internal, feature or vnf cases.
511 * **opnfv_tests**: This directory includes the scripts required by
512 Functest internal test cases and other feature projects test cases.
513 * **tests**: This directory includes the functest unit tests
514 * **utils**: this directory holds Python source code for some general
515 purpose helper utilities, which testers can also re-use in their
516 own test code. See for an example the Openstack helper utility:
517 'openstack_utils.py'.
522 By default all the logs are put un /home/opnfv/functest/results/functest.log.
523 If you want to have more logs in console, you may edit the logging.ini file
525 Connect on the docker then edit the file located in
526 /usr/lib/python2.7/site-packages/functest/ci/logging.ini
528 Change wconsole to console in the desired module to get more traces.
534 You may also directly modify the python code or the configuration file (e.g.
535 testcases.yaml used to declare test constraints) under
536 /usr/lib/python2.7/site-packages/functest
544 When typing **exit** in the container prompt, this will cause exiting
545 the container and probably stopping it. When stopping a running Docker
546 container all the changes will be lost, there is a keyboard shortcut
547 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
548 reconnect to the running container **DO NOT** use the *run* command
549 again (since it will create a new container), use the *exec* or *attach*
552 docker ps # <check the container ID from the output>
553 docker exec -ti <CONTAINER_ID> /bin/bash
555 There are other useful Docker commands that might be needed to manage possible
556 issues with the containers.
558 List the running containers::
562 List all the containers including the stopped ones::
566 Start a stopped container named "FunTest"::
570 Attach to a running container named "StrikeTwo"::
572 docker attach StrikeTwo
574 It is useful sometimes to remove a container if there are some problems::
576 docker rm <CONTAINER_ID>
578 Use the *-f* option if the container is still running, it will force to
581 docker rm -f <CONTAINER_ID>
583 Check the Docker documentation [`dockerdocs`_] for more information.
586 Checking Openstack and credentials
587 ----------------------------------
588 It is recommended and fairly straightforward to check that Openstack
589 and credentials are working as expected.
591 Once the credentials are there inside the container, they should be
592 sourced before running any Openstack commands::
594 source /home/opnfv/functest/conf/openstack.creds
596 After this, try to run any OpenStack command to see if you get any
597 output, for instance::
601 This will return a list of the actual users in the OpenStack
602 deployment. In any other case, check that the credentials are sourced::
606 This command must show a set of environment variables starting with
609 OS_REGION_NAME=RegionOne
610 OS_USER_DOMAIN_NAME=Default
611 OS_PROJECT_NAME=admin
613 OS_IDENTITY_API_VERSION=3
614 OS_PASSWORD=da54c27ae0d10dfae5297e6f0d6be54ebdb9f58d0f9dfc
615 OS_AUTH_URL=http://10.1.0.9:5000/v3
618 OS_ENDPOINT_TYPE=internalURL
619 OS_INTERFACE=internalURL
621 OS_PROJECT_DOMAIN_NAME=Default
624 If the OpenStack command still does not show anything or complains
625 about connectivity issues, it could be due to an incorrect url given to
626 the OS_AUTH_URL environment variable. Check the deployment settings.
630 If your Jumphost node is operating behind a http proxy, then there are
631 2 places where some special actions may be needed to make operations
634 #. Initial installation of docker engine First, try following the
635 official Docker documentation for Proxy settings. Some issues were
636 experienced on CentOS 7 based Jumphost. Some tips are documented
637 in section: `Docker Installation on CentOS behind http proxy`_
640 If that is the case, make sure the resolv.conf and the needed
641 http_proxy and https_proxy environment variables, as well as the
642 'no_proxy' environment variable are set correctly::
644 # Make double sure that the 'no_proxy=...' line in the
645 # 'openstack.creds' file is commented out first. Otherwise, the
646 # values set into the 'no_proxy' environment variable below will
647 # be ovewrwritten, each time the command
648 # 'source ~/functest/conf/openstack.creds' is issued.
651 sed -i 's/export no_proxy/#export no_proxy/' openstack.creds
652 source ./openstack.creds
654 # Next calculate some IP addresses for which http_proxy
655 # usage should be excluded:
657 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
659 adminURL_IP=$(openstack catalog show identity | \
660 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
662 export http_proxy="<your http proxy settings>"
663 export https_proxy="<your https proxy settings>"
664 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
666 # Ensure that "git" uses the http_proxy
667 # This may be needed if your firewall forbids SSL based git fetch
668 git config --global http.sslVerify True
669 git config --global http.proxy <Your http proxy settings>
671 For example, try to use the **nc** command from inside the functest
675 Connection to opnfv.org 80 port [tcp/http] succeeded!
678 Connection to opnfv.org 443 port [tcp/https] succeeded!
680 Note: In a Jumphost node based on the CentOS family OS, the **nc**
681 commands might not work. You can use the **curl** command instead.
683 curl http://www.opnfv.org:80
685 <HTML><HEAD><meta http-equiv="content-type"
690 curl https://www.opnfv.org:443
692 <HTML><HEAD><meta http-equiv="content-type"
697 (Ignore the content. If command returns a valid HTML page, it proves
700 Docker Installation on CentOS behind http proxy
701 -----------------------------------------------
702 This section is applicable for CentOS family OS on Jumphost which
703 itself is behind a proxy server. In that case, the instructions below
704 should be followed **before** installing the docker engine::
706 1) # Make a directory '/etc/systemd/system/docker.service.d'
707 # if it does not exist
708 sudo mkdir /etc/systemd/system/docker.service.d
710 2) # Create a file called 'env.conf' in that directory with
711 # the following contents:
713 EnvironmentFile=-/etc/sysconfig/docker
715 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
716 # with the following contents:
717 HTTP_PROXY="<Your http proxy settings>"
718 HTTPS_PROXY="<Your https proxy settings>"
719 http_proxy="${HTTP_PROXY}"
720 https_proxy="${HTTPS_PROXY}"
722 4) # Reload the daemon
723 systemctl daemon-reload
725 5) # Sanity check - check the following docker settings:
726 systemctl show docker | grep -i env
730 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
731 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
733 Now follow the instructions in [`Install Docker on CentOS`_] to download
734 and install the **docker-engine**. The instructions conclude with a
735 "test pull" of a sample "Hello World" docker container. This should now
736 work with the above pre-requisite actions.
739 .. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
740 .. _`dockerdocs`: https://docs.docker.com/
741 .. _`dockerhub`: https://hub.docker.com/r/opnfv/functest/
742 .. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy
743 .. _`FunctestDockerTags`: https://hub.docker.com/r/opnfv/functest/tags/
744 .. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
745 .. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
746 .. _`images/CentOS-7-x86_64-GenericCloud.qcow2` http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
747 .. _`images/cirros-0.3.5-x86_64-disk.img` https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img
748 .. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img` https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2