1 .. SPDX-License-Identifier: CC-BY-4.0
3 Installation and configuration
4 ==============================
6 Alpine containers have been introduced in Euphrates.
7 Alpine allows Functest testing in several very light containers and thanks to
8 the refactoring on dependency management should allow the creation of light and
9 fully customized docker images.
14 Docker images are available on the dockerhub:
17 * opnfv/functest-healthcheck
18 * opnfv/functest-smoke
19 * opnfv/functest-features
20 * opnfv/functest-components
22 * opnfv/functest-parser
23 * opnfv/functest-restapi
25 The tag "opnfv-5.0.0" is the official release image in Euphrates, but you can
26 also pull "euphrates" tag as it is being maintained by Functest team and might
29 The Functest docker container environment can -in principle- be also
30 used with non-OPNFV official installers (e.g. 'devstack'), with the
31 **disclaimer** that support for such environments is outside of the
32 scope and responsibility of the OPNFV project.
35 Preparing your environment
36 --------------------------
44 See section on environment variables for details.
48 export OS_AUTH_URL=XXX
49 export OS_USER_DOMAIN_NAME=XXX
50 export OS_PROJECT_DOMAIN_NAME=XXX
51 export OS_USERNAME=XXX
52 export OS_TENANT_NAME=XXX
53 export OS_PROJECT_NAME=XXX
54 export OS_PASSWORD=XXX
55 export OS_VOLUME_API_VERSION=XXX
56 export OS_IDENTITY_API_VERSION=XXX
57 export OS_IMAGE_API_VERSION=XXX
59 See section on OpenStack credentials for details.
61 Create a directory for the different images (attached as a Docker volume)::
63 mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/euphrates | bash -s -- images && ls -1 images/*
65 images/CentOS-7-aarch64-GenericCloud.qcow2
66 images/CentOS-7-aarch64-GenericCloud.qcow2.xz
67 images/CentOS-7-x86_64-GenericCloud.qcow2
68 images/cirros-0.4.0-x86_64-disk.img
69 images/cirros-0.4.0-x86_64-lxc.tar.gz
70 images/cirros-d161201-aarch64-disk.img
71 images/cirros-d161201-aarch64-initramfs
72 images/cirros-d161201-aarch64-kernel
73 images/cloudify-manager-premium-4.0.1.qcow2
75 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
76 images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
77 images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
80 Testing healthcheck suite
81 --------------------------
83 Run healthcheck suite::
85 sudo docker run --env-file env \
86 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
87 -v $(pwd)/images:/home/opnfv/functest/images \
88 opnfv/functest-healthcheck
90 Results shall be displayed as follows::
92 +----------------------------+------------------+---------------------+------------------+----------------+
93 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
94 +----------------------------+------------------+---------------------+------------------+----------------+
95 | connection_check | functest | healthcheck | 00:06 | PASS |
96 | api_check | functest | healthcheck | 06:52 | PASS |
97 | snaps_health_check | functest | healthcheck | 00:42 | PASS |
98 +----------------------------+------------------+---------------------+------------------+----------------+
99 NOTE: the duration is a reference and it might vary depending on your SUT.
106 sudo docker run --env-file env \
107 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
108 -v $(pwd)/images:/home/opnfv/functest/images \
111 Results shall be displayed as follows::
113 +------------------------------+------------------+---------------+------------------+----------------+
114 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
115 +------------------------------+------------------+---------------+------------------+----------------+
116 | vping_ssh | functest | smoke | 00:45 | PASS |
117 | vping_userdata | functest | smoke | 00:36 | PASS |
118 | tempest_smoke_serial | functest | smoke | 14:04 | PASS |
119 | rally_sanity | functest | smoke | 23:59 | PASS |
120 | refstack_defcore | functest | smoke | 14:37 | FAIL |
121 | snaps_smoke | functest | smoke | 42:09 | PASS |
122 | odl | functest | smoke | 00:00 | SKIP |
123 | odl_netvirt | functest | smoke | 00:00 | SKIP |
124 +------------------------------+------------------+---------------+------------------+----------------+
125 Note: if the scenario does not support some tests, they are indicated as SKIP.
126 See User guide for details.
128 Testing features suite
129 ----------------------
133 sudo docker run --env-file env \
134 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
135 -v $(pwd)/images:/home/opnfv/functest/images \
136 opnfv/functest-features
138 Results shall be displayed as follows::
140 +-----------------------------+------------------------+------------------+------------------+----------------+
141 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
142 +-----------------------------+------------------------+------------------+------------------+----------------+
143 | doctor-notification | doctor | features | 00:00 | SKIP |
144 | bgpvpn | sdnvpn | features | 00:00 | SKIP |
145 | functest-odl-sfc | sfc | features | 00:00 | SKIP |
146 | barometercollectd | barometer | features | 00:00 | SKIP |
147 | fds | fastdatastacks | features | 00:00 | SKIP |
148 +-----------------------------+------------------------+------------------+------------------+----------------+
149 Note: if the scenario does not support some tests, they are indicated as SKIP.
150 See User guide for details.
152 Testing components suite
153 ------------------------
155 Run components suite::
157 sudo docker run --env-file env \
158 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
159 -v $(pwd)/images:/home/opnfv/functest/images \
160 opnfv/functest-components
162 Results shall be displayed as follows::
164 +-------------------------------+------------------+--------------------+------------------+----------------+
165 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
166 +-------------------------------+------------------+--------------------+------------------+----------------+
167 | tempest_full_parallel | functest | components | 52:42 | PASS |
168 | rally_full | functest | components | 114:22 | PASS |
169 +-------------------------------+------------------+--------------------+------------------+----------------+
176 sudo docker run --env-file env \
177 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
178 -v $(pwd)/images:/home/opnfv/functest/images \
181 Results shall be displayed as follows::
183 +---------------------------------+-------------------+--------------+------------------+----------------+
184 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
185 +---------------------------------+-------------------+--------------+------------------+----------------+
186 | cloudify_ims | functest | vnf | 28:49 | FAIL |
187 | vyos_vrouter | functest | vnf | 27:57 | FAIL |
188 | juju_epc | functest | vnf | 55:03 | PASS |
189 | orchestra_openims | orchestra | vnf | 00:00 | SKIP |
190 | orchestra_clearwaterims | orchestra | vnf | 00:00 | SKIP |
191 +---------------------------------+-------------------+--------------+------------------+----------------+
194 Environment variables
195 =====================
197 Several environement variables may be specified:
198 * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid)
199 * INSTALLER_IP=<Specific IP Address>
200 * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
203 INSTALLER_IP is required by Barometer in order to access the installer node and
206 The format for the DEPLOY_SCENARIO env variable can be described as follows:
207 * vim: (os|k8s) = OpenStack or Kubernetes
208 * controller is one of ( nosdn | odl )
209 * nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
210 * ha_mode (high availability) is one of ( ha | noha )
212 If several features are pertinent then use the underscore character '_' to
213 separate each feature (e.g. ovs_kvm). 'nofeature' indicates that no OPNFV
216 The list of supported scenarios per release/installer is indicated in the
219 **NOTE:** The scenario name is mainly used to automatically detect
220 if a test suite is runnable or not (e.g. it will prevent ODL test suite to be
221 run on 'nosdn' scenarios). If not set, Functest will try to run the default
222 test cases that might not include SDN controller or a specific feature.
224 **NOTE:** An HA scenario means that 3 OpenStack controller nodes are
225 deployed. It does not necessarily mean that the whole system is HA. See
226 installer release notes for details.
228 Finally, three additional environment variables can also be passed in
229 to the Functest Docker Container, using the -e
230 "<EnvironmentVariable>=<Value>" mechanism. The first two parameters are
231 only relevant to Jenkins CI invoked testing and **should not be used**
232 when performing manual test scenarios:
234 * NODE_NAME = <Test POD Name>
235 * BUILD_TAG = <Jenkins Build Tag>
239 * <Test POD Name> = Symbolic name of the POD where the tests are run.
240 Visible in test results files, which are stored
241 to the database. This option is only used when
242 tests are activated under Jenkins CI control.
243 It indicates the POD/hardware where the test has
244 been run. If not specified, then the POD name is
245 defined as "Unknown" by default.
246 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
247 * <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
248 Visible in test results files, which are stored
249 to the database. This option is only set when
250 tests are activated under Jenkins CI control.
251 It enables the correlation of test results,
252 which are independently pushed to the results database
253 from different Jenkins jobs.
254 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
257 Openstack credentials
258 =====================
259 OpenStack credentials are mandatory and must be provided to Functest.
260 When running the command "functest env prepare", the framework will
261 automatically look for the Openstack credentials file
262 "/home/opnfv/functest/conf/env_file" and will exit with
263 error if it is not present or is empty.
265 There are 2 ways to provide that file:
267 * by using a Docker volume with -v option when creating the Docker container.
268 This is referred to in docker documentation as "Bind Mounting".
269 See the usage of this parameter in the following chapter.
270 * or creating manually the file '/home/opnfv/functest/conf/env_file'
271 inside the running container and pasting the credentials in it. Consult
272 your installer guide for further details. This is however not
273 instructed in this document.
275 In proxified environment you may need to change the credentials file.
276 There are some tips in chapter: `Proxy support`_
280 If you need to connect to a server that is TLS-enabled (the auth URL
281 begins with "https") and it uses a certificate from a private CA or a
282 self-signed certificate, then you will need to specify the path to an
283 appropriate CA certificate to use, to validate the server certificate
284 with the environment variable OS_CACERT::
287 /etc/ssl/certs/ca.crt
289 However, this certificate does not exist in the container by default.
290 It has to be copied manually from the OpenStack deployment. This can be
293 #. Create manually that file and copy the contents from the OpenStack
295 #. (Recommended) Add the file using a Docker volume when starting the
298 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
300 You might need to export OS_CACERT environment variable inside the
303 export OS_CACERT=/etc/ssl/certs/ca.crt
305 Certificate verification can be turned off using OS_INSECURE=true. For
306 example, Fuel uses self-signed cacerts by default, so an pre step would
309 export OS_INSECURE=true
311 Functest docker container directory structure
312 =============================================
313 Inside the Functest docker container, the following directory structure
314 should now be in place::
336 Underneath the '/home/opnfv/functest' directory, the Functest docker container
337 includes two main directories:
339 * The **conf** directory stores configuration files (e.g. the
340 OpenStack creds are stored in path '/home/opnfv/functest/conf/env_file'),
341 * the **results** directory stores some temporary result log files
343 src and repos directories are used to host third party code used for the tests.
345 The structure of functest repo can be described as follows::
362 | |-- thirdparty-requirements.txt
367 | | `-- release-notes
396 │ |-- check_deployment.py
397 │ |-- config_aarch64_patch.yaml
398 │ |-- config_functest.yaml
399 │ |-- config_patch.yaml
400 │ |-- download_images.sh
402 │ |-- rally_aarch64_patch.conf
404 │ |-- testcases.yaml
405 │ |-- tier_builder.py
406 │ |-- tier_handler.py
410 │ │ |-- cli_env.py
411 │ │ |-- cli_os.py
412 │ │ |-- cli_testcase.py
413 │ │ |-- cli_tier.py
414 │ |-- functest-complete.sh
417 │ |-- robotframework.py
426 │ |-- refstack_client
445 │ |-- test_utils.py
453 | |-- functest_utils.py
454 | |-- functest_vacation.py
455 | |-- openstack_clean.py
456 | |-- openstack_tacker.py
457 | `-- openstack_utils.py
461 |-- test-requirements.txt
463 |-- upper-constraints.txt
465 (Note: All *.pyc files removed from above list for brevity...)
467 We may distinguish several directories, the first level has 5 directories:
469 * **api**: This directory is dedicated to the API (framework) documentations.
470 * **commons**: This directory is dedicated for storage of traffic profile or
471 any other test inputs that could be reused by any test project.
472 * **docker**: This directory includes the needed files and tools to
473 build the Functest Docker images.
474 * **docs**: This directory includes documentation: Release Notes,
475 User Guide, Configuration Guide and Developer Guide.
476 * **functest**: This directory contains all the code needed to run
477 functest internal cases and OPNFV onboarded feature or VNF test cases.
479 Functest directory has 7 sub-directories, which is located under
480 /usr/lib/python2.7/site-packages/functest:
482 * **api**: This directory is dedicated for the internal Functest API.
483 * **ci**: This directory contains test structure definition files
484 (e.g <filename>.yaml) and bash shell/python scripts used to
485 configure and execute Functional tests. The test execution script
486 can be executed under the control of Jenkins CI jobs.
487 * **cli**: This directory holds the python based Functest CLI utility
488 source code, which is based on the Python 'click' framework.
489 * **core**: This directory holds the python based Functest core
490 source code. Three abstraction classes have been created to ease
491 the integration of internal, feature or vnf cases.
492 * **opnfv_tests**: This directory includes the scripts required by
493 Functest internal test cases and other feature projects test cases.
494 * **tests**: This directory includes the functest unit tests.
495 * **utils**: this directory holds Python source code for some general
496 purpose helper utilities, which testers can also re-use in their
497 own test code. See for an example the Openstack helper utility:
498 'openstack_utils.py'.
503 By default all the logs are put un /home/opnfv/functest/results/functest.log.
504 If you want to have more logs in console, you may edit the logging.ini file
506 Connect on the docker then edit the file located in
507 /usr/lib/python2.7/site-packages/functest/ci/logging.ini
509 Change wconsole to console in the desired module to get more traces.
515 You may also directly modify the python code or the configuration file (e.g.
516 testcases.yaml used to declare test constraints) under
517 /usr/lib/python2.7/site-packages/functest
525 When typing **exit** in the container prompt, this will cause exiting
526 the container and probably stopping it. When stopping a running Docker
527 container all the changes will be lost, there is a keyboard shortcut
528 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
529 reconnect to the running container **DO NOT** use the *run* command
530 again (since it will create a new container), use the *exec* or *attach*
533 docker ps # <check the container ID from the output>
534 docker exec -ti <CONTAINER_ID> /bin/bash
536 There are other useful Docker commands that might be needed to manage possible
537 issues with the containers.
539 List the running containers::
543 List all the containers including the stopped ones::
547 Start a stopped container named "FunTest"::
551 Attach to a running container named "StrikeTwo"::
553 docker attach StrikeTwo
555 It is useful sometimes to remove a container if there are some problems::
557 docker rm <CONTAINER_ID>
559 Use the *-f* option if the container is still running, it will force to
562 docker rm -f <CONTAINER_ID>
564 Check the Docker documentation [`dockerdocs`_] for more information.
567 Checking Openstack and credentials
568 ----------------------------------
569 It is recommended and fairly straightforward to check that Openstack
570 and credentials are working as expected.
572 Once the credentials are there inside the container, they should be
573 sourced before running any Openstack commands::
575 source /home/opnfv/functest/conf/env_file
577 After this, try to run any OpenStack command to see if you get any
578 output, for instance::
582 This will return a list of the actual users in the OpenStack
583 deployment. In any other case, check that the credentials are sourced::
587 This command must show a set of environment variables starting with
590 OS_REGION_NAME=RegionOne
591 OS_USER_DOMAIN_NAME=Default
592 OS_PROJECT_NAME=admin
594 OS_IDENTITY_API_VERSION=3
595 OS_PASSWORD=da54c27ae0d10dfae5297e6f0d6be54ebdb9f58d0f9dfc
596 OS_AUTH_URL=http://10.1.0.9:5000/v3
599 OS_ENDPOINT_TYPE=internalURL
600 OS_INTERFACE=internalURL
602 OS_PROJECT_DOMAIN_NAME=Default
605 If the OpenStack command still does not show anything or complains
606 about connectivity issues, it could be due to an incorrect url given to
607 the OS_AUTH_URL environment variable. Check the deployment settings.
613 If your Jumphost node is operating behind a http proxy, then there are
614 2 places where some special actions may be needed to make operations
617 #. Initial installation of docker engine First, try following the
618 official Docker documentation for Proxy settings. Some issues were
619 experienced on CentOS 7 based Jumphost. Some tips are documented
620 in section: :ref:`Docker Installation on CentOS behind http proxy`
623 If that is the case, make sure the resolv.conf and the needed
624 http_proxy and https_proxy environment variables, as well as the
625 'no_proxy' environment variable are set correctly::
627 # Make double sure that the 'no_proxy=...' line in the
628 # 'env_file' file is commented out first. Otherwise, the
629 # values set into the 'no_proxy' environment variable below will
630 # be ovewrwritten, each time the command
631 # 'source ~/functest/conf/env_file' is issued.
634 sed -i 's/export no_proxy/#export no_proxy/' env_file
637 # Next calculate some IP addresses for which http_proxy
638 # usage should be excluded:
640 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
642 adminURL_IP=$(openstack catalog show identity | \
643 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
645 export http_proxy="<your http proxy settings>"
646 export https_proxy="<your https proxy settings>"
647 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
649 # Ensure that "git" uses the http_proxy
650 # This may be needed if your firewall forbids SSL based git fetch
651 git config --global http.sslVerify True
652 git config --global http.proxy <Your http proxy settings>
654 For example, try to use the **nc** command from inside the functest
658 Connection to opnfv.org 80 port [tcp/http] succeeded!
661 Connection to opnfv.org 443 port [tcp/https] succeeded!
663 Note: In a Jumphost node based on the CentOS family OS, the **nc**
664 commands might not work. You can use the **curl** command instead.
666 curl http://www.opnfv.org:80
668 <HTML><HEAD><meta http-equiv="content-type"
673 curl https://www.opnfv.org:443
675 <HTML><HEAD><meta http-equiv="content-type"
680 (Ignore the content. If command returns a valid HTML page, it proves
683 .. _`Docker Installation on CentOS behind http proxy`:
685 Docker Installation on CentOS behind http proxy
686 -----------------------------------------------
687 This section is applicable for CentOS family OS on Jumphost which
688 itself is behind a proxy server. In that case, the instructions below
689 should be followed **before** installing the docker engine::
691 1) # Make a directory '/etc/systemd/system/docker.service.d'
692 # if it does not exist
693 sudo mkdir /etc/systemd/system/docker.service.d
695 2) # Create a file called 'env.conf' in that directory with
696 # the following contents:
698 EnvironmentFile=-/etc/sysconfig/docker
700 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
701 # with the following contents:
702 HTTP_PROXY="<Your http proxy settings>"
703 HTTPS_PROXY="<Your https proxy settings>"
704 http_proxy="${HTTP_PROXY}"
705 https_proxy="${HTTPS_PROXY}"
707 4) # Reload the daemon
708 systemctl daemon-reload
710 5) # Sanity check - check the following docker settings:
711 systemctl show docker | grep -i env
715 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
716 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
718 Now follow the instructions in [`Install Docker on CentOS`_] to download
719 and install the **docker-engine**. The instructions conclude with a
720 "test pull" of a sample "Hello World" docker container. This should now
721 work with the above pre-requisite actions.
724 .. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
725 .. _`dockerdocs`: https://docs.docker.com/
726 .. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy
727 .. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
728 .. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
729 .. _`images/CentOS-7-x86_64-GenericCloud.qcow2`: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
730 .. _`images/cirros-0.4.0-x86_64-disk.img`: http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img
731 .. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img`: https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img