1 .. SPDX-License-Identifier: CC-BY-4.0
3 Installation and configuration
4 ==============================
6 Alpine containers have been introduced in Euphrates.
7 Alpine allows Functest testing in several very light containers and thanks to
8 the refactoring on dependency management should allow the creation of light and
9 fully customized docker images.
12 Functest Dockers for OpenStack deployment
13 -----------------------------------------
14 Docker images are available on the dockerhub:
17 * opnfv/functest-healthcheck
18 * opnfv/functest-smoke
19 * opnfv/functest-benchmarking
20 * opnfv/functest-features
21 * opnfv/functest-components
25 Preparing your environment
26 ^^^^^^^^^^^^^^^^^^^^^^^^^^
31 DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario
32 NAMESERVER=XXX # if not 8.8.8.8
34 See section on environment variables for details.
38 export OS_AUTH_URL=XXX
39 export OS_USER_DOMAIN_NAME=XXX
40 export OS_PROJECT_DOMAIN_NAME=XXX
41 export OS_USERNAME=XXX
42 export OS_PROJECT_NAME=XXX
43 export OS_PASSWORD=XXX
44 export OS_IDENTITY_API_VERSION=3
46 See section on OpenStack credentials for details.
48 Create a directory for the different images (attached as a Docker volume)::
50 mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/fraser | bash -s -- images && ls -1 images/*
52 images/CentOS-7-aarch64-GenericCloud.qcow2
53 images/CentOS-7-aarch64-GenericCloud.qcow2.xz
54 images/CentOS-7-x86_64-GenericCloud.qcow2
55 images/cirros-0.4.0-x86_64-disk.img
56 images/cirros-0.4.0-x86_64-lxc.tar.gz
57 images/cloudify-manager-premium-4.0.1.qcow2
58 images/shaker-image-arm64.qcow2
59 images/shaker-image.qcow
60 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
61 images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
62 images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
65 Testing healthcheck suite
66 ^^^^^^^^^^^^^^^^^^^^^^^^^
68 Run healthcheck suite::
70 sudo docker run --env-file env \
71 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
72 -v $(pwd)/images:/home/opnfv/functest/images \
73 opnfv/functest-healthcheck
75 Results shall be displayed as follows::
77 +----------------------------+------------------+---------------------+------------------+----------------+
78 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
79 +----------------------------+------------------+---------------------+------------------+----------------+
80 | connection_check | functest | healthcheck | 00:09 | PASS |
81 | tenantnetwork1 | functest | healthcheck | 00:14 | PASS |
82 | tenantnetwork2 | functest | healthcheck | 00:11 | PASS |
83 | vmready1 | functest | healthcheck | 00:19 | PASS |
84 | vmready2 | functest | healthcheck | 00:16 | PASS |
85 | singlevm1 | functest | healthcheck | 00:41 | PASS |
86 | singlevm2 | functest | healthcheck | 00:36 | PASS |
87 | vping_ssh | functest | healthcheck | 00:46 | PASS |
88 | vping_userdata | functest | healthcheck | 00:41 | PASS |
89 | cinder_test | functest | healthcheck | 01:18 | PASS |
90 | api_check | functest | healthcheck | 10:33 | PASS |
91 | snaps_health_check | functest | healthcheck | 00:44 | PASS |
92 | odl | functest | healthcheck | 00:00 | SKIP |
93 +----------------------------+------------------+---------------------+------------------+----------------+
95 NOTE: the duration is a reference and it might vary depending on your SUT.
102 sudo docker run --env-file env \
103 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
104 -v $(pwd)/images:/home/opnfv/functest/images \
107 Results shall be displayed as follows::
109 +------------------------------------+------------------+---------------+------------------+----------------+
110 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
111 +------------------------------------+------------------+---------------+------------------+----------------+
112 | tempest_smoke | functest | smoke | 04:24 | PASS |
113 | neutron-tempest-plugin-api | functest | smoke | 10:43 | PASS |
114 | rally_sanity | functest | smoke | 23:16 | PASS |
115 | refstack_defcore | functest | smoke | 06:30 | PASS |
116 | patrole | functest | smoke | 02:08 | PASS |
117 | snaps_smoke | functest | smoke | 00:00 | SKIP |
118 | neutron_trunk | functest | smoke | 02:09 | PASS |
119 | networking-bgpvpn | functest | smoke | 00:00 | SKIP |
120 | networking-sfc | functest | smoke | 00:00 | SKIP |
121 | barbican | functest | smoke | 00:00 | SKIP |
122 +------------------------------------+------------------+---------------+------------------+----------------+
124 Note: if the scenario does not support some tests, they are indicated as SKIP.
125 See User guide for details.
127 Testing benchmarking suite
128 ^^^^^^^^^^^^^^^^^^^^^^^^^^
130 Run benchmarking suite::
132 sudo docker run --env-file env \
133 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
134 -v $(pwd)/images:/home/opnfv/functest/images \
135 opnfv/functest-benchmarking
137 Results shall be displayed as follows::
139 +-------------------+------------------+----------------------+------------------+----------------+
140 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
141 +-------------------+------------------+----------------------+------------------+----------------+
142 | vmtp | functest | benchmarking | 18:43 | PASS |
143 | shaker | functest | benchmarking | 29:45 | PASS |
144 +-------------------+------------------+----------------------+------------------+----------------+
146 Note: if the scenario does not support some tests, they are indicated as SKIP.
147 See User guide for details.
149 Testing features suite
150 ^^^^^^^^^^^^^^^^^^^^^^
154 sudo docker run --env-file env \
155 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
156 -v $(pwd)/images:/home/opnfv/functest/images \
157 opnfv/functest-features
159 Results shall be displayed as follows::
161 +-----------------------------+------------------------+------------------+------------------+----------------+
162 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
163 +-----------------------------+------------------------+------------------+------------------+----------------+
164 | doctor-notification | doctor | features | 00:00 | SKIP |
165 | bgpvpn | sdnvpn | features | 00:00 | SKIP |
166 | functest-odl-sfc | sfc | features | 00:00 | SKIP |
167 | barometercollectd | barometer | features | 00:00 | SKIP |
168 | fds | fastdatastacks | features | 00:00 | SKIP |
169 | vgpu | functest | features | 00:00 | SKIP |
170 | stor4nfv_os | stor4nfv | features | 00:00 | SKIP |
171 +-----------------------------+------------------------+------------------+------------------+----------------+
173 Note: if the scenario does not support some tests, they are indicated as SKIP.
174 See User guide for details.
176 Testing components suite
177 ^^^^^^^^^^^^^^^^^^^^^^^^
179 Run components suite::
181 sudo docker run --env-file env \
182 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
183 -v $(pwd)/images:/home/opnfv/functest/images \
184 opnfv/functest-components
186 Results shall be displayed as follows::
188 +--------------------------+------------------+--------------------+------------------+----------------+
189 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
190 +--------------------------+------------------+--------------------+------------------+----------------+
191 | tempest_full | functest | components | 53:40 | FAIL |
192 | tempest_scenario | functest | components | 18:50 | PASS |
193 | rally_full | functest | components | 167:13 | PASS |
194 +--------------------------+------------------+--------------------+------------------+----------------+
201 sudo docker run --env-file env \
202 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
203 -v $(pwd)/images:/home/opnfv/functest/images \
206 Results shall be displayed as follows::
208 +----------------------+------------------+--------------+------------------+----------------+
209 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
210 +----------------------+------------------+--------------+------------------+----------------+
211 | cloudify | functest | vnf | 04:05 | PASS |
212 | cloudify_ims | functest | vnf | 24:07 | PASS |
213 | heat_ims | functest | vnf | 18:15 | PASS |
214 | vyos_vrouter | functest | vnf | 15:48 | PASS |
215 | juju_epc | functest | vnf | 29:38 | PASS |
216 +----------------------+------------------+--------------+------------------+----------------+
218 Functest Dockers for Kubernetes deployment
219 ------------------------------------------
220 Docker images are available on the dockerhub:
222 * opnfv/functest-kubernetes-core
223 * opnfv/functest-kubernetest-healthcheck
224 * opnfv/functest-kubernetest-smoke
225 * opnfv/functest-kubernetest-features
227 Preparing your environment
228 ^^^^^^^^^^^^^^^^^^^^^^^^^^
232 DEPLOY_SCENARIO=k8s-XXX
234 Testing healthcheck suite
235 ^^^^^^^^^^^^^^^^^^^^^^^^^
237 Run healthcheck suite::
239 sudo docker run -it --env-file env \
240 -v $(pwd)/config:/root/.kube/config \
241 opnfv/functest-kubernetes-healthcheck
243 A config file in the current dir 'config' is also required, which should be
244 volume mapped to ~/.kube/config inside kubernetes container.
246 Results shall be displayed as follows::
248 +-------------------+------------------+---------------------+------------------+----------------+
249 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
250 +-------------------+------------------+---------------------+------------------+----------------+
251 | k8s_smoke | functest | healthcheck | 02:27 | PASS |
252 +-------------------+------------------+---------------------+------------------+----------------+
259 sudo docker run -it --env-file env \
260 -v $(pwd)/config:/root/.kube/config \
261 opnfv/functest-kubernetes-smoke
263 Results shall be displayed as follows::
265 +-------------------------+------------------+---------------+------------------+----------------+
266 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
267 +-------------------------+------------------+---------------+------------------+----------------+
268 | k8s_conformance | functest | smoke | 57:14 | PASS |
269 +-------------------------+------------------+---------------+------------------+----------------+
271 Testing features suite
272 ^^^^^^^^^^^^^^^^^^^^^^
276 sudo docker run -it --env-file env \
277 -v $(pwd)/config:/root/.kube/config \
278 opnfv/functest-kubernetes-features
280 Results shall be displayed as follows::
282 +----------------------+------------------+------------------+------------------+----------------+
283 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
284 +----------------------+------------------+------------------+------------------+----------------+
285 | stor4nfv_k8s | stor4nfv | stor4nfv | 00:00 | SKIP |
286 | clover_k8s | clover | clover | 00:00 | SKIP |
287 +----------------------+------------------+------------------+------------------+----------------+
289 Environment variables
290 =====================
292 Several environement variables may be specified:
294 * INSTALLER_IP=<Specific IP Address>
295 * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
296 * NAMESERVER=XXX # if not 8.8.8.8
297 * VOLUME_DEVICE_NAME=XXX # if not vdb
298 * EXTERNAL_NETWORK=XXX # if not first network with router:external=True
299 * NEW_USER_ROLE=XXX # if not member
301 INSTALLER_IP is required by Barometer in order to access the installer node and
304 The format for the DEPLOY_SCENARIO env variable can be described as follows:
305 * vim: (os|k8s) = OpenStack or Kubernetes
306 * controller is one of ( nosdn | odl )
307 * nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
308 * ha_mode (high availability) is one of ( ha | noha )
310 If several features are pertinent then use the underscore character '_' to
311 separate each feature (e.g. ovs_kvm). 'nofeature' indicates that no OPNFV
314 The list of supported scenarios per release/installer is indicated in the
317 **NOTE:** The scenario name is mainly used to automatically detect
318 if a test suite is runnable or not (e.g. it will prevent ODL test suite to be
319 run on 'nosdn' scenarios). If not set, Functest will try to run the default
320 test cases that might not include SDN controller or a specific feature.
322 **NOTE:** An HA scenario means that 3 OpenStack controller nodes are
323 deployed. It does not necessarily mean that the whole system is HA. See
324 installer release notes for details.
326 Finally, three additional environment variables can also be passed in
327 to the Functest Docker Container, using the -e
328 "<EnvironmentVariable>=<Value>" mechanism. The first two parameters are
329 only relevant to Jenkins CI invoked testing and **should not be used**
330 when performing manual test scenarios:
332 * INSTALLER_TYPE=(apex|compass|daisy|fuel)
333 * NODE_NAME=<Test POD Name>
334 * BUILD_TAG=<Jenkins Build Tag>
338 * <Test POD Name> = Symbolic name of the POD where the tests are run.
339 Visible in test results files, which are stored
340 to the database. This option is only used when
341 tests are activated under Jenkins CI control.
342 It indicates the POD/hardware where the test has
343 been run. If not specified, then the POD name is
344 defined as "Unknown" by default.
345 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
346 * <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
347 Visible in test results files, which are stored
348 to the database. This option is only set when
349 tests are activated under Jenkins CI control.
350 It enables the correlation of test results,
351 which are independently pushed to the results database
352 from different Jenkins jobs.
353 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
356 Openstack credentials
357 =====================
358 OpenStack credentials are mandatory and must be provided to Functest.
359 When running the command "functest env prepare", the framework will
360 automatically look for the Openstack credentials file
361 "/home/opnfv/functest/conf/env_file" and will exit with
362 error if it is not present or is empty.
364 There are 2 ways to provide that file:
366 * by using a Docker volume with -v option when creating the Docker container.
367 This is referred to in docker documentation as "Bind Mounting".
368 See the usage of this parameter in the following chapter.
369 * or creating manually the file '/home/opnfv/functest/conf/env_file'
370 inside the running container and pasting the credentials in it. Consult
371 your installer guide for further details. This is however not
372 instructed in this document.
374 In proxified environment you may need to change the credentials file.
375 There are some tips in chapter: `Proxy support`_
379 If you need to connect to a server that is TLS-enabled (the auth URL
380 begins with "https") and it uses a certificate from a private CA or a
381 self-signed certificate, then you will need to specify the path to an
382 appropriate CA certificate to use, to validate the server certificate
383 with the environment variable OS_CACERT::
386 /etc/ssl/certs/ca.crt
388 However, this certificate does not exist in the container by default.
389 It has to be copied manually from the OpenStack deployment. This can be
392 #. Create manually that file and copy the contents from the OpenStack
394 #. (Recommended) Add the file using a Docker volume when starting the
397 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
399 You might need to export OS_CACERT environment variable inside the
402 export OS_CACERT=/etc/ssl/certs/ca.crt
404 Certificate verification can be turned off using OS_INSECURE=true. For
405 example, Fuel uses self-signed cacerts by default, so an pre step would
408 export OS_INSECURE=true
413 By default all the logs are put un /home/opnfv/functest/results/functest.log.
414 If you want to have more logs in console, you may edit the logging.ini file
416 Connect on the docker then edit the file located in
417 /usr/lib/python2.7/site-packages/xtesting/ci/logging.ini
419 Change wconsole to console in the desired module to get more traces.
425 You may also directly modify the python code or the configuration file (e.g.
426 testcases.yaml used to declare test constraints) under
427 /usr/lib/python2.7/site-packages/xtesting and
428 /usr/lib/python2.7/site-packages/functest
436 When typing **exit** in the container prompt, this will cause exiting
437 the container and probably stopping it. When stopping a running Docker
438 container all the changes will be lost, there is a keyboard shortcut
439 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
440 reconnect to the running container **DO NOT** use the *run* command
441 again (since it will create a new container), use the *exec* or *attach*
444 docker ps # <check the container ID from the output>
445 docker exec -ti <CONTAINER_ID> /bin/bash
447 There are other useful Docker commands that might be needed to manage possible
448 issues with the containers.
450 List the running containers::
454 List all the containers including the stopped ones::
458 Start a stopped container named "FunTest"::
462 Attach to a running container named "StrikeTwo"::
464 docker attach StrikeTwo
466 It is useful sometimes to remove a container if there are some problems::
468 docker rm <CONTAINER_ID>
470 Use the *-f* option if the container is still running, it will force to
473 docker rm -f <CONTAINER_ID>
475 Check the Docker documentation [`dockerdocs`_] for more information.
478 Checking Openstack and credentials
479 ----------------------------------
480 It is recommended and fairly straightforward to check that Openstack
481 and credentials are working as expected.
483 Once the credentials are there inside the container, they should be
484 sourced before running any Openstack commands::
486 source /home/opnfv/functest/conf/env_file
488 After this, try to run any OpenStack command to see if you get any
489 output, for instance::
493 This will return a list of the actual users in the OpenStack
494 deployment. In any other case, check that the credentials are sourced::
498 This command must show a set of environment variables starting with
501 OS_REGION_NAME=RegionOne
502 OS_USER_DOMAIN_NAME=Default
503 OS_PROJECT_NAME=admin
505 OS_IDENTITY_API_VERSION=3
506 OS_PASSWORD=da54c27ae0d10dfae5297e6f0d6be54ebdb9f58d0f9dfc
507 OS_AUTH_URL=http://10.1.0.9:5000/v3
510 OS_ENDPOINT_TYPE=internalURL
511 OS_INTERFACE=internalURL
513 OS_PROJECT_DOMAIN_NAME=Default
516 If the OpenStack command still does not show anything or complains
517 about connectivity issues, it could be due to an incorrect url given to
518 the OS_AUTH_URL environment variable. Check the deployment settings.
524 If your Jumphost node is operating behind a http proxy, then there are
525 2 places where some special actions may be needed to make operations
528 #. Initial installation of docker engine First, try following the
529 official Docker documentation for Proxy settings. Some issues were
530 experienced on CentOS 7 based Jumphost. Some tips are documented
531 in section: :ref:`Docker Installation on CentOS behind http proxy`
534 If that is the case, make sure the resolv.conf and the needed
535 http_proxy and https_proxy environment variables, as well as the
536 'no_proxy' environment variable are set correctly::
538 # Make double sure that the 'no_proxy=...' line in the
539 # 'env_file' file is commented out first. Otherwise, the
540 # values set into the 'no_proxy' environment variable below will
541 # be ovewrwritten, each time the command
542 # 'source ~/functest/conf/env_file' is issued.
545 sed -i 's/export no_proxy/#export no_proxy/' env_file
548 # Next calculate some IP addresses for which http_proxy
549 # usage should be excluded:
551 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
553 adminURL_IP=$(openstack catalog show identity | \
554 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
556 export http_proxy="<your http proxy settings>"
557 export https_proxy="<your https proxy settings>"
558 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
560 # Ensure that "git" uses the http_proxy
561 # This may be needed if your firewall forbids SSL based git fetch
562 git config --global http.sslVerify True
563 git config --global http.proxy <Your http proxy settings>
565 For example, try to use the **nc** command from inside the functest
569 Connection to opnfv.org 80 port [tcp/http] succeeded!
572 Connection to opnfv.org 443 port [tcp/https] succeeded!
574 Note: In a Jumphost node based on the CentOS family OS, the **nc**
575 commands might not work. You can use the **curl** command instead.
577 curl http://www.opnfv.org:80
579 <HTML><HEAD><meta http-equiv="content-type"
584 curl https://www.opnfv.org:443
586 <HTML><HEAD><meta http-equiv="content-type"
591 (Ignore the content. If command returns a valid HTML page, it proves
594 .. _`Docker Installation on CentOS behind http proxy`:
596 Docker Installation on CentOS behind http proxy
597 -----------------------------------------------
598 This section is applicable for CentOS family OS on Jumphost which
599 itself is behind a proxy server. In that case, the instructions below
600 should be followed **before** installing the docker engine::
602 1) # Make a directory '/etc/systemd/system/docker.service.d'
603 # if it does not exist
604 sudo mkdir /etc/systemd/system/docker.service.d
606 2) # Create a file called 'env.conf' in that directory with
607 # the following contents:
609 EnvironmentFile=-/etc/sysconfig/docker
611 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
612 # with the following contents:
613 HTTP_PROXY="<Your http proxy settings>"
614 HTTPS_PROXY="<Your https proxy settings>"
615 http_proxy="${HTTP_PROXY}"
616 https_proxy="${HTTPS_PROXY}"
618 4) # Reload the daemon
619 systemctl daemon-reload
621 5) # Sanity check - check the following docker settings:
622 systemctl show docker | grep -i env
626 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
627 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
629 Now follow the instructions in [`Install Docker on CentOS`_] to download
630 and install the **docker-engine**. The instructions conclude with a
631 "test pull" of a sample "Hello World" docker container. This should now
632 work with the above pre-requisite actions.
635 .. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
636 .. _`dockerdocs`: https://docs.docker.com/
637 .. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy
638 .. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
639 .. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
640 .. _`images/CentOS-7-x86_64-GenericCloud.qcow2`: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
641 .. _`images/cirros-0.4.0-x86_64-disk.img`: http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img
642 .. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img`: https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img