1 .. SPDX-License-Identifier: CC-BY-4.0
3 Installation and configuration
4 ==============================
6 Alpine containers have been introduced in Euphrates.
7 Alpine allows Functest testing in several very light containers and thanks to
8 the refactoring on dependency management should allow the creation of light and
9 fully customized docker images.
12 Functest Dockers for OpenStack deployment
13 -----------------------------------------
14 Docker images are available on the dockerhub:
17 * opnfv/functest-healthcheck
18 * opnfv/functest-smoke
19 * opnfv/functest-benchmarking
23 Preparing your environment
24 ^^^^^^^^^^^^^^^^^^^^^^^^^^
28 DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario
29 NAMESERVER=XXX # if not 8.8.8.8
30 EXTERNAL_NETWORK=XXX # if not first network with router:external=True
31 NEW_USER_ROLE=XXX # if not member
32 SDN_CONTROLLER_IP=XXX # if odl scenario
33 VOLUME_DEVICE_NAME=XXX # if not vdb
34 FLAVOR_EXTRA_SPECS=hw:mem_page_size:large # if fdio scenarios
36 See section on environment variables for details.
40 export OS_AUTH_URL=XXX
41 export OS_USER_DOMAIN_NAME=XXX
42 export OS_PROJECT_DOMAIN_NAME=XXX
43 export OS_USERNAME=XXX
44 export OS_PROJECT_NAME=XXX
45 export OS_PASSWORD=XXX
46 export OS_IDENTITY_API_VERSION=3
47 export OS_REGION_NAME=XXX
49 See section on OpenStack credentials for details.
51 Create a directory for the different images (attached as a Docker volume)::
53 mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/*
55 images/cirros-0.4.0-aarch64-disk.img
56 images/cirros-0.4.0-x86_64-disk.img
57 images/cloudify-docker-manager-community-19.01.24.tar
58 images/shaker-image-1.3.0+stretch.qcow2
59 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
60 images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
61 images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
62 images/vyos-1.1.8-amd64.qcow2
64 Testing healthcheck suite
65 ^^^^^^^^^^^^^^^^^^^^^^^^^
67 Run healthcheck suite::
69 sudo docker run --env-file env \
70 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
71 -v $(pwd)/images:/home/opnfv/functest/images \
72 opnfv/functest-healthcheck
74 Results shall be displayed as follows::
76 +--------------------------+------------------+---------------------+------------------+----------------+
77 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
78 +--------------------------+------------------+---------------------+------------------+----------------+
79 | connection_check | functest | healthcheck | 00:02 | PASS |
80 | tenantnetwork1 | functest | healthcheck | 00:06 | PASS |
81 | tenantnetwork2 | functest | healthcheck | 00:06 | PASS |
82 | vmready1 | functest | healthcheck | 00:06 | PASS |
83 | vmready2 | functest | healthcheck | 00:08 | PASS |
84 | singlevm1 | functest | healthcheck | 00:28 | PASS |
85 | singlevm2 | functest | healthcheck | 00:25 | PASS |
86 | vping_ssh | functest | healthcheck | 00:36 | PASS |
87 | vping_userdata | functest | healthcheck | 00:34 | PASS |
88 | cinder_test | functest | healthcheck | 01:03 | PASS |
89 | tempest_smoke | functest | healthcheck | 05:13 | PASS |
90 | odl | functest | healthcheck | 00:00 | SKIP |
91 +--------------------------+------------------+---------------------+------------------+----------------+
93 NOTE: the duration is a reference and it might vary depending on your SUT.
100 sudo docker run --env-file env \
101 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
102 -v $(pwd)/images:/home/opnfv/functest/images \
105 Results shall be displayed as follows::
107 +------------------------------------+------------------+---------------+------------------+----------------+
108 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
109 +------------------------------------+------------------+---------------+------------------+----------------+
110 | neutron-tempest-plugin-api | functest | smoke | 09:12 | PASS |
111 | rally_sanity | functest | smoke | 16:29 | PASS |
112 | refstack_compute | functest | smoke | 06:25 | PASS |
113 | refstack_object | functest | smoke | 01:54 | PASS |
114 | refstack_platform | functest | smoke | 06:52 | PASS |
115 | tempest_full | functest | smoke | 30:26 | PASS |
116 | tempest_scenario | functest | smoke | 09:23 | PASS |
117 | tempest_slow | functest | smoke | 24:42 | PASS |
118 | patrole | functest | smoke | 02:36 | PASS |
119 | barbican | functest | smoke | 02:13 | PASS |
120 | neutron_trunk | functest | smoke | 00:00 | SKIP |
121 | networking-bgpvpn | functest | smoke | 00:00 | SKIP |
122 | networking-sfc | functest | smoke | 00:00 | SKIP |
123 | octavia | functest | smoke | 00:00 | SKIP |
124 +------------------------------------+------------------+---------------+------------------+----------------+
126 Note: if the scenario does not support some tests, they are indicated as SKIP.
127 See User guide for details.
129 Testing benchmarking suite
130 ^^^^^^^^^^^^^^^^^^^^^^^^^^
132 Run benchmarking suite::
134 sudo docker run --env-file env \
135 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
136 -v $(pwd)/images:/home/opnfv/functest/images \
137 opnfv/functest-benchmarking
139 Results shall be displayed as follows::
141 +--------------------+------------------+----------------------+------------------+----------------+
142 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
143 +--------------------+------------------+----------------------+------------------+----------------+
144 | rally_full | functest | benchmarking | 92:16 | PASS |
145 | rally_jobs | functest | benchmarking | 18:49 | PASS |
146 | vmtp | functest | benchmarking | 15:28 | PASS |
147 | shaker | functest | benchmarking | 24:04 | PASS |
148 +--------------------+------------------+----------------------+------------------+----------------+
150 Note: if the scenario does not support some tests, they are indicated as SKIP.
151 See User guide for details.
158 sudo docker run --env-file env \
159 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
160 -v $(pwd)/images:/home/opnfv/functest/images \
163 Results shall be displayed as follows::
165 +----------------------+------------------+--------------+------------------+----------------+
166 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
167 +----------------------+------------------+--------------+------------------+----------------+
168 | cloudify | functest | vnf | 03:49 | PASS |
169 | cloudify_ims | functest | vnf | 24:20 | PASS |
170 | heat_ims | functest | vnf | 32:13 | PASS |
171 | vyos_vrouter | functest | vnf | 14:55 | PASS |
172 | juju_epc | functest | vnf | 41:24 | PASS |
173 +----------------------+------------------+--------------+------------------+----------------+
175 Functest Dockers for Kubernetes deployment
176 ------------------------------------------
177 Docker images are available on the dockerhub:
179 * opnfv/functest-kubernetes-core
180 * opnfv/functest-kubernetest-healthcheck
181 * opnfv/functest-kubernetest-smoke
183 Preparing your environment
184 ^^^^^^^^^^^^^^^^^^^^^^^^^^
188 DEPLOY_SCENARIO=k8s-XXX
190 Testing healthcheck suite
191 ^^^^^^^^^^^^^^^^^^^^^^^^^
193 Run healthcheck suite::
195 sudo docker run -it --env-file env \
196 -v $(pwd)/config:/root/.kube/config \
197 opnfv/functest-kubernetes-healthcheck
199 A config file in the current dir 'config' is also required, which should be
200 volume mapped to ~/.kube/config inside kubernetes container.
202 Results shall be displayed as follows::
204 +-------------------+------------------+---------------------+------------------+----------------+
205 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
206 +-------------------+------------------+---------------------+------------------+----------------+
207 | k8s_smoke | functest | healthcheck | 01:09 | PASS |
208 +-------------------+------------------+---------------------+------------------+----------------+
215 sudo docker run -it --env-file env \
216 -v $(pwd)/config:/root/.kube/config \
217 opnfv/functest-kubernetes-smoke
219 Results shall be displayed as follows::
221 +---------------------------+------------------+---------------+------------------+----------------+
222 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
223 +---------------------------+------------------+---------------+------------------+----------------+
224 | xrally_kubernetes | functest | smoke | 22:04 | PASS |
225 | k8s_conformance | functest | smoke | 173:48 | PASS |
226 +---------------------------+------------------+---------------+------------------+----------------+
228 Environment variables
229 =====================
231 Several environement variables may be specified:
233 * INSTALLER_IP=<Specific IP Address>
234 * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
235 * NAMESERVER=XXX # if not 8.8.8.8
236 * VOLUME_DEVICE_NAME=XXX # if not vdb
237 * EXTERNAL_NETWORK=XXX # if not first network with router:external=True
238 * NEW_USER_ROLE=XXX # if not member
240 INSTALLER_IP is required by Barometer in order to access the installer node and
243 The format for the DEPLOY_SCENARIO env variable can be described as follows:
244 * vim: (os|k8s) = OpenStack or Kubernetes
245 * controller is one of ( nosdn | odl )
246 * nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
247 * ha_mode (high availability) is one of ( ha | noha )
249 If several features are pertinent then use the underscore character '_' to
250 separate each feature (e.g. ovs_kvm). 'nofeature' indicates that no OPNFV
253 The list of supported scenarios per release/installer is indicated in the
256 **NOTE:** The scenario name is mainly used to automatically detect
257 if a test suite is runnable or not (e.g. it will prevent ODL test suite to be
258 run on 'nosdn' scenarios). If not set, Functest will try to run the default
259 test cases that might not include SDN controller or a specific feature.
261 **NOTE:** An HA scenario means that 3 OpenStack controller nodes are
262 deployed. It does not necessarily mean that the whole system is HA. See
263 installer release notes for details.
265 Finally, three additional environment variables can also be passed in
266 to the Functest Docker Container, using the -e
267 "<EnvironmentVariable>=<Value>" mechanism. The first two parameters are
268 only relevant to Jenkins CI invoked testing and **should not be used**
269 when performing manual test scenarios:
271 * INSTALLER_TYPE=(apex|compass|daisy|fuel)
272 * NODE_NAME=<Test POD Name>
273 * BUILD_TAG=<Jenkins Build Tag>
277 * <Test POD Name> = Symbolic name of the POD where the tests are run.
278 Visible in test results files, which are stored
279 to the database. This option is only used when
280 tests are activated under Jenkins CI control.
281 It indicates the POD/hardware where the test has
282 been run. If not specified, then the POD name is
283 defined as "Unknown" by default.
284 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
285 * <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
286 Visible in test results files, which are stored
287 to the database. This option is only set when
288 tests are activated under Jenkins CI control.
289 It enables the correlation of test results,
290 which are independently pushed to the results database
291 from different Jenkins jobs.
292 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
295 Openstack credentials
296 =====================
297 OpenStack credentials are mandatory and must be provided to Functest.
298 When running the command "functest env prepare", the framework will
299 automatically look for the Openstack credentials file
300 "/home/opnfv/functest/conf/env_file" and will exit with
301 error if it is not present or is empty.
303 There are 2 ways to provide that file:
305 * by using a Docker volume with -v option when creating the Docker container.
306 This is referred to in docker documentation as "Bind Mounting".
307 See the usage of this parameter in the following chapter.
308 * or creating manually the file '/home/opnfv/functest/conf/env_file'
309 inside the running container and pasting the credentials in it. Consult
310 your installer guide for further details. This is however not
311 instructed in this document.
313 In proxified environment you may need to change the credentials file.
314 There are some tips in chapter: `Proxy support`_
318 If you need to connect to a server that is TLS-enabled (the auth URL
319 begins with "https") and it uses a certificate from a private CA or a
320 self-signed certificate, then you will need to specify the path to an
321 appropriate CA certificate to use, to validate the server certificate
322 with the environment variable OS_CACERT::
325 /etc/ssl/certs/ca.crt
327 However, this certificate does not exist in the container by default.
328 It has to be copied manually from the OpenStack deployment. This can be
331 #. Create manually that file and copy the contents from the OpenStack
333 #. (Recommended) Add the file using a Docker volume when starting the
336 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
338 You might need to export OS_CACERT environment variable inside the
341 export OS_CACERT=/etc/ssl/certs/ca.crt
343 Certificate verification can be turned off using OS_INSECURE=true. For
344 example, Fuel uses self-signed cacerts by default, so an pre step would
347 export OS_INSECURE=true
352 By default all the logs are put un /home/opnfv/functest/results/functest.log.
353 If you want to have more logs in console, you may edit the logging.ini file
355 Connect on the docker then edit the file located in
356 /usr/lib/python3.7/site-packages/xtesting/ci/logging.ini
358 Change wconsole to console in the desired module to get more traces.
364 You may also directly modify the python code or the configuration file (e.g.
365 testcases.yaml used to declare test constraints) under
366 /usr/lib/python3.7/site-packages/xtesting and
367 /usr/lib/python3.7/site-packages/functest
375 When typing **exit** in the container prompt, this will cause exiting
376 the container and probably stopping it. When stopping a running Docker
377 container all the changes will be lost, there is a keyboard shortcut
378 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
379 reconnect to the running container **DO NOT** use the *run* command
380 again (since it will create a new container), use the *exec* or *attach*
383 docker ps # <check the container ID from the output>
384 docker exec -ti <CONTAINER_ID> /bin/bash
386 There are other useful Docker commands that might be needed to manage possible
387 issues with the containers.
389 List the running containers::
393 List all the containers including the stopped ones::
397 Start a stopped container named "FunTest"::
401 Attach to a running container named "StrikeTwo"::
403 docker attach StrikeTwo
405 It is useful sometimes to remove a container if there are some problems::
407 docker rm <CONTAINER_ID>
409 Use the *-f* option if the container is still running, it will force to
412 docker rm -f <CONTAINER_ID>
414 Check the Docker documentation [`dockerdocs`_] for more information.
417 Checking Openstack and credentials
418 ----------------------------------
419 It is recommended and fairly straightforward to check that Openstack
420 and credentials are working as expected.
422 Once the credentials are there inside the container, they should be
423 sourced before running any Openstack commands::
425 source /home/opnfv/functest/conf/env_file
427 After this, try to run any OpenStack command to see if you get any
428 output, for instance::
432 This will return a list of the actual users in the OpenStack
433 deployment. In any other case, check that the credentials are sourced::
437 This command must show a set of environment variables starting with
440 OS_REGION_NAME=RegionOne
441 OS_USER_DOMAIN_NAME=Default
442 OS_PROJECT_NAME=admin
444 OS_IDENTITY_API_VERSION=3
445 OS_PASSWORD=da54c27ae0d10dfae5297e6f0d6be54ebdb9f58d0f9dfc
446 OS_AUTH_URL=http://10.1.0.9:5000/v3
449 OS_ENDPOINT_TYPE=internalURL
450 OS_INTERFACE=internalURL
452 OS_PROJECT_DOMAIN_NAME=Default
455 If the OpenStack command still does not show anything or complains
456 about connectivity issues, it could be due to an incorrect url given to
457 the OS_AUTH_URL environment variable. Check the deployment settings.
463 If your Jumphost node is operating behind a http proxy, then there are
464 2 places where some special actions may be needed to make operations
467 #. Initial installation of docker engine First, try following the
468 official Docker documentation for Proxy settings. Some issues were
469 experienced on CentOS 7 based Jumphost. Some tips are documented
470 in section: :ref:`Docker Installation on CentOS behind http proxy`
473 If that is the case, make sure the resolv.conf and the needed
474 http_proxy and https_proxy environment variables, as well as the
475 'no_proxy' environment variable are set correctly::
477 # Make double sure that the 'no_proxy=...' line in the
478 # 'env_file' file is commented out first. Otherwise, the
479 # values set into the 'no_proxy' environment variable below will
480 # be ovewrwritten, each time the command
481 # 'source ~/functest/conf/env_file' is issued.
484 sed -i 's/export no_proxy/#export no_proxy/' env_file
487 # Next calculate some IP addresses for which http_proxy
488 # usage should be excluded:
490 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
492 adminURL_IP=$(openstack catalog show identity | \
493 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
495 export http_proxy="<your http proxy settings>"
496 export https_proxy="<your https proxy settings>"
497 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
499 # Ensure that "git" uses the http_proxy
500 # This may be needed if your firewall forbids SSL based git fetch
501 git config --global http.sslVerify True
502 git config --global http.proxy <Your http proxy settings>
504 For example, try to use the **nc** command from inside the functest
508 Connection to opnfv.org 80 port [tcp/http] succeeded!
511 Connection to opnfv.org 443 port [tcp/https] succeeded!
513 Note: In a Jumphost node based on the CentOS family OS, the **nc**
514 commands might not work. You can use the **curl** command instead.
516 curl https://www.opnfv.org/
518 <HTML><HEAD><meta http-equiv="content-type"
523 curl https://www.opnfv.org:443
525 <HTML><HEAD><meta http-equiv="content-type"
530 (Ignore the content. If command returns a valid HTML page, it proves
533 .. _`Docker Installation on CentOS behind http proxy`:
535 Docker Installation on CentOS behind http proxy
536 -----------------------------------------------
537 This section is applicable for CentOS family OS on Jumphost which
538 itself is behind a proxy server. In that case, the instructions below
539 should be followed **before** installing the docker engine::
541 1) # Make a directory '/etc/systemd/system/docker.service.d'
542 # if it does not exist
543 sudo mkdir /etc/systemd/system/docker.service.d
545 2) # Create a file called 'env.conf' in that directory with
546 # the following contents:
548 EnvironmentFile=-/etc/sysconfig/docker
550 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
551 # with the following contents:
552 HTTP_PROXY="<Your http proxy settings>"
553 HTTPS_PROXY="<Your https proxy settings>"
554 http_proxy="${HTTP_PROXY}"
555 https_proxy="${HTTPS_PROXY}"
557 4) # Reload the daemon
558 systemctl daemon-reload
560 5) # Sanity check - check the following docker settings:
561 systemctl show docker | grep -i env
565 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
566 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
568 Now follow the instructions in [`Install Docker on CentOS`_] to download
569 and install the **docker-engine**. The instructions conclude with a
570 "test pull" of a sample "Hello World" docker container. This should now
571 work with the above pre-requisite actions.
574 .. _`dockerdocs`: https://docs.docker.com/
575 .. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy
576 .. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
577 .. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
578 .. _`images/CentOS-7-x86_64-GenericCloud.qcow2`: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
579 .. _`images/cirros-0.4.0-x86_64-disk.img`: http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img
580 .. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img`: https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img