1 .. SPDX-License-Identifier: CC-BY-4.0
3 Installation and configuration
4 ==============================
6 Alpine containers have been introduced in Euphrates.
7 Alpine allows Functest testing in several very light containers and thanks to
8 the refactoring on dependency management should allow the creation of light and
9 fully customized docker images.
12 Functest Dockers for OpenStack deployment
13 -----------------------------------------
14 Docker images are available on the dockerhub:
17 * opnfv/functest-healthcheck
18 * opnfv/functest-smoke
19 * opnfv/functest-smoke-cntt
20 * opnfv/functest-benchmarking
21 * opnfv/functest-benchmarking-cntt
25 Preparing your environment
26 ^^^^^^^^^^^^^^^^^^^^^^^^^^
30 DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario
31 NAMESERVER=XXX # if not 8.8.8.8
32 EXTERNAL_NETWORK=XXX # if not first network with router:external=True
33 DASHBOARD_URL=XXX # else tempest_horizon will be skipped
34 NEW_USER_ROLE=XXX # if not member
35 SDN_CONTROLLER_IP=XXX # if odl scenario
36 VOLUME_DEVICE_NAME=XXX # if not vdb
37 FLAVOR_EXTRA_SPECS=hw:mem_page_size:large # if fdio scenarios
39 See section on environment variables for details.
43 export OS_AUTH_URL=XXX
44 export OS_USER_DOMAIN_NAME=XXX
45 export OS_PROJECT_DOMAIN_NAME=XXX
46 export OS_USERNAME=XXX
47 export OS_PROJECT_NAME=XXX
48 export OS_PASSWORD=XXX
49 export OS_IDENTITY_API_VERSION=3
50 export OS_REGION_NAME=XXX
52 See section on OpenStack credentials for details.
54 Create a directory for the different images (attached as a Docker volume)::
56 mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/*
58 images/cirros-0.4.0-aarch64-disk.img
59 images/cirros-0.4.0-x86_64-disk.img
60 images/cloudify-docker-manager-community-19.01.24.tar
61 images/Fedora-Cloud-Base-30-1.2.x86_64.qcow2
62 images/shaker-image-1.3.0+stretch.qcow2
63 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
64 images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
65 images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
66 images/vyos-1.1.8-amd64.qcow2
68 Testing healthcheck suite
69 ^^^^^^^^^^^^^^^^^^^^^^^^^
71 Run healthcheck suite::
73 sudo docker run --env-file env \
74 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
75 -v $(pwd)/images:/home/opnfv/functest/images \
76 opnfv/functest-healthcheck
78 Results shall be displayed as follows::
80 +--------------------------+------------------+---------------------+------------------+----------------+
81 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
82 +--------------------------+------------------+---------------------+------------------+----------------+
83 | connection_check | functest | healthcheck | 00:03 | PASS |
84 | tenantnetwork1 | functest | healthcheck | 00:05 | PASS |
85 | tenantnetwork2 | functest | healthcheck | 00:06 | PASS |
86 | vmready1 | functest | healthcheck | 00:06 | PASS |
87 | vmready2 | functest | healthcheck | 00:08 | PASS |
88 | singlevm1 | functest | healthcheck | 00:32 | PASS |
89 | singlevm2 | functest | healthcheck | 00:37 | PASS |
90 | vping_ssh | functest | healthcheck | 00:46 | PASS |
91 | vping_userdata | functest | healthcheck | 00:39 | PASS |
92 | cinder_test | functest | healthcheck | 01:05 | PASS |
93 | tempest_smoke | functest | healthcheck | 05:39 | PASS |
94 | tempest_horizon | functest | healthcheck | 01:05 | PASS |
95 | odl | functest | healthcheck | 00:00 | SKIP |
96 +--------------------------+------------------+---------------------+------------------+----------------+
98 NOTE: the duration is a reference and it might vary depending on your SUT.
105 sudo docker run --env-file env \
106 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
107 -v $(pwd)/images:/home/opnfv/functest/images \
110 Results shall be displayed as follows::
112 +---------------------------+------------------+---------------+------------------+----------------+
113 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
114 +---------------------------+------------------+---------------+------------------+----------------+
115 | tempest_neutron | functest | smoke | 15:30 | PASS |
116 | tempest_cinder | functest | smoke | 02:01 | PASS |
117 | tempest_keystone | functest | smoke | 01:17 | PASS |
118 | tempest_heat | functest | smoke | 22:14 | PASS |
119 | tempest_telemetry | functest | smoke | 00:00 | SKIP |
120 | rally_sanity | functest | smoke | 17:24 | PASS |
121 | refstack_compute | functest | smoke | 07:03 | PASS |
122 | refstack_object | functest | smoke | 02:09 | PASS |
123 | refstack_platform | functest | smoke | 07:31 | PASS |
124 | tempest_full | functest | smoke | 41:52 | PASS |
125 | tempest_scenario | functest | smoke | 08:42 | PASS |
126 | tempest_slow | functest | smoke | 43:42 | PASS |
127 | patrole | functest | smoke | 02:42 | PASS |
128 | tempest_barbican | functest | smoke | 02:30 | PASS |
129 | tempest_octavia | functest | smoke | 00:00 | SKIP |
130 +---------------------------+------------------+---------------+------------------+----------------+
132 Note: if the scenario does not support some tests, they are indicated as SKIP.
133 See User guide for details.
135 Testing smoke CNTT suite
136 ^^^^^^^^^^^^^^^^^^^^^^^^
138 Run smoke-cntt suite::
140 sudo docker run --env-file env \
141 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
142 -v $(pwd)/images:/home/opnfv/functest/images \
143 opnfv/functest-smoke-cntt
145 Results shall be displayed as follows::
147 +-------------------------------+------------------+---------------+------------------+----------------+
148 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
149 +-------------------------------+------------------+---------------+------------------+----------------+
150 | tempest_neutron_cntt | functest | smoke | 11:35 | PASS |
151 | tempest_cinder_cntt | functest | smoke | 01:58 | PASS |
152 | tempest_keystone_cntt | functest | smoke | 01:13 | PASS |
153 | tempest_heat_cntt | functest | smoke | 22:32 | PASS |
154 | rally_sanity_cntt | functest | smoke | 17:16 | PASS |
155 | tempest_full_cntt | functest | smoke | 41:13 | PASS |
156 | tempest_scenario_cntt | functest | smoke | 08:57 | PASS |
157 | tempest_slow_cntt | functest | smoke | 35:58 | PASS |
158 +-------------------------------+------------------+---------------+------------------+----------------+
160 Testing benchmarking suite
161 ^^^^^^^^^^^^^^^^^^^^^^^^^^
163 Run benchmarking suite::
165 sudo docker run --env-file env \
166 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
167 -v $(pwd)/images:/home/opnfv/functest/images \
168 opnfv/functest-benchmarking
170 Results shall be displayed as follows::
172 +--------------------+------------------+----------------------+------------------+----------------+
173 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
174 +--------------------+------------------+----------------------+------------------+----------------+
175 | rally_full | functest | benchmarking | 93:03 | PASS |
176 | rally_jobs | functest | benchmarking | 27:05 | PASS |
177 | vmtp | functest | benchmarking | 17:56 | PASS |
178 | shaker | functest | benchmarking | 24:02 | PASS |
179 +--------------------+------------------+----------------------+------------------+----------------+
181 Note: if the scenario does not support some tests, they are indicated as SKIP.
182 See User guide for details.
184 Testing benchmarking CNTT suite
185 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
187 Run benchmarking-cntt suite::
189 sudo docker run --env-file env \
190 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
191 -v $(pwd)/images:/home/opnfv/functest/images \
192 opnfv/functest-benchmarking-cntt
194 Results shall be displayed as follows::
196 +-------------------------+------------------+----------------------+------------------+----------------+
197 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
198 +-------------------------+------------------+----------------------+------------------+----------------+
199 | rally_full_cntt | functest | benchmarking | 89:52 | PASS |
200 | rally_jobs_cntt | functest | benchmarking | 19:39 | PASS |
201 | vmtp | functest | benchmarking | 16:59 | PASS |
202 | shaker | functest | benchmarking | 23:43 | PASS |
203 +-------------------------+------------------+----------------------+------------------+----------------+
210 sudo docker run --env-file env \
211 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
212 -v $(pwd)/images:/home/opnfv/functest/images \
215 Results shall be displayed as follows::
217 +----------------------+------------------+--------------+------------------+----------------+
218 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
219 +----------------------+------------------+--------------+------------------+----------------+
220 | cloudify | functest | vnf | 05:08 | PASS |
221 | cloudify_ims | functest | vnf | 24:46 | PASS |
222 | heat_ims | functest | vnf | 33:12 | PASS |
223 | vyos_vrouter | functest | vnf | 15:53 | PASS |
224 | juju_epc | functest | vnf | 27:52 | PASS |
225 +----------------------+------------------+--------------+------------------+----------------+
227 Functest Dockers for Kubernetes deployment
228 ------------------------------------------
229 Docker images are available on the dockerhub:
231 * opnfv/functest-kubernetes-core
232 * opnfv/functest-kubernetes-healthcheck
233 * opnfv/functest-kubernetes-smoke
235 Preparing your environment
236 ^^^^^^^^^^^^^^^^^^^^^^^^^^
240 DEPLOY_SCENARIO=k8s-XXX
242 Testing healthcheck suite
243 ^^^^^^^^^^^^^^^^^^^^^^^^^
245 Run healthcheck suite::
247 sudo docker run -it --env-file env \
248 -v $(pwd)/config:/root/.kube/config \
249 opnfv/functest-kubernetes-healthcheck
251 A config file in the current dir 'config' is also required, which should be
252 volume mapped to ~/.kube/config inside kubernetes container.
254 Results shall be displayed as follows::
256 +-------------------+------------------+---------------------+------------------+----------------+
257 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
258 +-------------------+------------------+---------------------+------------------+----------------+
259 | k8s_smoke | functest | healthcheck | 01:09 | PASS |
260 +-------------------+------------------+---------------------+------------------+----------------+
267 sudo docker run -it --env-file env \
268 -v $(pwd)/config:/root/.kube/config \
269 opnfv/functest-kubernetes-smoke
271 Results shall be displayed as follows::
273 +---------------------------+------------------+---------------+------------------+----------------+
274 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
275 +---------------------------+------------------+---------------+------------------+----------------+
276 | xrally_kubernetes | functest | smoke | 22:04 | PASS |
277 | k8s_conformance | functest | smoke | 173:48 | PASS |
278 +---------------------------+------------------+---------------+------------------+----------------+
280 Environment variables
281 =====================
283 Several environment variables may be specified:
285 * INSTALLER_IP=<Specific IP Address>
286 * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
287 * NAMESERVER=XXX # if not 8.8.8.8
288 * VOLUME_DEVICE_NAME=XXX # if not vdb
289 * EXTERNAL_NETWORK=XXX # if not first network with router:external=True
290 * NEW_USER_ROLE=XXX # if not member
292 INSTALLER_IP is required by Barometer in order to access the installer node and
295 The format for the DEPLOY_SCENARIO env variable can be described as follows:
296 * vim: (os|k8s) = OpenStack or Kubernetes
297 * controller is one of ( nosdn | odl )
298 * nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
299 * ha_mode (high availability) is one of ( ha | noha )
301 If several features are pertinent then use the underscore character '_' to
302 separate each feature (e.g. ovs_kvm). 'nofeature' indicates that no OPNFV
305 The list of supported scenarios per release/installer is indicated in the
308 **NOTE:** The scenario name is mainly used to automatically detect
309 if a test suite is runnable or not (e.g. it will prevent ODL test suite to be
310 run on 'nosdn' scenarios). If not set, Functest will try to run the default
311 test cases that might not include SDN controller or a specific feature.
313 **NOTE:** An HA scenario means that 3 OpenStack controller nodes are
314 deployed. It does not necessarily mean that the whole system is HA. See
315 installer release notes for details.
317 Finally, three additional environment variables can also be passed in
318 to the Functest Docker Container, using the -e
319 "<EnvironmentVariable>=<Value>" mechanism. The first two parameters are
320 only relevant to Jenkins CI invoked testing and **should not be used**
321 when performing manual test scenarios:
323 * INSTALLER_TYPE=(apex|compass|daisy|fuel)
324 * NODE_NAME=<Test POD Name>
325 * BUILD_TAG=<Jenkins Build Tag>
329 * <Test POD Name> = Symbolic name of the POD where the tests are run.
330 Visible in test results files, which are stored
331 to the database. This option is only used when
332 tests are activated under Jenkins CI control.
333 It indicates the POD/hardware where the test has
334 been run. If not specified, then the POD name is
335 defined as "Unknown" by default.
336 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
337 * <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
338 Visible in test results files, which are stored
339 to the database. This option is only set when
340 tests are activated under Jenkins CI control.
341 It enables the correlation of test results,
342 which are independently pushed to the results database
343 from different Jenkins jobs.
344 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
347 Openstack credentials
348 =====================
349 OpenStack credentials are mandatory and must be provided to Functest.
350 When running the command "functest env prepare", the framework will
351 automatically look for the Openstack credentials file
352 "/home/opnfv/functest/conf/env_file" and will exit with
353 error if it is not present or is empty.
355 There are 2 ways to provide that file:
357 * by using a Docker volume with -v option when creating the Docker container.
358 This is referred to in docker documentation as "Bind Mounting".
359 See the usage of this parameter in the following chapter.
360 * or creating manually the file '/home/opnfv/functest/conf/env_file'
361 inside the running container and pasting the credentials in it. Consult
362 your installer guide for further details. This is however not
363 instructed in this document.
365 In proxified environment you may need to change the credentials file.
366 There are some tips in chapter: `Proxy support`_
370 If you need to connect to a server that is TLS-enabled (the auth URL
371 begins with "https") and it uses a certificate from a private CA or a
372 self-signed certificate, then you will need to specify the path to an
373 appropriate CA certificate to use, to validate the server certificate
374 with the environment variable OS_CACERT::
377 /etc/ssl/certs/ca.crt
379 However, this certificate does not exist in the container by default.
380 It has to be copied manually from the OpenStack deployment. This can be
383 #. Create manually that file and copy the contents from the OpenStack
385 #. (Recommended) Add the file using a Docker volume when starting the
388 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
390 You might need to export OS_CACERT environment variable inside the
393 export OS_CACERT=/etc/ssl/certs/ca.crt
395 Certificate verification can be turned off using OS_INSECURE=true. For
396 example, Fuel uses self-signed cacerts by default, so an pre step would
399 export OS_INSECURE=true
404 By default all the logs are put un /home/opnfv/functest/results/functest.log.
405 If you want to have more logs in console, you may edit the logging.ini file
407 Connect on the docker then edit the file located in
408 /usr/lib/python3.8/site-packages/xtesting/ci/logging.ini
410 Change wconsole to console in the desired module to get more traces.
416 You may also directly modify the python code or the configuration file (e.g.
417 testcases.yaml used to declare test constraints) under
418 /usr/lib/python3.8/site-packages/xtesting and
419 /usr/lib/python3.8/site-packages/functest
427 When typing **exit** in the container prompt, this will cause exiting
428 the container and probably stopping it. When stopping a running Docker
429 container all the changes will be lost, there is a keyboard shortcut
430 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
431 reconnect to the running container **DO NOT** use the *run* command
432 again (since it will create a new container), use the *exec* or *attach*
435 docker ps # <check the container ID from the output>
436 docker exec -ti <CONTAINER_ID> /bin/bash
438 There are other useful Docker commands that might be needed to manage possible
439 issues with the containers.
441 List the running containers::
445 List all the containers including the stopped ones::
449 Start a stopped container named "FunTest"::
453 Attach to a running container named "StrikeTwo"::
455 docker attach StrikeTwo
457 It is useful sometimes to remove a container if there are some problems::
459 docker rm <CONTAINER_ID>
461 Use the *-f* option if the container is still running, it will force to
464 docker rm -f <CONTAINER_ID>
466 Check the Docker documentation [`dockerdocs`_] for more information.
469 Checking Openstack and credentials
470 ----------------------------------
471 It is recommended and fairly straightforward to check that Openstack
472 and credentials are working as expected.
474 Once the credentials are there inside the container, they should be
475 sourced before running any Openstack commands::
477 source /home/opnfv/functest/conf/env_file
479 After this, try to run any OpenStack command to see if you get any
480 output, for instance::
484 This will return a list of the actual users in the OpenStack
485 deployment. In any other case, check that the credentials are sourced::
489 This command must show a set of environment variables starting with
492 OS_REGION_NAME=RegionOne
493 OS_USER_DOMAIN_NAME=Default
494 OS_PROJECT_NAME=admin
496 OS_IDENTITY_API_VERSION=3
497 OS_PASSWORD=da54c27ae0d10dfae5297e6f0d6be54ebdb9f58d0f9dfc
498 OS_AUTH_URL=http://10.1.0.9:5000/v3
501 OS_ENDPOINT_TYPE=internalURL
502 OS_INTERFACE=internalURL
504 OS_PROJECT_DOMAIN_NAME=Default
507 If the OpenStack command still does not show anything or complains
508 about connectivity issues, it could be due to an incorrect url given to
509 the OS_AUTH_URL environment variable. Check the deployment settings.
515 If your Jumphost node is operating behind a http proxy, then there are
516 2 places where some special actions may be needed to make operations
519 #. Initial installation of docker engine First, try following the
520 official Docker documentation for Proxy settings. Some issues were
521 experienced on CentOS 7 based Jumphost. Some tips are documented
522 in section: :ref:`Docker Installation on CentOS behind http proxy`
525 If that is the case, make sure the resolv.conf and the needed
526 http_proxy and https_proxy environment variables, as well as the
527 'no_proxy' environment variable are set correctly::
529 # Make double sure that the 'no_proxy=...' line in the
530 # 'env_file' file is commented out first. Otherwise, the
531 # values set into the 'no_proxy' environment variable below will
532 # be ovewrwritten, each time the command
533 # 'source ~/functest/conf/env_file' is issued.
536 sed -i 's/export no_proxy/#export no_proxy/' env_file
539 # Next calculate some IP addresses for which http_proxy
540 # usage should be excluded:
542 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
544 adminURL_IP=$(openstack catalog show identity | \
545 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
547 export http_proxy="<your http proxy settings>"
548 export https_proxy="<your https proxy settings>"
549 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
551 # Ensure that "git" uses the http_proxy
552 # This may be needed if your firewall forbids SSL based git fetch
553 git config --global http.sslVerify True
554 git config --global http.proxy <Your http proxy settings>
556 For example, try to use the **nc** command from inside the functest
560 Connection to opnfv.org 80 port [tcp/http] succeeded!
563 Connection to opnfv.org 443 port [tcp/https] succeeded!
565 Note: In a Jumphost node based on the CentOS family OS, the **nc**
566 commands might not work. You can use the **curl** command instead.
568 curl https://www.opnfv.org/
570 <HTML><HEAD><meta http-equiv="content-type"
575 curl https://www.opnfv.org:443
577 <HTML><HEAD><meta http-equiv="content-type"
582 (Ignore the content. If command returns a valid HTML page, it proves
585 .. _`Docker Installation on CentOS behind http proxy`:
587 Docker Installation on CentOS behind http proxy
588 -----------------------------------------------
589 This section is applicable for CentOS family OS on Jumphost which
590 itself is behind a proxy server. In that case, the instructions below
591 should be followed **before** installing the docker engine::
593 1) # Make a directory '/etc/systemd/system/docker.service.d'
594 # if it does not exist
595 sudo mkdir /etc/systemd/system/docker.service.d
597 2) # Create a file called 'env.conf' in that directory with
598 # the following contents:
600 EnvironmentFile=-/etc/sysconfig/docker
602 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
603 # with the following contents:
604 HTTP_PROXY="<Your http proxy settings>"
605 HTTPS_PROXY="<Your https proxy settings>"
606 http_proxy="${HTTP_PROXY}"
607 https_proxy="${HTTPS_PROXY}"
609 4) # Reload the daemon
610 systemctl daemon-reload
612 5) # Sanity check - check the following docker settings:
613 systemctl show docker | grep -i env
617 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
618 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
620 Now follow the instructions in [`Install Docker on CentOS`_] to download
621 and install the **docker-engine**. The instructions conclude with a
622 "test pull" of a sample "Hello World" docker container. This should now
623 work with the above pre-requisite actions.
626 .. _`dockerdocs`: https://docs.docker.com/
627 .. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy
628 .. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
629 .. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
630 .. _`images/CentOS-7-x86_64-GenericCloud.qcow2`: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
631 .. _`images/cirros-0.4.0-x86_64-disk.img`: http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img
632 .. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img`: https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img