1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. SPDX-License-Identifier: CC-BY-4.0
5 Pulling the Docker image
6 ------------------------
7 Pull the Functest Docker image ('opnfv/functest') from the public
8 dockerhub registry under the OPNFV account: [dockerhub_], with the
9 following docker command::
11 docker pull opnfv/functest:<TagIdentifier>
13 where <TagIdentifier> identifies a release of the Functest docker
14 container image in the public Dockerhub registry. There are many tags
15 created automatically by the CI mechanisms, and you must ensure you
16 pull an image with the **correct tag** to match the OPNFV software
17 release installed in your environment. All available tagged images can
18 be seen from location [FunctestDockerTags_]. For example, when running
19 on the first official release of the OPNFV Danube system platform,
20 tag "danube.1.0" is needed. For the second and third releases, the tag
21 "danube.2.0" and "danube.3.0" can be used respectively.
22 Pulling other tags might cause some problems while running the tests.
23 Docker images pulled without a tag specifier bear the implicitly
24 assigned label "latest". If you need to specifically pull the latest
25 Functest docker image, then omit the tag argument::
27 docker pull opnfv/functest
29 After pulling the Docker image, check that it is available with the
30 following docker command::
32 [functester@jumphost ~]$ docker images
33 REPOSITORY TAG IMAGE ID CREATED SIZE
34 opnfv/functest latest 8cd6683c32ae 2 weeks ago 1.321 GB
35 opnfv/functest danube.2.0 d2c174a91911 7 minutes ago 1.471 GB
36 opnfv/functest danube.1.0 13fa54a1b238 4 weeks ago 1.29 GB
38 The Functest docker container environment can -in principle- be also
39 used with non-OPNFV official installers (e.g. 'devstack'), with the
40 **disclaimer** that support for such environments is outside of the
41 scope and responsibility of the OPNFV project.
43 Accessing the Openstack credentials
44 -----------------------------------
45 OpenStack credentials are mandatory and must be provided to Functest.
46 When running the command "functest env prepare", the framework will
47 automatically look for the Openstack credentials file
48 "/home/opnfv/functest/conf/openstack.creds" and will exit with
49 error if it is not present or it is empty.
51 There are 2 ways to provide that file:
53 * by using a Docker volume with -v option when creating the Docker container.
54 This is referred to in docker documentation as "Bind Mounting".
55 See the usage of this parameter in the following chapter.
56 * or creating manually the file '/home/opnfv/functest/conf/openstack.creds'
57 inside the running container and pasting the credentials in it. Consult
58 your installer guide for further details. This is however not
59 instructed in this document.
61 There is a default environment variable in the Functest container **$creds**
62 that points to the credentials absolute path to help the user with this task.
64 In proxified environment you may need to change the credentials file.
65 There are some tips in chapter: `Proxy support`_
67 Functest Docker parameters
68 --------------------------
69 This chapter explains how to run a container for executing functest
70 test suites. Numbered list below explains some details of the
71 recommended parameters for invoking docker container
73 #. It is a good practice to assign a precise container name through
74 the **--name** option.
76 #. Assign parameter for installer type::
78 -e "INSTALLER_TYPE=<type>"
79 # Use one of following apex, compass, fuel or joid
81 #. Functest needs to know the IP of some installers::
83 -e "INSTALLER_IP=<Specific IP Address>"
85 These two env variables are useful extract some information
86 from the deployment. However, for some test cases like
87 SFC or Barometer they are mandatory since the tests
88 need to access the installer node and the deployment.
90 #. Credentials for accessing the Openstack.
91 Most convenient way of passing them to container is by having a
92 local copy of the credentials file in Jumphost and then using the
93 **-v** option. In the example we have local file by the name of
94 "overcloudrc" and we are using that as an argument::
96 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds
98 The credentials file needs to exist in the Docker container
99 under the path: '/home/opnfv/functest/conf/openstack.creds'.
101 **WARNING:** If you are using the Joid installer, you must pass the
102 credentials using the **-v** option:
103 -v /var/lib/jenkins/admin-openrc:/home/opnfv/functest/conf/openstack.creds.
104 See the section `Accessing the Openstack credentials`_ above.
106 #. Passing deployment scenario
107 When running Functest against any of the supported OPNFV scenarios,
108 it is recommended to include also the environment variable
109 **DEPLOY_SCENARIO**. The **DEPLOY_SCENARIO** environment variable
110 is passed with the format::
112 -e "DEPLOY_SCENARIO=os-<controller>-<nfv_feature>-<ha_mode>"
114 os = OpenStack (No other VIM choices currently available)
115 controller is one of ( nosdn | odl_l2 | odl_l3 | onos | ocl)
116 nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
117 If several features are pertinent then use the underscore
118 character '_' to separate each feature (e.g. ovs_kvm)
119 'nofeature' indicates no NFV feature is deployed
120 ha_mode (high availability) is one of ( ha | noha )
122 **NOTE:** Not all possible combinations of "DEPLOY_SCENARIO" are
123 supported. The name passed in to the Functest Docker container
124 must match the scenario used when the actual OPNFV platform was
125 deployed. See release note to see the list of supported scenarios.
127 **NOTE:** The scenario name is mainly used to automatically detect
128 if a test suite is runnable or not (e.g. it will prevent ONOS test suite
129 to be run on ODL scenarios). If not set, Functest will try to run the
130 default test cases that might not include SDN controller or a specific
133 **NOTE:** A HA scenario means that 3 OpenStack controller nodes are
134 deployed. It does not necessarily mean that the whole system is HA. See
135 installer release notes for details.
138 Putting all above together, when using installer 'fuel' and an invented
139 INSTALLER_IP of '10.20.0.2', the recommended command to create the
140 Functest Docker container is as follows::
142 docker run --name "FunctestContainer" -it \
143 -e "INSTALLER_IP=10.20.0.2" \
144 -e "INSTALLER_TYPE=fuel" \
145 -e "DEPLOY_SCENARIO=os-odl_l2-ovs_kvm-ha" \
146 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
147 opnfv/functest /bin/bash
149 After the *run* command, a new prompt appears which means that we are inside
150 the container and ready to move to the next step.
152 For tips on how to set up container with installer Apex, see chapter
153 `Apex Installer Tips`_.
155 Finally, three additional environment variables can also be passed in
156 to the Functest Docker Container, using the -e
157 "<EnvironmentVariable>=<Value>" mechanism. The first two of these are
158 only relevant to Jenkins CI invoked testing and **should not be used**
159 when performing manual test scenarios::
161 -e "NODE_NAME=<Test POD Name>" \
162 -e "BUILD_TAG=<Jenkins Build Tag>" \
163 -e "CI_DEBUG=<DebugTraceValue>"
165 <Test POD Name> = Symbolic name of the POD where the tests are run.
166 Visible in test results files, which are stored
167 to the database. This option is only used when
168 tests are activated under Jenkins CI control.
169 It indicates the POD/hardware where the test has
170 been run. If not specified, then the POD name is
171 defined as "Unknown" by default.
172 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
173 <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
174 Visible in test results files, which are stored
175 to the database. This option is only set when
176 tests are activated under Jenkins CI control.
177 It enables the correlation of test results,
179 are independently pushed to the results database
180 from different Jenkins jobs.
181 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
182 <DebugTraceValue> = "true" or "false"
183 Default = "false", if not specified
184 If "true" is specified, then additional debug trace
185 text can be sent to the test results file / log files
186 and also to the standard console output.
190 Some specific tips are useful for the Apex Installer case. If not using
191 Apex Installer; ignore this section.
193 In case of Triple-O based installer (like Apex) the docker container
194 needs to connect to the installer VM, so it is then required that some
195 known SSH keys are present in docker container. Since the Jumphost root
196 SSH keys are already known, easiest way is to use those using the
197 'Bind mount' method. See below for sample parameter::
199 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa
201 NOTE: You need the "sudo" when creating the container to access root
202 users ssh credentials even the docker command itself might not
205 HINT! In case of Triple-O installers you can find value for the
206 INSTALLER_IP parameter by executing command and note the returned IP
209 inst=$(sudo virsh list | grep -iEo "undercloud|instack")
210 sudo virsh domifaddr ${inst}
212 NOTE: In releases prior to Colorado, the name 'instack' was
213 used. Currently the name 'undercloud' is used.
215 You can copy the credentials file from the "stack" users home directory
216 in installer VM to Jumphost. Please check the correct IP from the
217 command above. In the example below we are using invented IP address
220 scp stack@192.168.122.89:overcloudrc .
222 Here is an example of the full docker command invocation for an Apex
223 installed system, using latest Functest docker container, for
224 illustration purposes::
226 sudo docker run -it --name "ApexFuncTestODL" \
227 -e "INSTALLER_IP=192.168.122.89" \
228 -e "INSTALLER_TYPE=apex" \
229 -e "DEPLOY_SCENARIO=os-odl_l2-nofeature-ha" \
230 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa \
231 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
232 opnfv/functest /bin/bash
234 Compass installer local development env usage Tips
235 --------------------------------------------------
236 In the compass-functest local test case check and development environment,
237 in order to get openstack service inside the functest container, some
238 parameters should be configured during container creation, which are
239 hard to guess for freshman. This section will provide the guideline, the
240 parameters values are defaults here, which should be adjusted according
241 to the settings, the complete steps are given here so as not to appear
244 1, Pull Functest docker image from public dockerhub::
246 docker pull opnfv/functest:<Tag>
248 <Tag> here can be "brahmaputra.1.0", "colorado.1.0", etc.
249 Tag omitted means the latest docker image::
251 docker pull opnfv/functest
253 2, Functest Docker container creation
255 To make a file used for the environment, such as 'functest-docker-env'::
257 OS_AUTH_URL=http://172.16.1.222:35357/v2.0
261 OS_VOLUME_API_VERSION=2
262 OS_PROJECT_NAME=admin
263 INSTALLER_TYPE=compass
264 INSTALLER_IP=192.168.200.2
265 EXTERNAL_NETWORK=ext-net
267 Note: please adjust the content according to the environment, such as
268 'TENANT_ID' maybe used for some special cases.
270 Then to create the Functest docker::
272 docker run --privileged=true --rm -t \
273 --env-file functest-docker-env \
274 --name <Functest_Container_Name> \
275 opnfv/functest:<Tag> /bin/bash
277 3, To attach Functest container
279 Before trying to attach the Functest container, the status can be checked by::
283 to attach the 'Up' status Functest container and start bash mode::
285 docker exec -it <Functest_Container_Name> bash
287 4, Functest environment preparation and check
289 To see the Section below `Preparing the Functest environment`_.
291 Functest docker container directory structure
292 ---------------------------------------------
293 Inside the Functest docker container, the following directory structure
294 should now be in place::
323 Underneath the '/home/opnfv/' directory, the Functest docker container
324 includes two main directories:
326 * The **functest** directory stores configuration files (e.g. the
327 OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
328 the **data** directory stores a 'cirros' test image used in some
329 functional tests and the **results** directory stores some temporary
331 * The **repos** directory holds various repositories. The directories
332 are used for the installation of the needed tooling (e.g. rally) or
333 for the retrieval of feature projects scenarios (e.g. promise)
335 The structure under the **functest** repository can be described as
341 |-- run_unit_tests.sh
343 |-- test-requirements.txt
347 | `--traffic-profile-guidelines.rst
350 | |-- config_install_env.sh
351 | `-- docker_remote_api
365 | |-- check_deployment.py
366 | |-- config_functest.yaml
367 | |-- config_patch.yaml
368 | |-- generate_report.py
372 | |-- tier_builder.py
373 | `-- tier_handler.py
378 | |-- functest-complete.sh
383 | |-- pytest_suite_runner.py
402 |-- functest_logger.py
403 |-- functest_utils.py
405 |-- openstack_clean.py
406 |-- openstack_snapshot.py
407 |-- openstack_tacker.py
408 `-- openstack_utils.py
411 (Note: All *.pyc files removed from above list for brevity...)
413 We may distinguish several directories, the first level has 4 directories:
415 * **commons**: This directory is dedicated for storage of traffic
416 profile or any other test inputs that could be reused by any test
418 * **docker**: This directory includes the needed files and tools to
419 build the Functest Docker image.
420 * **docs**: This directory includes documentation: Release Notes,
421 User Guide, Configuration Guide and Developer Guide.
422 * **functest**: This directory contains all the code needed to run
423 functest internal cases and OPNFV onboarded feature or VNF test cases.
425 Functest directory has 6 directories:
426 * **ci**: This directory contains test structure definition files
427 (e.g <filename>.yaml) and bash shell/python scripts used to
428 configure and execute Functional tests. The test execution script
429 can be executed under the control of Jenkins CI jobs.
430 * **cli**: This directory holds the python based Functest CLI utility
431 source code, which is based on the Python 'click' framework.
432 * **core**: This directory holds the python based Functest core
433 source code. Three abstraction classes have been created to ease
434 the integration of internal, feature or vnf cases.
435 * **opnfv_tests**: This directory includes the scripts required by
436 Functest internal test cases and other feature projects test cases.
437 * **tests**: This directory includes the functest unit tests
438 * **utils**: this directory holds Python source code for some general
439 purpose helper utilities, which testers can also re-use in their
440 own test code. See for an example the Openstack helper utility:
441 'openstack_utils.py'.
443 Useful Docker commands
444 ----------------------
445 When typing **exit** in the container prompt, this will cause exiting
446 the container and probably stopping it. When stopping a running Docker
447 container all the changes will be lost, there is a keyboard shortcut
448 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
449 reconnect to the running container **DO NOT** use the *run* command
450 again (since it will create a new container), use the *exec* or *attach*
453 docker ps # <check the container ID from the output>
454 docker exec -ti <CONTAINER_ID> /bin/bash
456 There are other useful Docker commands that might be needed to manage possible
457 issues with the containers.
459 List the running containers::
463 List all the containers including the stopped ones::
467 Start a stopped container named "FunTest"::
471 Attach to a running container named "StrikeTwo"::
473 docker attach StrikeTwo
475 It is useful sometimes to remove a container if there are some problems::
477 docker rm <CONTAINER_ID>
479 Use the *-f* option if the container is still running, it will force to
482 docker rm -f <CONTAINER_ID>
484 Check the Docker documentation dockerdocs_ for more information.
486 Preparing the Functest environment
487 ----------------------------------
488 Once the Functest docker container is up and running, the required
489 Functest environment needs to be prepared. A custom built **functest**
490 CLI utility is available to perform the needed environment preparation
491 action. Once the environment is prepared, the **functest** CLI utility
492 can be used to run different functional tests. The usage of the
493 **functest** CLI utility to run tests is described further in the
494 Functest User Guide `OPNFV_FuncTestUserGuide`_
496 Prior to commencing the Functest environment preparation, we can check
497 the initial status of the environment. Issue the **functest env status**
498 command at the prompt::
501 Functest environment is not installed.
503 Note: When the Functest environment is prepared, the command will
504 return the status: "Functest environment ready to run tests."
506 To prepare the Functest docker container for test case execution, issue
507 the **functest env prepare** command at the prompt::
511 This script will make sure that the requirements to run the tests are
512 met and will install the needed libraries and tools by all Functest
513 test cases. It should be run only once every time the Functest docker
514 container is started from scratch. If you try to run this command, on
515 an already prepared environment, you will be prompted whether you really
516 want to continue or not::
519 It seems that the environment has been already prepared.
520 Do you want to do it again? [y|n]
522 (Type 'n' to abort the request, or 'y' to repeat the
523 environment preparation)
526 To list some basic information about an already prepared Functest
527 docker container environment, issue the **functest env show** at the
531 +======================================================+
532 | Functest Environment info |
533 +======================================================+
534 | INSTALLER: apex, 192.168.122.89 |
535 | SCENARIO: os-odl_l2-nofeature-ha |
537 | GIT BRANCH: master |
538 | GIT HASH: 5bf1647dec6860464eeb082b2875798f0759aa91 |
539 | DEBUG FLAG: false |
540 +------------------------------------------------------+
542 +------------------------------------------------------+
546 INSTALLER: Displays the INSTALLER_TYPE value
548 and the INSTALLER_IP value
549 - here = "192.168.122.89"
550 SCENARIO: Displays the DEPLOY_SCENARIO value
551 - here = "os-odl_l2-nofeature-ha"
552 POD: Displays the value passed in NODE_NAME
554 GIT BRANCH: Displays the git branch of the OPNFV Functest
555 project repository included in the Functest
558 (In first official colorado release
559 would be "colorado.1.0")
560 GIT HASH: Displays the git hash of the OPNFV Functest
561 project repository included in the Functest
563 - here = "5bf1647dec6860464eeb082b2875798f0759aa91"
564 DEBUG FLAG: Displays the CI_DEBUG value
567 NOTE: In Jenkins CI runs, an additional item "BUILD TAG"
568 would also be listed. The value is set by Jenkins CI.
570 Finally, the **functest** CLI has a **--help** options:
574 functest --help Usage: functest [OPTIONS] COMMAND [ARGS]...
577 --version Show the version and exit.
578 -h, --help Show this message and exit.
587 Usage: functest env [OPTIONS] COMMAND [ARGS]...
590 -h, --help Show this message and exit.
593 prepare Prepares the Functest environment.
594 show Shows information about the current...
595 status Checks if the Functest environment is ready...
597 Checking Openstack and credentials
598 ----------------------------------
599 It is recommended and fairly straightforward to check that Openstack
600 and credentials are working as expected.
602 Once the credentials are there inside the container, they should be
603 sourced before running any Openstack commands::
605 source /home/opnfv/functest/conf/openstack.creds
607 After this, try to run any OpenStack command to see if you get any
608 output, for instance::
612 This will return a list of the actual users in the OpenStack
613 deployment. In any other case, check that the credentials are sourced::
617 This command must show a set of environment variables starting with
620 OS_REGION_NAME=RegionOne
621 OS_DEFAULT_DOMAIN=default
622 OS_PROJECT_NAME=admin
624 OS_AUTH_STRATEGY=keystone
625 OS_AUTH_URL=http://172.30.10.3:5000/v2.0
628 OS_ENDPOINT_TYPE=internalURL
631 If the OpenStack command still does not show anything or complains
632 about connectivity issues, it could be due to an incorrect url given to
633 the OS_AUTH_URL environment variable. Check the deployment settings.
637 If you need to connect to a server that is TLS-enabled (the auth URL
638 begins with "https") and it uses a certificate from a private CA or a
639 self-signed certificate, then you will need to specify the path to an
640 appropriate CA certificate to use, to validate the server certificate
641 with the environment variable OS_CACERT::
644 /etc/ssl/certs/ca.crt
646 However, this certificate does not exist in the container by default.
647 It has to be copied manually from the OpenStack deployment. This can be
650 #. Create manually that file and copy the contents from the OpenStack
652 #. (Recommended) Add the file using a Docker volume when starting the
655 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
657 You might need to export OS_CACERT environment variable inside the
660 export OS_CACERT=/etc/ssl/certs/ca.crt
662 Certificate verification can be turned off using OS_INSECURE=true. For
663 example, Fuel uses self-signed cacerts by default, so an pre step would
666 export OS_INSECURE=true
670 If your Jumphost node is operating behind a http proxy, then there are
671 2 places where some special actions may be needed to make operations
674 #. Initial installation of docker engine First, try following the
675 official Docker documentation for Proxy settings. Some issues were
676 experienced on CentOS 7 based Jumphost. Some tips are documented
677 in section: `Docker Installation on CentOS behind http proxy`_
680 #. Execution of the Functest environment preparation inside the
681 created docker container Functest needs internet access to
682 download some resources for some test cases. This might not
683 work properly if the Jumphost is connecting to internet
684 through a http Proxy.
686 If that is the case, make sure the resolv.conf and the needed
687 http_proxy and https_proxy environment variables, as well as the
688 'no_proxy' environment variable are set correctly::
690 # Make double sure that the 'no_proxy=...' line in the
691 # 'openstack.creds' file is commented out first. Otherwise, the
692 # values set into the 'no_proxy' environment variable below will
693 # be ovewrwritten, each time the command
694 # 'source ~/functest/conf/openstack.creds' is issued.
697 sed -i 's/export no_proxy/#export no_proxy/' openstack.creds
698 source ./openstack.creds
700 # Next calculate some IP addresses for which http_proxy
701 # usage should be excluded:
703 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
705 adminURL_IP=$(openstack catalog show identity | \
706 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
708 export http_proxy="<your http proxy settings>"
709 export https_proxy="<your https proxy settings>"
710 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
712 # Ensure that "git" uses the http_proxy
713 # This may be needed if your firewall forbids SSL based git fetch
714 git config --global http.sslVerify True
715 git config --global http.proxy <Your http proxy settings>
717 Validation check: Before running **'functest env prepare'** CLI command,
718 make sure you can reach http and https sites from inside the Functest
721 For example, try to use the **nc** command from inside the functest
725 Connection to opnfv.org 80 port [tcp/http] succeeded!
728 Connection to opnfv.org 443 port [tcp/https] succeeded!
730 Note: In a Jumphost node based on the CentOS family OS, the **nc**
731 commands might not work. You can use the **curl** command instead.
733 curl http://www.opnfv.org:80
734 <HTML><HEAD><meta http-equiv="content-type"
739 curl https://www.opnfv.org:443
740 <HTML><HEAD><meta http-equiv="content-type"
745 (Ignore the content. If command returns a valid HTML page, it proves
748 Docker Installation on CentOS behind http proxy
749 -----------------------------------------------
750 This section is applicable for CentOS family OS on Jumphost which
751 itself is behind a proxy server. In that case, the instructions below
752 should be followed **before** installing the docker engine::
754 1) # Make a directory '/etc/systemd/system/docker.service.d'
755 # if it does not exist
756 sudo mkdir /etc/systemd/system/docker.service.d
758 2) # Create a file called 'env.conf' in that directory with
759 # the following contents:
761 EnvironmentFile=-/etc/sysconfig/docker
763 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
764 # with the following contents:
765 HTTP_PROXY="<Your http proxy settings>"
766 HTTPS_PROXY="<Your https proxy settings>"
767 http_proxy="${HTTP_PROXY}"
768 https_proxy="${HTTPS_PROXY}"
770 4) # Reload the daemon
771 systemctl daemon-reload
773 5) # Sanity check - check the following docker settings:
774 systemctl show docker | grep -i env
778 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
779 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
781 Now follow the instructions in [`InstallDockerCentOS`_] to download
782 and install the **docker-engine**. The instructions conclude with a
783 "test pull" of a sample "Hello World" docker container. This should now
784 work with the above pre-requisite actions.
786 .. _dockerdocs: https://docs.docker.com/
787 .. _dockerhub: https://hub.docker.com/r/opnfv/functest/
788 .. _Proxy: https://docs.docker.com/engine/admin/systemd/#http-proxy
789 .. _FunctestDockerTags: https://hub.docker.com/r/opnfv/functest/tags/
790 .. _InstallDockerCentOS: https://docs.docker.com/engine/installation/linux/centos/
791 .. _OPNFV_FuncTestUserGuide: http://artifacts.opnfv.org/functest/docs/userguide/index.html