1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. SPDX-License-Identifier: CC-BY-4.0
5 Pulling the Docker image
6 ------------------------
7 Pull the Functest Docker image ('opnfv/functest') from the public
8 dockerhub registry under the OPNFV account: [dockerhub_], with the
9 following docker command::
11 docker pull opnfv/functest:<TagIdentifier>
13 where <TagIdentifier> identifies a release of the Functest docker
14 container image in the public dockerhub registry. There are many tags
15 created automatically by the CI mechanisms, and you must ensure you
16 pull an image with the **correct tag** to match the OPNFV software
17 release installed in your environment. All available tagged images can
18 be seen from location [FunctestDockerTags_]. For example, when running
19 on the first official release of the OPNFV Danube system platform,
20 tag "danube.1.0" is needed. Pulling other tags might cause some
21 problems while running the tests.
22 Docker images pulled without a tag specifier bear the implicitly
23 assigned label "latest". If you need to specifically pull the latest
24 Functest docker image, then omit the tag argument::
26 docker pull opnfv/functest
28 After pulling the Docker image, check that it is available with the
29 following docker command::
31 [functester@jumphost ~]$ docker images
32 REPOSITORY TAG IMAGE ID CREATED SIZE
33 opnfv/functest latest 8cd6683c32ae 2 weeks ago 1.321 GB
34 opnfv/functest danube.1.0 13fa54a1b238 4 weeks ago 1.29 GB
35 opnfv/functest colorado.1.0 94b78faa94f7 9 weeks ago 968 MB
37 The Functest docker container environment can -in principle- be also
38 used with non-OPNFV official installers (e.g. 'devstack'), with the
39 **disclaimer** that support for such environments is outside of the
40 scope and responsibility of the OPNFV project.
42 Accessing the Openstack credentials
43 -----------------------------------
44 OpenStack credentials are mandatory and must be provided to Functest.
45 When running the command "functest env prepare", the framework will
46 automatically look for the Openstack credentials file
47 "/home/opnfv/functest/conf/openstack.creds" and will exit with
48 error if it is not present or it is empty.
50 There are 2 ways to provide that file:
52 * by using a Docker volume with -v option when creating the Docker container.
53 This is referred to in docker documentation as "Bind Mounting".
54 See the usage of this parameter in the following chapter.
55 * or creating manually the file '/home/opnfv/functest/conf/openstack.creds'
56 inside the running container and pasting the credentials in it. Consult
57 your installer guide for further details. This is however not
58 instructed in this document.
60 There is a default environment variable in the Functest container **$creds**
61 that points to the credentials absolute path to help the user with this task.
63 In proxified environment you may need to change the credentials file.
64 There are some tips in chapter: `Proxy support`_
66 Functest Docker parameters
67 --------------------------
68 This chapter explains how to run a container for executing functest
69 test suites. Numbered list below explains some details of the
70 recommended parameters for invoking docker container
72 #. It is a good practice to assign a precise container name through
73 the **--name** option.
75 #. Assign parameter for installer type::
77 -e "INSTALLER_TYPE=<type>"
78 # Use one of following apex, compass, fuel or joid
80 #. Functest needs to know the IP of some installers::
82 -e "INSTALLER_IP=<Specific IP Address>"
84 These two env variables are useful extract some information
85 from the deployment. However, for some test cases like
86 SFC or Barometer they are mandatory since the tests
87 need to access the installer node and the deployment.
89 #. Credentials for accessing the Openstack.
90 Most convenient way of passing them to container is by having a
91 local copy of the credentials file in Jumphost and then using the
92 **-v** option. In the example we have local file by the name of
93 "overcloudrc" and we are using that as an argument::
95 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds
97 The credentials file needs to exist in the Docker container
98 under the path: '/home/opnfv/functest/conf/openstack.creds'.
100 **WARNING:** If you are using the Joid installer, you must pass the
101 credentials using the **-v** option:
102 -v /var/lib/jenkins/admin-openrc:/home/opnfv/functest/conf/openstack.creds.
103 See the section `Accessing the Openstack credentials`_ above.
105 #. Passing deployment scenario
106 When running Functest against any of the supported OPNFV scenarios,
107 it is recommended to include also the environment variable
108 **DEPLOY_SCENARIO**. The **DEPLOY_SCENARIO** environment variable
109 is passed with the format::
111 -e "DEPLOY_SCENARIO=os-<controller>-<nfv_feature>-<ha_mode>"
113 os = OpenStack (No other VIM choices currently available)
114 controller is one of ( nosdn | odl_l2 | odl_l3 | onos | ocl)
115 nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
116 If several features are pertinent then use the underscore
117 character '_' to separate each feature (e.g. ovs_kvm)
118 'nofeature' indicates no NFV feature is deployed
119 ha_mode (high availability) is one of ( ha | noha )
121 **NOTE:** Not all possible combinations of "DEPLOY_SCENARIO" are
122 supported. The name passed in to the Functest Docker container
123 must match the scenario used when the actual OPNFV platform was
124 deployed. See release note to see the list of supported scenarios.
126 **NOTE:** The scenario name is mainly used to automatically detect
127 if a test suite is runnable or not (e.g. it will prevent ONOS test suite
128 to be run on ODL scenarios). If not set, Functest will try to run the
129 default test cases that might not include SDN controller or a specific
132 **NOTE:** A HA scenario means that 3 OpenStack controller nodes are
133 deployed. It does not necessarily mean that the whole system is HA. See
134 installer release notes for details.
137 Putting all above together, when using installer 'fuel' and an invented
138 INSTALLER_IP of '10.20.0.2', the recommended command to create the
139 Functest Docker container is as follows::
141 docker run --name "FunctestContainer" -it \
142 -e "INSTALLER_IP=10.20.0.2" \
143 -e "INSTALLER_TYPE=fuel" \
144 -e "DEPLOY_SCENARIO=os-odl_l2-ovs_kvm-ha" \
145 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
146 opnfv/functest /bin/bash
148 After the *run* command, a new prompt appears which means that we are inside
149 the container and ready to move to the next step.
151 For tips on how to set up container with installer Apex, see chapter
152 `Apex Installer Tips`_.
154 Finally, three additional environment variables can also be passed in
155 to the Functest Docker Container, using the -e
156 "<EnvironmentVariable>=<Value>" mechanism. The first two of these are
157 only relevant to Jenkins CI invoked testing and **should not be used**
158 when performing manual test scenarios::
160 -e "NODE_NAME=<Test POD Name>" \
161 -e "BUILD_TAG=<Jenkins Build Tag>" \
162 -e "CI_DEBUG=<DebugTraceValue>"
164 <Test POD Name> = Symbolic name of the POD where the tests are run.
165 Visible in test results files, which are stored
166 to the database. This option is only used when
167 tests are activated under Jenkins CI control.
168 It indicates the POD/hardware where the test has
169 been run. If not specified, then the POD name is
170 defined as "Unknown" by default.
171 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
172 <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
173 Visible in test results files, which are stored
174 to the database. This option is only set when
175 tests are activated under Jenkins CI control.
176 It enables the correlation of test results,
178 are independently pushed to the results database
179 from different Jenkins jobs.
180 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
181 <DebugTraceValue> = "true" or "false"
182 Default = "false", if not specified
183 If "true" is specified, then additional debug trace
184 text can be sent to the test results file / log files
185 and also to the standard console output.
189 Some specific tips are useful for the Apex Installer case. If not using
190 Apex Installer; ignore this section.
192 In case of Triple-O based installer (like Apex) the docker container
193 needs to connect to the installer VM, so it is then required that some
194 known SSH keys are present in docker container. Since the Jumphost root
195 SSH keys are already known, easiest way is to use those using the
196 'Bind mount' method. See below for sample parameter::
198 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa
200 NOTE: You need the "sudo" when creating the container to access root
201 users ssh credentials even the docker command itself might not
204 HINT! In case of Triple-O installers you can find value for the
205 INSTALLER_IP parameter by executing command and note the returned IP
208 inst=$(sudo virsh list | grep -iEo "undercloud|instack")
209 sudo virsh domifaddr ${inst}
211 NOTE: In releases prior to Colorado, the name 'instack' was
212 used. Currently the name 'undercloud' is used.
214 You can copy the credentials file from the "stack" users home directory
215 in installer VM to Jumphost. Please check the correct IP from the
216 command above. In the example below we are using invented IP address
219 scp stack@192.168.122.89:overcloudrc .
221 Here is an example of the full docker command invocation for an Apex
222 installed system, using latest Functest docker container, for
223 illustration purposes::
225 sudo docker run -it --name "ApexFuncTestODL" \
226 -e "INSTALLER_IP=192.168.122.89" \
227 -e "INSTALLER_TYPE=apex" \
228 -e "DEPLOY_SCENARIO=os-odl_l2-nofeature-ha" \
229 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa \
230 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
231 opnfv/functest /bin/bash
233 Compass installer local development env usage Tips
234 --------------------------------------------------
235 In the compass-functest local test case check and development environment,
236 in order to get openstack service inside the functest container, some
237 parameters should be configured during container creation, which are
238 hard to guess for freshman. This section will provide the guideline, the
239 parameters values are defaults here, which should be adjusted according
240 to the settings, the complete steps are given here so as not to appear
243 1, Pull Functest docker image from public dockerhub::
245 docker pull opnfv/functest:<Tag>
247 <Tag> here can be "brahmaputra.1.0", "colorado.1.0", etc.
248 Tag omitted means the latest docker image::
250 docker pull opnfv/functest
252 2, Functest Docker container creation
254 To make a file used for the environment, such as 'functest-docker-env'::
256 OS_AUTH_URL=http://172.16.1.222:35357/v2.0
260 OS_VOLUME_API_VERSION=2
261 OS_PROJECT_NAME=admin
262 INSTALLER_TYPE=compass
263 INSTALLER_IP=192.168.200.2
264 EXTERNAL_NETWORK=ext-net
266 Note: please adjust the content according to the environment, such as
267 'TENANT_ID' maybe used for some special cases.
269 Then to create the Functest docker::
271 docker run --privileged=true --rm -t \
272 --env-file functest-docker-env \
273 --name <Functest_Container_Name> \
274 opnfv/functest:<Tag> /bin/bash
276 3, To attach Functest container
278 Before trying to attach the Functest container, the status can be checked by::
282 to attach the 'Up' status Functest container and start bash mode::
284 docker exec -it <Functest_Container_Name> bash
286 4, Functest environment preparation and check
288 To see the Section below `Preparing the Functest environment`_.
290 Functest docker container directory structure
291 ---------------------------------------------
292 Inside the Functest docker container, the following directory structure
293 should now be in place::
322 Underneath the '/home/opnfv/' directory, the Functest docker container
323 includes two main directories:
325 * The **functest** directory stores configuration files (e.g. the
326 OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
327 the **data** directory stores a 'cirros' test image used in some
328 functional tests and the **results** directory stores some temporary
330 * The **repos** directory holds various repositories. The directories
331 are used for the installation of the needed tooling (e.g. rally) or
332 for the retrieval of feature projects scenarios (e.g. promise)
334 The structure under the **functest** repository can be described as
340 |-- run_unit_tests.sh
342 |-- test-requirements.txt
346 | `--traffic-profile-guidelines.rst
349 | |-- config_install_env.sh
350 | `-- docker_remote_api
364 | |-- check_deployment.py
365 | |-- config_functest.yaml
366 | |-- config_patch.yaml
367 | |-- generate_report.py
371 | |-- tier_builder.py
372 | `-- tier_handler.py
377 | |-- functest-complete.sh
382 | |-- pytest_suite_runner.py
401 |-- functest_logger.py
402 |-- functest_utils.py
404 |-- openstack_clean.py
405 |-- openstack_snapshot.py
406 |-- openstack_tacker.py
407 `-- openstack_utils.py
410 (Note: All *.pyc files removed from above list for brevity...)
412 We may distinguish several directories, the first level has 4 directories:
414 * **commons**: This directory is dedicated for storage of traffic
415 profile or any other test inputs that could be reused by any test
417 * **docker**: This directory includes the needed files and tools to
418 build the Functest Docker image.
419 * **docs**: This directory includes documentation: Release Notes,
420 User Guide, Configuration Guide and Developer Guide.
421 * **functest**: This directory contains all the code needed to run
422 functest internal cases and OPNFV onboarded feature or VNF test cases.
424 Functest directory has 6 directories:
425 * **ci**: This directory contains test structure definition files
426 (e.g <filename>.yaml) and bash shell/python scripts used to
427 configure and execute Functional tests. The test execution script
428 can be executed under the control of Jenkins CI jobs.
429 * **cli**: This directory holds the python based Functest CLI utility
430 source code, which is based on the Python 'click' framework.
431 * **core**: This directory holds the python based Functest core
432 source code. Three abstraction classes have been created to ease
433 the integration of internal, feature or vnf cases.
434 * **opnfv_tests**: This directory includes the scripts required by
435 Functest internal test cases and other feature projects test cases.
436 * **tests**: This directory includes the functest unit tests
437 * **utils**: this directory holds Python source code for some general
438 purpose helper utilities, which testers can also re-use in their
439 own test code. See for an example the Openstack helper utility:
440 'openstack_utils.py'.
442 Useful Docker commands
443 ----------------------
444 When typing **exit** in the container prompt, this will cause exiting
445 the container and probably stopping it. When stopping a running Docker
446 container all the changes will be lost, there is a keyboard shortcut
447 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
448 reconnect to the running container **DO NOT** use the *run* command
449 again (since it will create a new container), use the *exec* or *attach*
452 docker ps # <check the container ID from the output>
453 docker exec -ti <CONTAINER_ID> /bin/bash
455 There are other useful Docker commands that might be needed to manage possible
456 issues with the containers.
458 List the running containers::
462 List all the containers including the stopped ones::
466 Start a stopped container named "FunTest"::
470 Attach to a running container named "StrikeTwo"::
472 docker attach StrikeTwo
474 It is useful sometimes to remove a container if there are some problems::
476 docker rm <CONTAINER_ID>
478 Use the *-f* option if the container is still running, it will force to
481 docker rm -f <CONTAINER_ID>
483 Check the Docker documentation dockerdocs_ for more information.
485 Preparing the Functest environment
486 ----------------------------------
487 Once the Functest docker container is up and running, the required
488 Functest environment needs to be prepared. A custom built **functest**
489 CLI utility is available to perform the needed environment preparation
490 action. Once the environment is prepared, the **functest** CLI utility
491 can be used to run different functional tests. The usage of the
492 **functest** CLI utility to run tests is described further in the
493 Functest User Guide `OPNFV_FuncTestUserGuide`_
495 Prior to commencing the Functest environment preparation, we can check
496 the initial status of the environment. Issue the **functest env status**
497 command at the prompt::
500 Functest environment is not installed.
502 Note: When the Functest environment is prepared, the command will
503 return the status: "Functest environment ready to run tests."
505 To prepare the Functest docker container for test case execution, issue
506 the **functest env prepare** command at the prompt::
510 This script will make sure that the requirements to run the tests are
511 met and will install the needed libraries and tools by all Functest
512 test cases. It should be run only once every time the Functest docker
513 container is started from scratch. If you try to run this command, on
514 an already prepared environment, you will be prompted whether you really
515 want to continue or not::
518 It seems that the environment has been already prepared.
519 Do you want to do it again? [y|n]
521 (Type 'n' to abort the request, or 'y' to repeat the
522 environment preparation)
525 To list some basic information about an already prepared Functest
526 docker container environment, issue the **functest env show** at the
530 +======================================================+
531 | Functest Environment info |
532 +======================================================+
533 | INSTALLER: apex, 192.168.122.89 |
534 | SCENARIO: os-odl_l2-nofeature-ha |
536 | GIT BRANCH: master |
537 | GIT HASH: 5bf1647dec6860464eeb082b2875798f0759aa91 |
538 | DEBUG FLAG: false |
539 +------------------------------------------------------+
541 +------------------------------------------------------+
545 INSTALLER: Displays the INSTALLER_TYPE value
547 and the INSTALLER_IP value
548 - here = "192.168.122.89"
549 SCENARIO: Displays the DEPLOY_SCENARIO value
550 - here = "os-odl_l2-nofeature-ha"
551 POD: Displays the value passed in NODE_NAME
553 GIT BRANCH: Displays the git branch of the OPNFV Functest
554 project repository included in the Functest
557 (In first official colorado release
558 would be "colorado.1.0")
559 GIT HASH: Displays the git hash of the OPNFV Functest
560 project repository included in the Functest
562 - here = "5bf1647dec6860464eeb082b2875798f0759aa91"
563 DEBUG FLAG: Displays the CI_DEBUG value
566 NOTE: In Jenkins CI runs, an additional item "BUILD TAG"
567 would also be listed. The value is set by Jenkins CI.
569 Finally, the **functest** CLI has a **--help** options:
573 functest --help Usage: functest [OPTIONS] COMMAND [ARGS]...
576 --version Show the version and exit.
577 -h, --help Show this message and exit.
586 Usage: functest env [OPTIONS] COMMAND [ARGS]...
589 -h, --help Show this message and exit.
592 prepare Prepares the Functest environment.
593 show Shows information about the current...
594 status Checks if the Functest environment is ready...
596 Checking Openstack and credentials
597 ----------------------------------
598 It is recommended and fairly straightforward to check that Openstack
599 and credentials are working as expected.
601 Once the credentials are there inside the container, they should be
602 sourced before running any Openstack commands::
604 source /home/opnfv/functest/conf/openstack.creds
606 After this, try to run any OpenStack command to see if you get any
607 output, for instance::
611 This will return a list of the actual users in the OpenStack
612 deployment. In any other case, check that the credentials are sourced::
616 This command must show a set of environment variables starting with
619 OS_REGION_NAME=RegionOne
620 OS_DEFAULT_DOMAIN=default
621 OS_PROJECT_NAME=admin
623 OS_AUTH_STRATEGY=keystone
624 OS_AUTH_URL=http://172.30.10.3:5000/v2.0
627 OS_ENDPOINT_TYPE=internalURL
630 If the OpenStack command still does not show anything or complains
631 about connectivity issues, it could be due to an incorrect url given to
632 the OS_AUTH_URL environment variable. Check the deployment settings.
636 If you need to connect to a server that is TLS-enabled (the auth URL
637 begins with "https") and it uses a certificate from a private CA or a
638 self-signed certificate, then you will need to specify the path to an
639 appropriate CA certificate to use, to validate the server certificate
640 with the environment variable OS_CACERT::
643 /etc/ssl/certs/ca.crt
645 However, this certificate does not exist in the container by default.
646 It has to be copied manually from the OpenStack deployment. This can be
649 #. Create manually that file and copy the contents from the OpenStack
651 #. (Recommended) Add the file using a Docker volume when starting the
654 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
656 You might need to export OS_CACERT environment variable inside the
659 export OS_CACERT=/etc/ssl/certs/ca.crt
661 Certificate verification can be turned off using OS_INSECURE=true. For
662 example, Fuel uses self-signed cacerts by default, so an pre step would
665 export OS_INSECURE=true
669 If your Jumphost node is operating behind a http proxy, then there are
670 2 places where some special actions may be needed to make operations
673 #. Initial installation of docker engine First, try following the
674 official Docker documentation for Proxy settings. Some issues were
675 experienced on CentOS 7 based Jumphost. Some tips are documented
676 in section: `Docker Installation on CentOS behind http proxy`_
679 #. Execution of the Functest environment preparation inside the
680 created docker container Functest needs internet access to
681 download some resources for some test cases. This might not
682 work properly if the Jumphost is connecting to internet
683 through a http Proxy.
685 If that is the case, make sure the resolv.conf and the needed
686 http_proxy and https_proxy environment variables, as well as the
687 'no_proxy' environment variable are set correctly::
689 # Make double sure that the 'no_proxy=...' line in the
690 # 'openstack.creds' file is commented out first. Otherwise, the
691 # values set into the 'no_proxy' environment variable below will
692 # be ovewrwritten, each time the command
693 # 'source ~/functest/conf/openstack.creds' is issued.
696 sed -i 's/export no_proxy/#export no_proxy/' openstack.creds
697 source ./openstack.creds
699 # Next calculate some IP addresses for which http_proxy
700 # usage should be excluded:
702 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
704 adminURL_IP=$(openstack catalog show identity | \
705 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
707 export http_proxy="<your http proxy settings>"
708 export https_proxy="<your https proxy settings>"
709 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
711 # Ensure that "git" uses the http_proxy
712 # This may be needed if your firewall forbids SSL based git fetch
713 git config --global http.sslVerify True
714 git config --global http.proxy <Your http proxy settings>
716 Validation check: Before running **'functest env prepare'** CLI command,
717 make sure you can reach http and https sites from inside the Functest
720 For example, try to use the **nc** command from inside the functest
724 Connection to opnfv.org 80 port [tcp/http] succeeded!
727 Connection to opnfv.org 443 port [tcp/https] succeeded!
729 Note: In a Jumphost node based on the CentOS family OS, the **nc**
730 commands might not work. You can use the **curl** command instead.
732 curl http://www.opnfv.org:80
733 <HTML><HEAD><meta http-equiv="content-type"
738 curl https://www.opnfv.org:443
739 <HTML><HEAD><meta http-equiv="content-type"
744 (Ignore the content. If command returns a valid HTML page, it proves
747 Docker Installation on CentOS behind http proxy
748 -----------------------------------------------
749 This section is applicable for CentOS family OS on Jumphost which
750 itself is behind a proxy server. In that case, the instructions below
751 should be followed **before** installing the docker engine::
753 1) # Make a directory '/etc/systemd/system/docker.service.d'
754 # if it does not exist
755 sudo mkdir /etc/systemd/system/docker.service.d
757 2) # Create a file called 'env.conf' in that directory with
758 # the following contents:
760 EnvironmentFile=-/etc/sysconfig/docker
762 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
763 # with the following contents:
764 HTTP_PROXY="<Your http proxy settings>"
765 HTTPS_PROXY="<Your https proxy settings>"
766 http_proxy="${HTTP_PROXY}"
767 https_proxy="${HTTPS_PROXY}"
769 4) # Reload the daemon
770 systemctl daemon-reload
772 5) # Sanity check - check the following docker settings:
773 systemctl show docker | grep -i env
777 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
778 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
780 Now follow the instructions in [`InstallDockerCentOS`_] to download
781 and install the **docker-engine**. The instructions conclude with a
782 "test pull" of a sample "Hello World" docker container. This should now
783 work with the above pre-requisite actions.
785 .. _dockerdocs: https://docs.docker.com/
786 .. _dockerhub: https://hub.docker.com/r/opnfv/functest/
787 .. _Proxy: https://docs.docker.com/engine/admin/systemd/#http-proxy
788 .. _FunctestDockerTags: https://hub.docker.com/r/opnfv/functest/tags/
789 .. _InstallDockerCentOS: https://docs.docker.com/engine/installation/linux/centos/
790 .. _OPNFV_FuncTestUserGuide: http://artifacts.opnfv.org/functest/docs/userguide/index.html