1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
5 Pulling the Docker image
6 ------------------------
7 Pull the Functest Docker image ('opnfv/functest') from the public
8 dockerhub registry under the OPNFV account: [dockerhub_], with the
9 following docker command::
11 docker pull opnfv/functest:<TagIdentifier>
13 where <TagIdentifier> identifies a release of the Functest docker
14 container image in the public dockerhub registry. There are many tags
15 created automatically by the CI mechanisms, and you must ensure you
16 pull an image with the **correct tag** to match the OPNFV software
17 release installed in your environment. All available tagged images can
18 be seen from location [FunctestDockerTags_]. For example, when running
19 on the first official release of the OPNFV Colorado system platform,
20 tag "colorado.1.0" is needed. Pulling other tags might cause some
21 problems while running the tests. If you need to specifically pull the
22 latest Functest docker image, then omit the tag argument::
24 docker pull opnfv/functest
26 After pulling the Docker image, check that it is available with the
27 following docker command::
29 [functester@jumphost ~]$ docker images
30 REPOSITORY TAG IMAGE ID CREATED SIZE
31 opnfv/functest latest 8cd6683c32ae 2 weeks ago 1.611 GB
32 opnfv/functest brahmaputra.3.0 94b78faa94f7 4 weeks ago 874.9 MB
33 hello-world latest 94df4f0ce8a4 7 weeks ago 967 B
35 Docker images pulled without a tag specifier bear the implicitly
36 assigned label "latest", as seen above.
38 The Functest docker container environment can -in principle- be also
39 used with non-OPNFV official installers (e.g. 'devstack'), with the
40 **disclaimer** that support for such environments is outside of the
41 scope and responsibility of the OPNFV project.
43 Accessing the Openstack credentials
44 -----------------------------------
45 OpenStack credentials are mandatory and can be retrieved in different
46 ways. From inside the running Functest docker container the
47 "functest env prepare" command will automatically look for the
48 Openstack credentials file "/home/opnfv/functest/conf/openstack.creds"
49 and retrieve it unless the file already exists. This Functest
50 environment preparation step is described later in this document.
52 WARNING: When the installer type is "joid" you have to have the
53 credentials file inside the running container **before** initiating the
54 functest environment preparation. For that reason you have to choose
55 either one of the options below, since the automated copying does not
58 You can also specifically pass in the needed file prior to running the
59 environment preparation either:
61 * by using the -v option when creating the Docker container. This is
62 referred to in docker documentation as "Bind Mounting". See the
63 usage of this parameter in the following chapter.
64 * or creating a local file '/home/opnfv/functest/conf/openstack.creds'
65 inside the running container with the credentials in it. Consult
66 your installer guide for further details. This is however not
67 instructed in this document.
69 NOTE: When the installer type is "fuel" and virtualized deployment
70 is used, there you have to explicitly fetch the credentials file
71 executing the following sequence
73 #. Create a container as described in next chapter but do not
74 "Bind Mount" the credentials
75 #. Log in to container and execute the following command. Replace
76 the IP with installer address after the "-a" parameter::
78 $repos_dir/releng/utils/fetch_os_creds.sh \
79 -d /home/opnfv/functest/conf/openstack.creds \
83 ( -d specifies the full path to the Openstack credential file
84 -i specifies the INSTALLER_TYPE
85 -a specifies the INSTALLER_IP
86 -v indicates a virtualized environment and takes no arguments )
88 #. Continue with your testing, initiate functest environment
89 preparation, run tests etc.
91 In proxified environment you may need to change the credentials file.
92 There are some tips in chapter: `Proxy support`_
94 Functest Docker parameters
95 --------------------------
96 This chapter explains how to run a container for executing functest
97 test suites. Numbered list below explains some details of the
98 recommended parameters for invoking docker container
100 #. It is a good practice to assign a precise container name through
101 the **--name** option.
103 #. Assign parameter for installer type::
105 -e "INSTALLER_TYPE=<type>"
106 # Use one of following apex, compass, fuel or joid
108 #. Functest needs to know the IP of the installer::
110 -e "INSTALLER_IP=<Specific IP Address>"
112 #. Credentials for accessing the Openstack.
113 Most convenient way of passing them to container is by having a
114 local copy of the credentials file in Jumphost and then using the
115 **-v** option. In the example we have local file by the name of
116 "overcloudrc" and we are using that as an argument::
118 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds
120 The credentials file needs to exist in the Docker container
121 under the path: '/home/opnfv/functest/conf/openstack.creds'.
123 WARNING: If you are using the Joid installer, you must pass the
124 credentials using the **-v** option. See the section
125 `Accessing the Openstack credentials`_ above.
127 #. Passing deployment scenario
128 When running Functest against any of the supported OPNFV scenarios,
129 it is recommended to include also the environment variable
130 **DEPLOY_SCENARIO**. The **DEPLOY_SCENARIO** environment variable
131 is passed with the format::
133 -e "DEPLOY_SCENARIO=os-<controller>-<nfv_feature>-<ha_mode>"
135 os = OpenStack (No other VIM choices currently available)
136 controller is one of ( nosdn | odl_l2 | odl_l3 | onos )
137 nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
138 If several features are pertinent then use the underscore
139 character '_' to separate each feature (e.g. ovs_kvm)
140 'nofeature' indicates no NFV feature is deployed
141 ha_mode is one of ( ha | noha )
143 **NOTE:** Not all possible combinations of "DEPLOY_SCENARIO" are
144 supported. The name passed in to the Functest Docker container
145 must match the scenario used when the actual OPNFV platform was
148 Putting all above together, when using installer 'fuel' and an invented
149 INSTALLER_IP of '10.20.0.2', the recommended command to create the
150 Functest Docker container is as follows::
152 docker run --name "FunctestContainer" -it \
153 -e "INSTALLER_IP=10.20.0.2" \
154 -e "INSTALLER_TYPE=fuel" \
155 -e "DEPLOY_SCENARIO=os-odl_l2-ovs_kvm-ha" \
156 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
157 opnfv/functest /bin/bash
159 After the *run* command, a new prompt appears which means that we are inside
160 the container and ready to move to the next step.
162 For tips on how to set up container with installer Apex, see chapter
163 `Apex Installer Tips`_.
165 Finally, three additional environment variables can also be passed in
166 to the Functest Docker Container, using the -e
167 "<EnvironmentVariable>=<Value>" mechanism. The first two of these are
168 only relevant to Jenkins CI invoked testing and **should not be used**
169 when performing manual test scenarios::
171 -e "NODE_NAME=<Test POD Name>" \
172 -e "BUILD_TAG=<Jenkins Build Tag>" \
173 -e "CI_DEBUG=<DebugTraceValue>"
175 <Test POD Name> = Symbolic name of the POD where the tests are run.
176 Visible in test results files, which are stored
177 to the database. This option is only used when
178 tests are activated under Jenkins CI control.
179 It indicates the POD/hardware where the test has
180 been run. If not specified, then the POD name is
181 defined as "Unknown" by default.
182 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
183 <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
184 Visible in test results files, which are stored
185 to the database. This option is only set when
186 tests are activated under Jenkins CI control.
187 It enables the correlation of test results,
189 are independently pushed to the results datbase
190 from different Jenkins jobs.
191 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
192 <DebugTraceValue> = "true" or "false"
193 Default = "false", if not specified
194 If "true" is specified, then additional debug trace
195 text can be sent to the test results file / log files
196 and also to the standard console output.
200 Some specific tips are useful for the Apex Installer case. If not using
201 Apex Installer; ignore this section.
203 In case of Triple-O based installer (like Apex) the docker container
204 needs to connect to the installer VM, so it is then required that some
205 known SSH keys are present in docker container. Since the Jumphost root
206 SSH keys are already known, easiest way is to use those using the
207 'Bind mount' method. See below for sample parameter::
209 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa
211 NOTE: You need the "sudo" when creating the container to access root
212 users ssh credentials even the docker command itself might not
215 HINT! In case of Triple-O installers you can find value for the
216 INSTALLER_IP parameter by executing command and note the returned IP
219 inst=$(sudo virsh list | grep -iEo "undercloud|instack")
220 sudo virsh domifaddr ${inst}
222 NOTE: In releases prior to Colorado, the name 'instack' was
223 used. Currently the name 'undercloud' is used.
225 You can copy the credentials file from the "stack" users home directory
226 in installer VM to Jumphost. Please check the correct IP from the
227 command above. In the example below we are using invented IP address
230 scp stack@192.168.122.89:overcloudrc .
232 Here is an example of the full docker command invocation for an Apex
233 installed system, using latest Functest docker container, for
234 illustration purposes::
236 sudo docker run -it --name "ApexFuncTestODL" \
237 -e "INSTALLER_IP=192.168.122.89" \
238 -e "INSTALLER_TYPE=apex" \
239 -e "DEPLOY_SCENARIO=os-odl_l2-nofeature-ha" \
240 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa \
241 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
242 opnfv/functest /bin/bash
244 Functest docker container directory structure
245 ---------------------------------------------
246 Inside the Functest docker container, the following directory structure
247 should now be in place::
266 Underneath the '/home/opnfv/' directory, the Functest docker container
267 includes two main directories:
269 * The **functest** directory stores configuration files (e.g. the
270 OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
271 the **data** directory stores a 'cirros' test image used in some
272 functional tests and the **results** directory stores some temporary
274 * The **repos** directory holds various repositories. The directory
275 '/home/opnfv/repos/functest' is used to prepare the needed Functest
276 environment and to run the tests. The other repository directories
277 are used for the installation of the needed tooling (e.g. rally) or
278 for the retrieval of feature projects scenarios (e.g. promise)
280 The structure under the **functest** repository can be described as
289 | |-- config_functest.yaml
294 | |-- tier_builder.py
295 | `-- tier_handler.py
300 | |-- functest-complete.sh
305 | `--traffic-profile-guidelines.rst
308 | |-- config_install_env.sh
309 | `-- requirements.pip
327 |-- functest_logger.py
328 |-- functest_utils.py
329 |-- openstack_clean.py
330 |-- openstack_snapshot.py
331 `-- openstack_utils.py
333 (Note: All *.pyc files removed from above list for brevity...)
335 We may distinguish 7 different directories:
337 * **ci**: This directory contains test structure definition files
338 (e.g <filename>.yaml) and bash shell/python scripts used to
339 configure and execute Functional tests. The test execution script
340 can be executed under the control of Jenkins CI jobs.
341 * **cli**: This directory holds the python based Functest CLI utility
342 source code, which is based on the Python 'click' framework.
343 * **commons**: This directory is dedicated for storage of traffic
344 profile or any other test inputs that could be reused by any test
346 * **docker**: This directory includes the needed files and tools to
347 build the Funtest Docker container image.
348 * **docs**: This directory includes documentation: Release Notes,
349 User Guide, Configuration Guide and Developer Guide. Test results
350 are also located in a sub--directory called 'results'.
351 * **testcases**: This directory includes the scripts required by
352 Functest internal test cases and other feature projects test cases.
353 * **utils**: this directory holds Python source code for some general
354 purpose helper utilities, which testers can also re-use in their
355 own test code. See for an example the Openstack helper utility:
356 'openstack_utils.py'.
358 Useful Docker commands
359 ----------------------
360 When typing **exit** in the container prompt, this will cause exiting
361 the container and probably stopping it. When stopping a running Docker
362 container all the changes will be lost, there is a keyboard shortcut
363 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
364 reconnect to the running container **DO NOT** use the *run* command
365 again (since it will create a new container), use the *exec* or *attach*
368 docker ps # <check the container ID from the output>
369 docker exec -ti <CONTAINER_ID> /bin/bash
371 There are other useful Docker commands that might be needed to manage possible
372 issues with the containers.
374 List the running containers::
378 List all the containers including the stopped ones::
382 Start a stopped container named "FunTest"::
386 Attach to a running container named "StrikeTwo"::
388 docker attach StrikeTwo
390 It is useful sometimes to remove a container if there are some problems::
392 docker rm <CONTAINER_ID>
394 Use the *-f* option if the container is still running, it will force to
397 docker -f rm <CONTAINER_ID>
399 Check the Docker documentation dockerdocs_ for more information.
401 Preparing the Functest environment
402 ----------------------------------
403 Once the Functest docker container is up and running, the required
404 Functest environment needs to be prepared. A custom built **functest**
405 CLI utility is available to perform the needed environment preparation
406 action. Once the environment is prepared, the **functest** CLI utility
407 can be used to run different functional tests. The usage of the
408 **functest** CLI utility to run tests is described further in the
409 Functest User Guide `OPNFV_FuncTestUserGuide`_
411 Prior to commencing the Functest environment preparation, we can check
412 the initial status of the environment. Issue the **functest env status**
413 command at the prompt::
416 Functest environment is not installed.
418 Note: When the Functest environment is prepared, the command will
419 return the status: "Functest environment ready to run tests."
421 To prepare the Functest docker container for test case execution, issue
422 the **functest env prepare** command at the prompt::
426 This script will make sure that the requirements to run the tests are
427 met and will install the needed libraries and tools by all Functest
428 test cases. It should be run only once every time the Functest docker
429 container is started from scratch. If you try to run this command, on
430 an already prepared enviroment, you will be prompted whether you really
431 want to continue or not::
434 It seems that the environment has been already prepared.
435 Do you want to do it again? [y|n]
437 (Type 'n' to abort the request, or 'y' to repeat the
438 environment preparation)
441 To list some basic information about an already prepared Functest
442 docker container environment, issue the **functest env show** at the
446 +======================================================+
447 | Functest Environment info |
448 +======================================================+
449 | INSTALLER: apex, 192.168.122.89 |
450 | SCENARIO: os-odl_l2-nofeature-ha |
452 | GIT BRANCH: master |
453 | GIT HASH: 5bf1647dec6860464eeb082b2875798f0759aa91 |
454 | DEBUG FLAG: false |
455 +------------------------------------------------------+
457 +------------------------------------------------------+
461 INSTALLER: Displays the INSTALLER_TYPE value
463 and the INSTALLER_IP value
464 - here = "192.168.122.89"
465 SCENARIO: Displays the DEPLOY_SCENARIO value
466 - here = "os-odl_l2-nofeature-ha"
467 POD: Displays the value passed in NODE_NAME
469 GIT BRANCH: Displays the git branch of the OPNFV Functest
470 project repository included in the Functest
473 (In first official colorado release
474 would be "colorado.1.0")
475 GIT HASH: Displays the git hash of the OPNFV Functest
476 project repository included in the Functest
478 - here = "5bf1647dec6860464eeb082b2875798f0759aa91"
479 DEBUG FLAG: Displays the CI_DEBUG value
482 NOTE: In Jenkins CI runs, an additional item "BUILD TAG"
483 would also be listed. The valaue is set by Jenkins CI.
485 Finally, the **functest** CLI has a **--help** options:
489 functest --help Usage: functest [OPTIONS] COMMAND [ARGS]...
492 --version Show the version and exit.
493 -h, --help Show this message and exit.
502 Usage: functest env [OPTIONS] COMMAND [ARGS]...
505 -h, --help Show this message and exit.
508 prepare Prepares the Functest environment.
509 show Shows information about the current...
510 status Checks if the Functest environment is ready...
512 Checking Openstack and credentials
513 ----------------------------------
514 It is recommended and fairly straightforward to check that Openstack
515 and credentials are working as expected.
517 Once the credentials are there inside the container, they should be
518 sourced before running any Openstack commands::
520 source /home/opnfv/functest/conf/openstack.creds
522 After this, try to run any OpenStack command to see if you get any
523 output, for instance::
527 This will return a list of the actual users in the OpenStack
528 deployment. In any other case, check that the credentials are sourced::
532 This command must show a set of environment variables starting with
535 OS_REGION_NAME=RegionOne
536 OS_DEFAULT_DOMAIN=default
537 OS_PROJECT_NAME=admin
539 OS_AUTH_STRATEGY=keystone
540 OS_AUTH_URL=http://172.30.10.3:5000/v2.0
543 OS_ENDPOINT_TYPE=internalURL
546 If the OpenStack command still does not show anything or complains
547 about connectivity issues, it could be due to an incorrect url given to
548 the OS_AUTH_URL environment variable. Check the deployment settings.
552 If you need to connect to a server that is TLS-enabled (the auth URL
553 begins with "https") and it uses a certificate from a private CA or a
554 self-signed certificate, then you will need to specify the path to an
555 appropriate CA certificate to use, to validate the server certificate
556 with the environment variable OS_CACERT::
559 /etc/ssl/certs/ca.crt
561 However, this certificate does not exist in the container by default.
562 It has to be copied manually from the OpenStack deployment. This can be
565 #. Create manually that file and copy the contents from the OpenStack
567 #. (Recommended) Add the file using a Docker volume when starting the
570 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
572 You might need to export OS_CACERT environment variable inside the
575 export OS_CACERT=/etc/ssl/certs/ca.crt
577 Certificate verification can be turned off using OS_INSECURE=true. For
578 example, Fuel uses self-signed cacerts by default, so an pre step would
581 export OS_INSECURE=true
585 If your Jumphost node is operating behind a http proxy, then there are
586 2 places where some special actions may be needed to make operations
589 #. Initial installation of docker engine First, try following the
590 official Docker documentation for Proxy settings. Some issues were
591 experienced on CentOS 7 based Jumphost. Some tips are documented
592 in section: `Docker Installation on CentOS behind http proxy`_
595 #. Execution of the Functest environment preparation inside the
596 created docker container Functest needs internet access to
597 download some resources for some test cases. This might not
598 work properly if the Jumphost is connecting to internet
599 through a http Proxy.
601 If that is the case, make sure the resolv.conf and the needed
602 http_proxy and https_proxy environment variables, as well as the
603 'no_proxy' environment variable are set correctly::
605 # Make double sure that the 'no_proxy=...' line in the
606 # 'openstack.creds' file is commented out first. Otherwise, the
607 # values set into the 'no_proxy' environment variable below will
608 # be ovewrwritten, each time the command
609 # 'source ~/functest/conf/openstack.creds' is issued.
612 sed -i 's/export no_proxy/#export no_proxy/' openstack.creds
613 source ./openstack.creds
615 # Next calculate some IP addresses for which http_proxy
616 # usage should be excluded:
618 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
620 adminURL_IP=$(openstack catalog show identity | \
621 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
623 export http_proxy="<your http proxy settings>"
624 export https_proxy="<your https proxy settings>"
625 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
627 # Ensure that "git" uses the http_proxy
628 # This may be needed if your firewall forbids SSL based git fetch
629 git config --global http.sslVerify True
630 git config --global http.proxy <Your http proxy settings>
632 Validation check: Before running **'functest env prepare'** CLI command,
633 make sure you can reach http and https sites from inside the Functest
636 For example, try to use the **nc** command from inside the functest
640 Connection to google.com 80 port [tcp/http] succeeded!
643 Connection to google.com 443 port [tcp/https] succeeded!
645 Note: In a Jumphost node based on the CentOS family OS, the **nc**
646 commands might not work. You can use the **curl** command instead.
648 curl http://www.google.com:80
649 <HTML><HEAD><meta http-equiv="content-type"
654 curl https://www.google.com:443
655 <HTML><HEAD><meta http-equiv="content-type"
660 (Ignore the content. If command returns a valid HTML page, it proves
663 Docker Installation on CentOS behind http proxy
664 -----------------------------------------------
665 This section is applicable for CentOS family OS on Jumphost which
666 itself is behind a proxy server. In that case, the instructions below
667 should be followed **before** installing the docker engine::
669 1) # Make a directory '/etc/systemd/system/docker.service.d'
670 # if it does not exist
671 sudo mkdir /etc/systemd/system/docker.service.d
673 2) # Create a file called 'env.conf' in that directory with
674 # the following contents:
676 EnvironmentFile=-/etc/sysconfig/docker
678 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
679 # with the following contents:
680 HTTP_PROXY="<Your http proxy settings>"
681 HTTPS_PROXY="<Your https proxy settings>"
682 http_proxy="${HTTP_PROXY}"
683 https_proxy="${HTTPS_PROXY}"
685 4) # Reload the daemon
686 systemctl daemon-reload
688 5) # Sanity check - check the following docker settings:
689 systemctl show docker | grep -i env
693 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
694 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
696 Now follow the instructions in [`InstallDockerCentOS`_] to download
697 and install the **docker-engine**. The instructions conclude with a
698 "test pull" of a sample "Hello World" docker container. This should now
699 work with the above pre-requisite actions.
701 .. _dockerdocs: https://docs.docker.com/
702 .. _dockerhub: https://hub.docker.com/r/opnfv/functest/
703 .. _Proxy: https://docs.docker.com/engine/admin/systemd/#http-proxy
704 .. _FunctestDockerTags: https://hub.docker.com/r/opnfv/functest/tags/
705 .. _InstallDockerCentOS: https://docs.docker.com/engine/installation/linux/centos/
706 .. _OPNFV_FuncTestUserGuide: http://artifacts.opnfv.org/functest/docs/userguide/index.html