1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
5 Pulling the Docker image
6 ------------------------
7 Pull the Functest Docker image ('opnfv/functest') from the public
8 dockerhub registry under the OPNFV account: [dockerhub_], with the
9 following docker command::
11 docker pull opnfv/functest:<TagIdentifier>
13 where <TagIdentifier> identifies a release of the Functest docker
14 container image in the public dockerhub registry. There are many tags
15 created automatically by the CI mechanisms, and you must ensure you
16 pull an image with the **correct tag** to match the OPNFV software
17 release installed in your environment. All available tagged images can
18 be seen from location [FunctestDockerTags_]. For example, when running
19 on the first official release of the OPNFV Colorado system platform,
20 tag "colorado.1.0" is needed. Pulling other tags might cause some
21 problems while running the tests. If you need to specifically pull the
22 latest Functest docker image, then omit the tag argument::
24 docker pull opnfv/functest
26 After pulling the Docker image, check that it is available with the
27 following docker command::
29 [functester@jumphost ~]$ docker images
30 REPOSITORY TAG IMAGE ID CREATED SIZE
31 opnfv/functest latest 8cd6683c32ae 2 weeks ago 1.611 GB
32 opnfv/functest brahmaputra.3.0 94b78faa94f7 4 weeks ago 874.9 MB
33 hello-world latest 94df4f0ce8a4 7 weeks ago 967 B
35 Docker images pulled without a tag specifier bear the implicitly
36 assigned label "latest", as seen above.
38 The Functest docker container environment can -in principle- be also
39 used with non-OPNFV official installers (e.g. 'devstack'), with the
40 **disclaimer** that support for such environments is outside of the
41 scope and responsibility of the OPNFV project.
43 Accessing the Openstack credentials
44 -----------------------------------
45 OpenStack credentials are mandatory and can be retrieved in different
46 ways. From inside the running Functest docker container the
47 "functest env prepare" command will automatically look for the
48 Openstack credentials file "/home/opnfv/functest/conf/openstack.creds"
49 and retrieve it unless the file already exists. This Functest
50 environment preparation step is described later in this document.
52 WARNING: When the installer type is "joid" you have to have the
53 credentials file inside the running container **before** initiating the
54 functest environment preparation. For that reason you have to choose
55 either one of the options below, since the automated copying does not
58 You can also specifically pass in the needed file prior to running the
59 environment preparation either:
61 * by using the -v option when creating the Docker container. This is
62 referred to in docker documentation as "Bind Mounting". See the
63 usage of this parameter in the following chapter.
64 * or creating a local file '/home/opnfv/functest/conf/openstack.creds'
65 inside the running container with the credentials in it. Consult
66 your installer guide for further details. This is however not
67 instructed in this document.
69 NOTE: When the installer type is "fuel" and virtualized deployment
70 is used, there you have to explicitly fetch the credentials file
71 executing the following sequence
73 #. Create a container as described in next chapter but do not
74 "Bind Mount" the credentials
75 #. Log in to container and execute the following command. Replace
76 the IP with installer address after the "-a" parameter::
78 $repos_dir/releng/utils/fetch_os_creds.sh \
79 -d /home/opnfv/functest/conf/openstack.creds \
83 ( -d specifies the full path to the Openstack credential file
84 -i specifies the INSTALLER_TYPE
85 -a specifies the INSTALLER_IP
86 -v indicates a virtualized environment and takes no arguments )
88 #. Continue with your testing, initiate functest environment
89 preparation, run tests etc.
91 In proxified environment you may need to change the credentials file.
92 There are some tips in chapter: `Proxy support`_
94 Functest Docker parameters
95 --------------------------
96 This chapter explains how to run a container for executing functest
97 test suites. Numbered list below explains some details of the
98 recommended parameters for invoking docker container
100 #. It is a good practice to assign a precise container name through
101 the **--name** option.
103 #. Assign parameter for installer type::
105 -e "INSTALLER_TYPE=<type>"
106 # Use one of following apex, compass, fuel or joid
108 #. Functest needs to know the IP of the installer::
110 -e "INSTALLER_IP=<Specific IP Address>"
112 #. Credentials for accessing the Openstack.
113 Most convenient way of passing them to container is by having a
114 local copy of the credentials file in Jumphost and then using the
115 **-v** option. In the example we have local file by the name of
116 "overcloudrc" and we are using that as an argument::
118 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds
120 The credentials file needs to exist in the Docker container
121 under the path: '/home/opnfv/functest/conf/openstack.creds'.
123 WARNING: If you are using the Joid installer, you must pass the
124 credentials using the **-v** option. See the section
125 `Accessing the Openstack credentials`_ above.
127 #. Passing deployment scenario
128 When running Functest against any of the supported OPNFV scenarios,
129 it is recommended to include also the environment variable
130 **DEPLOY_SCENARIO**. The **DEPLOY_SCENARIO** environment variable
131 is passed with the format::
133 -e "DEPLOY_SCENARIO=os-<controller>-<nfv_feature>-<ha_mode>"
135 os = OpenStack (No other VIM choices currently available)
136 controller is one of ( nosdn | odl_l2 | odl_l3 | onos )
137 nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
138 If several features are pertinent then use the underscore
139 character '_' to separate each feature (e.g. ovs_kvm)
140 'nofeature' indicates no NFV feature is deployed
141 ha_mode is one of ( ha | noha )
143 **NOTE:** Not all possible combinations of "DEPLOY_SCENARIO" are
144 supported. The name passed in to the Functest Docker container
145 must match the scenario used when the actual OPNFV platform was
148 Putting all above together, when using installer 'fuel' and an invented
149 INSTALLER_IP of '10.20.0.2', the recommended command to create the
150 Functest Docker container is as follows::
152 docker run --name "FunctestContainer" -it \
153 -e "INSTALLER_IP=10.20.0.2" \
154 -e "INSTALLER_TYPE=fuel" \
155 -e "DEPLOY_SCENARIO=os-odl_l2-ovs_kvm-ha" \
156 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
157 opnfv/functest /bin/bash
159 After the *run* command, a new prompt appears which means that we are inside
160 the container and ready to move to the next step.
162 For tips on how to set up container with installer Apex, see chapter
163 `Apex Installer Tips`_.
165 Finally, three additional environment variables can also be passed in
166 to the Functest Docker Container, using the -e
167 "<EnvironmentVariable>=<Value>" mechanism. The first two of these are
168 only relevant to Jenkins CI invoked testing and **should not be used**
169 when performing manual test scenarios::
171 -e "NODE_NAME=<Test POD Name>" \
172 -e "BUILD_TAG=<Jenkins Build Tag>" \
173 -e "CI_DEBUG=<DebugTraceValue>"
175 <Test POD Name> = Symbolic name of the POD where the tests are run.
176 Visible in test results files, which are stored
177 to the database. This option is only used when
178 tests are activated under Jenkins CI control.
179 It indicates the POD/hardware where the test has
180 been run. If not specified, then the POD name is
181 defined as "Unknown" by default.
182 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
183 <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
184 Visible in test results files, which are stored
185 to the database. This option is only set when
186 tests are activated under Jenkins CI control.
187 It enables the correlation of test results,
189 are independently pushed to the results datbase
190 from different Jenkins jobs.
191 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
192 <DebugTraceValue> = "true" or "false"
193 Default = "false", if not specified
194 If "true" is specified, then additional debug trace
195 text can be sent to the test results file / log files
196 and also to the standard console output.
200 Some specific tips are useful for the Apex Installer case. If not using
201 Apex Installer; ignore this section.
203 In case of Triple-O based installer (like Apex) the docker container
204 needs to connect to the installer VM, so it is then required that some
205 known SSH keys are present in docker container. Since the Jumphost root
206 SSH keys are already known, easiest way is to use those using the
207 'Bind mount' method. See below for sample parameter::
209 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa
211 NOTE: You need the "sudo" when creating the container to access root
212 users ssh credentials even the docker command itself might not
215 HINT! In case of Triple-O installers you can find value for the
216 INSTALLER_IP parameter by executing command and note the returned IP
219 inst=$(sudo virsh list | grep -iEo "undercloud|instack")
220 sudo virsh domifaddr ${inst}
222 NOTE: In releases prior to Colorado, the name 'instack' was
223 used. Currently the name 'undercloud' is used.
225 You can copy the credentials file from the "stack" users home directory
226 in installer VM to Jumphost. Please check the correct IP from the
227 command above. In the example below we are using invented IP address
230 scp stack@192.168.122.89:overcloudrc .
232 Here is an example of the full docker command invocation for an Apex
233 installed system, using latest Functest docker container, for
234 illustration purposes::
236 sudo docker run -it --name "ApexFuncTestODL" \
237 -e "INSTALLER_IP=192.168.122.89" \
238 -e "INSTALLER_TYPE=apex" \
239 -e "DEPLOY_SCENARIO=os-odl_l2-nofeature-ha" \
240 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa \
241 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
242 opnfv/functest /bin/bash
244 Compass installer local development env usage Tips
245 --------------------------------------------------
246 In the compass-functest local test case check and development environment, in order
247 to get openstack service inside the functest container, some parameters should be
248 configured during container creation, which are hard to guess for freshman. This
249 section will provide the guideline, the parameters values are defaults here, which should
250 be adjusted according to the settings, the complete steps are given here so as
251 not to appear too abruptly.
253 1, Pull Functest docker image from public dockerhub::
255 docker pull opnfv/functest:<Tag>
257 <Tag> here can be "brahmaputra.1.0", "colorado.1.0", etc. Tag omitted means the
258 latest docker image::
260 docker pull opnfv/functest
262 2, Functest Docker container creation
264 To make a file used for the environment, such as 'functest-docker-env'::
266 OS_AUTH_URL=http://172.16.1.222:35357/v2.0
270 OS_VOLUME_API_VERSION=2
271 OS_PROJECT_NAME=admin
272 INSTALLER_TYPE=compass
273 INSTALLER_IP=192.168.200.2
274 EXTERNAL_NETWORK=ext-net
276 Note: please adjust the content according to the environment, such as 'TENANT_ID'
277 maybe used for some special cases.
279 Then to create the Functest docker::
281 docker run --privileged=true --rm -t \
282 --env-file functest-docker-env \
283 --name <Functest_Container_Name> \
284 opnfv/functest:<Tag> /bin/bash
286 Note: it is recommended to be run on jumpserver.
288 3, To attach Functest container
290 Before trying to attach the Functest container, the status can be checked by::
294 to attach the 'Up' status Functest container and start bash mode::
296 docker exec -it <Functest_Container_Name> bash
298 4, Functest environemnt preparation and check
300 To see the Section below `Preparing the Functest environment`_.
302 Functest docker container directory structure
303 ---------------------------------------------
304 Inside the Functest docker container, the following directory structure
305 should now be in place::
324 Underneath the '/home/opnfv/' directory, the Functest docker container
325 includes two main directories:
327 * The **functest** directory stores configuration files (e.g. the
328 OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
329 the **data** directory stores a 'cirros' test image used in some
330 functional tests and the **results** directory stores some temporary
332 * The **repos** directory holds various repositories. The directory
333 '/home/opnfv/repos/functest' is used to prepare the needed Functest
334 environment and to run the tests. The other repository directories
335 are used for the installation of the needed tooling (e.g. rally) or
336 for the retrieval of feature projects scenarios (e.g. promise)
338 The structure under the **functest** repository can be described as
347 | |-- config_functest.yaml
352 | |-- tier_builder.py
353 | `-- tier_handler.py
358 | |-- functest-complete.sh
363 | `--traffic-profile-guidelines.rst
366 | |-- config_install_env.sh
367 | `-- requirements.pip
385 |-- functest_logger.py
386 |-- functest_utils.py
387 |-- openstack_clean.py
388 |-- openstack_snapshot.py
389 `-- openstack_utils.py
391 (Note: All *.pyc files removed from above list for brevity...)
393 We may distinguish 7 different directories:
395 * **ci**: This directory contains test structure definition files
396 (e.g <filename>.yaml) and bash shell/python scripts used to
397 configure and execute Functional tests. The test execution script
398 can be executed under the control of Jenkins CI jobs.
399 * **cli**: This directory holds the python based Functest CLI utility
400 source code, which is based on the Python 'click' framework.
401 * **commons**: This directory is dedicated for storage of traffic
402 profile or any other test inputs that could be reused by any test
404 * **docker**: This directory includes the needed files and tools to
405 build the Funtest Docker container image.
406 * **docs**: This directory includes documentation: Release Notes,
407 User Guide, Configuration Guide and Developer Guide. Test results
408 are also located in a sub--directory called 'results'.
409 * **testcases**: This directory includes the scripts required by
410 Functest internal test cases and other feature projects test cases.
411 * **utils**: this directory holds Python source code for some general
412 purpose helper utilities, which testers can also re-use in their
413 own test code. See for an example the Openstack helper utility:
414 'openstack_utils.py'.
416 Useful Docker commands
417 ----------------------
418 When typing **exit** in the container prompt, this will cause exiting
419 the container and probably stopping it. When stopping a running Docker
420 container all the changes will be lost, there is a keyboard shortcut
421 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
422 reconnect to the running container **DO NOT** use the *run* command
423 again (since it will create a new container), use the *exec* or *attach*
426 docker ps # <check the container ID from the output>
427 docker exec -ti <CONTAINER_ID> /bin/bash
429 There are other useful Docker commands that might be needed to manage possible
430 issues with the containers.
432 List the running containers::
436 List all the containers including the stopped ones::
440 Start a stopped container named "FunTest"::
444 Attach to a running container named "StrikeTwo"::
446 docker attach StrikeTwo
448 It is useful sometimes to remove a container if there are some problems::
450 docker rm <CONTAINER_ID>
452 Use the *-f* option if the container is still running, it will force to
455 docker -f rm <CONTAINER_ID>
457 Check the Docker documentation dockerdocs_ for more information.
459 Preparing the Functest environment
460 ----------------------------------
461 Once the Functest docker container is up and running, the required
462 Functest environment needs to be prepared. A custom built **functest**
463 CLI utility is available to perform the needed environment preparation
464 action. Once the environment is prepared, the **functest** CLI utility
465 can be used to run different functional tests. The usage of the
466 **functest** CLI utility to run tests is described further in the
467 Functest User Guide `OPNFV_FuncTestUserGuide`_
469 Prior to commencing the Functest environment preparation, we can check
470 the initial status of the environment. Issue the **functest env status**
471 command at the prompt::
474 Functest environment is not installed.
476 Note: When the Functest environment is prepared, the command will
477 return the status: "Functest environment ready to run tests."
479 To prepare the Functest docker container for test case execution, issue
480 the **functest env prepare** command at the prompt::
484 This script will make sure that the requirements to run the tests are
485 met and will install the needed libraries and tools by all Functest
486 test cases. It should be run only once every time the Functest docker
487 container is started from scratch. If you try to run this command, on
488 an already prepared enviroment, you will be prompted whether you really
489 want to continue or not::
492 It seems that the environment has been already prepared.
493 Do you want to do it again? [y|n]
495 (Type 'n' to abort the request, or 'y' to repeat the
496 environment preparation)
499 To list some basic information about an already prepared Functest
500 docker container environment, issue the **functest env show** at the
504 +======================================================+
505 | Functest Environment info |
506 +======================================================+
507 | INSTALLER: apex, 192.168.122.89 |
508 | SCENARIO: os-odl_l2-nofeature-ha |
510 | GIT BRANCH: master |
511 | GIT HASH: 5bf1647dec6860464eeb082b2875798f0759aa91 |
512 | DEBUG FLAG: false |
513 +------------------------------------------------------+
515 +------------------------------------------------------+
519 INSTALLER: Displays the INSTALLER_TYPE value
521 and the INSTALLER_IP value
522 - here = "192.168.122.89"
523 SCENARIO: Displays the DEPLOY_SCENARIO value
524 - here = "os-odl_l2-nofeature-ha"
525 POD: Displays the value passed in NODE_NAME
527 GIT BRANCH: Displays the git branch of the OPNFV Functest
528 project repository included in the Functest
531 (In first official colorado release
532 would be "colorado.1.0")
533 GIT HASH: Displays the git hash of the OPNFV Functest
534 project repository included in the Functest
536 - here = "5bf1647dec6860464eeb082b2875798f0759aa91"
537 DEBUG FLAG: Displays the CI_DEBUG value
540 NOTE: In Jenkins CI runs, an additional item "BUILD TAG"
541 would also be listed. The valaue is set by Jenkins CI.
543 Finally, the **functest** CLI has a **--help** options:
547 functest --help Usage: functest [OPTIONS] COMMAND [ARGS]...
550 --version Show the version and exit.
551 -h, --help Show this message and exit.
560 Usage: functest env [OPTIONS] COMMAND [ARGS]...
563 -h, --help Show this message and exit.
566 prepare Prepares the Functest environment.
567 show Shows information about the current...
568 status Checks if the Functest environment is ready...
570 Checking Openstack and credentials
571 ----------------------------------
572 It is recommended and fairly straightforward to check that Openstack
573 and credentials are working as expected.
575 Once the credentials are there inside the container, they should be
576 sourced before running any Openstack commands::
578 source /home/opnfv/functest/conf/openstack.creds
580 After this, try to run any OpenStack command to see if you get any
581 output, for instance::
585 This will return a list of the actual users in the OpenStack
586 deployment. In any other case, check that the credentials are sourced::
590 This command must show a set of environment variables starting with
593 OS_REGION_NAME=RegionOne
594 OS_DEFAULT_DOMAIN=default
595 OS_PROJECT_NAME=admin
597 OS_AUTH_STRATEGY=keystone
598 OS_AUTH_URL=http://172.30.10.3:5000/v2.0
601 OS_ENDPOINT_TYPE=internalURL
604 If the OpenStack command still does not show anything or complains
605 about connectivity issues, it could be due to an incorrect url given to
606 the OS_AUTH_URL environment variable. Check the deployment settings.
610 If you need to connect to a server that is TLS-enabled (the auth URL
611 begins with "https") and it uses a certificate from a private CA or a
612 self-signed certificate, then you will need to specify the path to an
613 appropriate CA certificate to use, to validate the server certificate
614 with the environment variable OS_CACERT::
617 /etc/ssl/certs/ca.crt
619 However, this certificate does not exist in the container by default.
620 It has to be copied manually from the OpenStack deployment. This can be
623 #. Create manually that file and copy the contents from the OpenStack
625 #. (Recommended) Add the file using a Docker volume when starting the
628 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
630 You might need to export OS_CACERT environment variable inside the
633 export OS_CACERT=/etc/ssl/certs/ca.crt
635 Certificate verification can be turned off using OS_INSECURE=true. For
636 example, Fuel uses self-signed cacerts by default, so an pre step would
639 export OS_INSECURE=true
643 If your Jumphost node is operating behind a http proxy, then there are
644 2 places where some special actions may be needed to make operations
647 #. Initial installation of docker engine First, try following the
648 official Docker documentation for Proxy settings. Some issues were
649 experienced on CentOS 7 based Jumphost. Some tips are documented
650 in section: `Docker Installation on CentOS behind http proxy`_
653 #. Execution of the Functest environment preparation inside the
654 created docker container Functest needs internet access to
655 download some resources for some test cases. This might not
656 work properly if the Jumphost is connecting to internet
657 through a http Proxy.
659 If that is the case, make sure the resolv.conf and the needed
660 http_proxy and https_proxy environment variables, as well as the
661 'no_proxy' environment variable are set correctly::
663 # Make double sure that the 'no_proxy=...' line in the
664 # 'openstack.creds' file is commented out first. Otherwise, the
665 # values set into the 'no_proxy' environment variable below will
666 # be ovewrwritten, each time the command
667 # 'source ~/functest/conf/openstack.creds' is issued.
670 sed -i 's/export no_proxy/#export no_proxy/' openstack.creds
671 source ./openstack.creds
673 # Next calculate some IP addresses for which http_proxy
674 # usage should be excluded:
676 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
678 adminURL_IP=$(openstack catalog show identity | \
679 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
681 export http_proxy="<your http proxy settings>"
682 export https_proxy="<your https proxy settings>"
683 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
685 # Ensure that "git" uses the http_proxy
686 # This may be needed if your firewall forbids SSL based git fetch
687 git config --global http.sslVerify True
688 git config --global http.proxy <Your http proxy settings>
690 Validation check: Before running **'functest env prepare'** CLI command,
691 make sure you can reach http and https sites from inside the Functest
694 For example, try to use the **nc** command from inside the functest
698 Connection to google.com 80 port [tcp/http] succeeded!
701 Connection to google.com 443 port [tcp/https] succeeded!
703 Note: In a Jumphost node based on the CentOS family OS, the **nc**
704 commands might not work. You can use the **curl** command instead.
706 curl http://www.google.com:80
707 <HTML><HEAD><meta http-equiv="content-type"
712 curl https://www.google.com:443
713 <HTML><HEAD><meta http-equiv="content-type"
718 (Ignore the content. If command returns a valid HTML page, it proves
721 Docker Installation on CentOS behind http proxy
722 -----------------------------------------------
723 This section is applicable for CentOS family OS on Jumphost which
724 itself is behind a proxy server. In that case, the instructions below
725 should be followed **before** installing the docker engine::
727 1) # Make a directory '/etc/systemd/system/docker.service.d'
728 # if it does not exist
729 sudo mkdir /etc/systemd/system/docker.service.d
731 2) # Create a file called 'env.conf' in that directory with
732 # the following contents:
734 EnvironmentFile=-/etc/sysconfig/docker
736 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
737 # with the following contents:
738 HTTP_PROXY="<Your http proxy settings>"
739 HTTPS_PROXY="<Your https proxy settings>"
740 http_proxy="${HTTP_PROXY}"
741 https_proxy="${HTTPS_PROXY}"
743 4) # Reload the daemon
744 systemctl daemon-reload
746 5) # Sanity check - check the following docker settings:
747 systemctl show docker | grep -i env
751 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
752 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
754 Now follow the instructions in [`InstallDockerCentOS`_] to download
755 and install the **docker-engine**. The instructions conclude with a
756 "test pull" of a sample "Hello World" docker container. This should now
757 work with the above pre-requisite actions.
759 .. _dockerdocs: https://docs.docker.com/
760 .. _dockerhub: https://hub.docker.com/r/opnfv/functest/
761 .. _Proxy: https://docs.docker.com/engine/admin/systemd/#http-proxy
762 .. _FunctestDockerTags: https://hub.docker.com/r/opnfv/functest/tags/
763 .. _InstallDockerCentOS: https://docs.docker.com/engine/installation/linux/centos/
764 .. _OPNFV_FuncTestUserGuide: http://artifacts.opnfv.org/functest/docs/userguide/index.html