1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. SPDX-License-Identifier: CC-BY-4.0
5 Pulling the Docker image
6 ------------------------
7 Pull the Functest Docker image ('opnfv/functest') from the public
8 dockerhub registry under the OPNFV account: [dockerhub_], with the
9 following docker command::
11 docker pull opnfv/functest:<TagIdentifier>
13 where <TagIdentifier> identifies a release of the Functest docker
14 container image in the public dockerhub registry. There are many tags
15 created automatically by the CI mechanisms, and you must ensure you
16 pull an image with the **correct tag** to match the OPNFV software
17 release installed in your environment. All available tagged images can
18 be seen from location [FunctestDockerTags_]. For example, when running
19 on the first official release of the OPNFV Danube system platform,
20 tag "danube.1.0" is needed. Pulling other tags might cause some
21 problems while running the tests.
22 Docker images pulled without a tag specifier bear the implicitly
23 assigned label "latest". If you need to specifically pull the latest
24 Functest docker image, then omit the tag argument::
26 docker pull opnfv/functest
28 After pulling the Docker image, check that it is available with the
29 following docker command::
31 [functester@jumphost ~]$ docker images
32 REPOSITORY TAG IMAGE ID CREATED SIZE
33 opnfv/functest latest 8cd6683c32ae 2 weeks ago 1.321 GB
34 opnfv/functest danube.1.0 13fa54a1b238 4 weeks ago 1.29 GB
35 opnfv/functest colorado.1.0 94b78faa94f7 9 weeks ago 968 MB
37 The Functest docker container environment can -in principle- be also
38 used with non-OPNFV official installers (e.g. 'devstack'), with the
39 **disclaimer** that support for such environments is outside of the
40 scope and responsibility of the OPNFV project.
42 Accessing the Openstack credentials
43 -----------------------------------
44 OpenStack credentials are mandatory and must be provided to Functest.
45 When running the command "functest env prepare", the framework will
46 automatically look for the Openstack credentials file
47 "/home/opnfv/functest/conf/openstack.creds" and will exit with
48 error if it is not present or it is empty.
50 There are 2 ways to provide that file:
52 * by using a Docker volume with -v option when creating the Docker container.
53 This is referred to in docker documentation as "Bind Mounting".
54 See the usage of this parameter in the following chapter.
55 * or creating manually the file '/home/opnfv/functest/conf/openstack.creds'
56 inside the running container and pasting the credentials in it. Consult
57 your installer guide for further details. This is however not
58 instructed in this document.
60 There is a default environment variable in the Functest container **$creds**
61 that points to the credentials absolute path to help the user with this task.
63 In proxified environment you may need to change the credentials file.
64 There are some tips in chapter: `Proxy support`_
66 Functest Docker parameters
67 --------------------------
68 This chapter explains how to run a container for executing functest
69 test suites. Numbered list below explains some details of the
70 recommended parameters for invoking docker container
72 #. It is a good practice to assign a precise container name through
73 the **--name** option.
75 #. Assign parameter for installer type::
77 -e "INSTALLER_TYPE=<type>"
78 # Use one of following apex, compass, fuel or joid
80 #. Functest needs to know the IP of some installers::
82 -e "INSTALLER_IP=<Specific IP Address>"
84 These two env variables are useful extract some information
85 from the deployment. However, for some test cases like
86 SFC or Barometer they are mandatory since the tests
87 need to access the installer node and the deployment.
89 #. Credentials for accessing the Openstack.
90 Most convenient way of passing them to container is by having a
91 local copy of the credentials file in Jumphost and then using the
92 **-v** option. In the example we have local file by the name of
93 "overcloudrc" and we are using that as an argument::
95 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds
97 The credentials file needs to exist in the Docker container
98 under the path: '/home/opnfv/functest/conf/openstack.creds'.
100 **WARNING:** If you are using the Joid installer, you must pass the
101 credentials using the **-v** option:
102 -v /var/lib/jenkins/admin-openrc:/home/opnfv/functest/conf/openstack.creds.
103 See the section `Accessing the Openstack credentials`_ above.
105 #. Passing deployment scenario
106 When running Functest against any of the supported OPNFV scenarios,
107 it is recommended to include also the environment variable
108 **DEPLOY_SCENARIO**. The **DEPLOY_SCENARIO** environment variable
109 is passed with the format::
111 -e "DEPLOY_SCENARIO=os-<controller>-<nfv_feature>-<ha_mode>"
113 os = OpenStack (No other VIM choices currently available)
114 controller is one of ( nosdn | odl_l2 | odl_l3 | onos | ocl)
115 nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
116 If several features are pertinent then use the underscore
117 character '_' to separate each feature (e.g. ovs_kvm)
118 'nofeature' indicates no NFV feature is deployed
119 ha_mode (high availability) is one of ( ha | noha )
121 **NOTE:** Not all possible combinations of "DEPLOY_SCENARIO" are
122 supported. The name passed in to the Functest Docker container
123 must match the scenario used when the actual OPNFV platform was
124 deployed. See release note to see the list of supported scenarios.
126 **NOTE:** The scenario name is mainly used to automatically detect
127 if a test suite is runnable or not (e.g. it will prevent ONOS test suite
128 to be run on ODL scenarios). If not set, Functest will try to run the
129 default test cases that might not include SDN controller or a specific
132 **NOTE:** A HA scenario means that 3 OpenStack controller nodes are
133 deployed. It does not necessarily mean that the whole system is HA. See
134 installer release notes for details.
137 Putting all above together, when using installer 'fuel' and an invented
138 INSTALLER_IP of '10.20.0.2', the recommended command to create the
139 Functest Docker container is as follows::
141 docker run --name "FunctestContainer" -it \
142 -e "INSTALLER_IP=10.20.0.2" \
143 -e "INSTALLER_TYPE=fuel" \
144 -e "DEPLOY_SCENARIO=os-odl_l2-ovs_kvm-ha" \
145 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
146 opnfv/functest /bin/bash
148 After the *run* command, a new prompt appears which means that we are inside
149 the container and ready to move to the next step.
151 For tips on how to set up container with installer Apex, see chapter
152 `Apex Installer Tips`_.
154 Finally, three additional environment variables can also be passed in
155 to the Functest Docker Container, using the -e
156 "<EnvironmentVariable>=<Value>" mechanism. The first two of these are
157 only relevant to Jenkins CI invoked testing and **should not be used**
158 when performing manual test scenarios::
160 -e "NODE_NAME=<Test POD Name>" \
161 -e "BUILD_TAG=<Jenkins Build Tag>" \
162 -e "CI_DEBUG=<DebugTraceValue>"
164 <Test POD Name> = Symbolic name of the POD where the tests are run.
165 Visible in test results files, which are stored
166 to the database. This option is only used when
167 tests are activated under Jenkins CI control.
168 It indicates the POD/hardware where the test has
169 been run. If not specified, then the POD name is
170 defined as "Unknown" by default.
171 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
172 <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
173 Visible in test results files, which are stored
174 to the database. This option is only set when
175 tests are activated under Jenkins CI control.
176 It enables the correlation of test results,
178 are independently pushed to the results database
179 from different Jenkins jobs.
180 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
181 <DebugTraceValue> = "true" or "false"
182 Default = "false", if not specified
183 If "true" is specified, then additional debug trace
184 text can be sent to the test results file / log files
185 and also to the standard console output.
189 Some specific tips are useful for the Apex Installer case. If not using
190 Apex Installer; ignore this section.
192 In case of Triple-O based installer (like Apex) the docker container
193 needs to connect to the installer VM, so it is then required that some
194 known SSH keys are present in docker container. Since the Jumphost root
195 SSH keys are already known, easiest way is to use those using the
196 'Bind mount' method. See below for sample parameter::
198 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa
200 NOTE: You need the "sudo" when creating the container to access root
201 users ssh credentials even the docker command itself might not
204 HINT! In case of Triple-O installers you can find value for the
205 INSTALLER_IP parameter by executing command and note the returned IP
208 inst=$(sudo virsh list | grep -iEo "undercloud|instack")
209 sudo virsh domifaddr ${inst}
211 NOTE: In releases prior to Colorado, the name 'instack' was
212 used. Currently the name 'undercloud' is used.
214 You can copy the credentials file from the "stack" users home directory
215 in installer VM to Jumphost. Please check the correct IP from the
216 command above. In the example below we are using invented IP address
219 scp stack@192.168.122.89:overcloudrc .
221 Here is an example of the full docker command invocation for an Apex
222 installed system, using latest Functest docker container, for
223 illustration purposes::
225 sudo docker run -it --name "ApexFuncTestODL" \
226 -e "INSTALLER_IP=192.168.122.89" \
227 -e "INSTALLER_TYPE=apex" \
228 -e "DEPLOY_SCENARIO=os-odl_l2-nofeature-ha" \
229 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa \
230 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
231 opnfv/functest /bin/bash
233 Compass installer local development env usage Tips
234 --------------------------------------------------
235 In the compass-functest local test case check and development environment,
236 in order to get openstack service inside the functest container, some
237 parameters should be configured during container creation, which are
238 hard to guess for freshman. This section will provide the guideline, the
239 parameters values are defaults here, which should be adjusted according
240 to the settings, the complete steps are given here so as not to appear
243 1, Pull Functest docker image from public dockerhub::
245 docker pull opnfv/functest:<Tag>
247 <Tag> here can be "brahmaputra.1.0", "colorado.1.0", etc.
248 Tag omitted means the latest docker image::
250 docker pull opnfv/functest
252 2, Functest Docker container creation
254 To make a file used for the environment, such as 'functest-docker-env'::
256 OS_AUTH_URL=http://172.16.1.222:35357/v2.0
260 OS_VOLUME_API_VERSION=2
261 OS_PROJECT_NAME=admin
262 INSTALLER_TYPE=compass
263 INSTALLER_IP=192.168.200.2
264 EXTERNAL_NETWORK=ext-net
266 Note: please adjust the content according to the environment, such as
267 'TENANT_ID' maybe used for some special cases.
269 Then to create the Functest docker::
271 docker run --privileged=true --rm -t \
272 --env-file functest-docker-env \
273 --name <Functest_Container_Name> \
274 opnfv/functest:<Tag> /bin/bash
276 3, To attach Functest container
278 Before trying to attach the Functest container, the status can be checked by::
282 to attach the 'Up' status Functest container and start bash mode::
284 docker exec -it <Functest_Container_Name> bash
286 4, Functest environment preparation and check
288 To see the Section below `Preparing the Functest environment`_.
290 Functest docker container directory structure
291 ---------------------------------------------
292 Inside the Functest docker container, the following directory structure
293 should now be in place::
322 Underneath the '/home/opnfv/' directory, the Functest docker container
323 includes two main directories:
325 * The **functest** directory stores configuration files (e.g. the
326 OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
327 the **data** directory stores a 'cirros' test image used in some
328 functional tests and the **results** directory stores some temporary
330 * The **repos** directory holds various repositories. The directory
331 '/home/opnfv/repos/functest' is used to prepare the needed Functest
332 environment and to run the tests. The other repository directories
333 are used for the installation of the needed tooling (e.g. rally) or
334 for the retrieval of feature projects scenarios (e.g. promise)
336 The structure under the **functest** repository can be described as
342 |-- run_unit_tests.sh
344 |-- test-requirements.txt
348 | `--traffic-profile-guidelines.rst
351 | |-- config_install_env.sh
352 | `-- docker_remote_api
367 | |-- config_functest.yaml
368 | |-- config_patch.yaml
369 | |-- generate_report.py
373 | |-- tier_builder.py
374 | `-- tier_handler.py
379 | |-- functest-complete.sh
384 | |-- pytest_suite_runner.py
403 |-- functest_logger.py
404 |-- functest_utils.py
406 |-- openstack_clean.py
407 |-- openstack_snapshot.py
408 |-- openstack_tacker.py
409 `-- openstack_utils.py
412 (Note: All *.pyc files removed from above list for brevity...)
414 We may distinguish several directories, the first level has 4 directories:
416 * **commons**: This directory is dedicated for storage of traffic
417 profile or any other test inputs that could be reused by any test
419 * **docker**: This directory includes the needed files and tools to
420 build the Functest Docker image.
421 * **docs**: This directory includes documentation: Release Notes,
422 User Guide, Configuration Guide and Developer Guide.
423 * **functest**: This directory contains all the code needed to run
424 functest internal cases and OPNFV onboarded feature or VNF test cases.
426 Functest directory has 6 directories:
427 * **ci**: This directory contains test structure definition files
428 (e.g <filename>.yaml) and bash shell/python scripts used to
429 configure and execute Functional tests. The test execution script
430 can be executed under the control of Jenkins CI jobs.
431 * **cli**: This directory holds the python based Functest CLI utility
432 source code, which is based on the Python 'click' framework.
433 * **core**: This directory holds the python based Functest core
434 source code. Three abstraction classes have been created to ease
435 the integration of internal, feature or vnf cases.
436 * **opnfv_tests**: This directory includes the scripts required by
437 Functest internal test cases and other feature projects test cases.
438 * **tests**: This directory includes the functest unit tests
439 * **utils**: this directory holds Python source code for some general
440 purpose helper utilities, which testers can also re-use in their
441 own test code. See for an example the Openstack helper utility:
442 'openstack_utils.py'.
444 Useful Docker commands
445 ----------------------
446 When typing **exit** in the container prompt, this will cause exiting
447 the container and probably stopping it. When stopping a running Docker
448 container all the changes will be lost, there is a keyboard shortcut
449 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
450 reconnect to the running container **DO NOT** use the *run* command
451 again (since it will create a new container), use the *exec* or *attach*
454 docker ps # <check the container ID from the output>
455 docker exec -ti <CONTAINER_ID> /bin/bash
457 There are other useful Docker commands that might be needed to manage possible
458 issues with the containers.
460 List the running containers::
464 List all the containers including the stopped ones::
468 Start a stopped container named "FunTest"::
472 Attach to a running container named "StrikeTwo"::
474 docker attach StrikeTwo
476 It is useful sometimes to remove a container if there are some problems::
478 docker rm <CONTAINER_ID>
480 Use the *-f* option if the container is still running, it will force to
483 docker rm -f <CONTAINER_ID>
485 Check the Docker documentation dockerdocs_ for more information.
487 Preparing the Functest environment
488 ----------------------------------
489 Once the Functest docker container is up and running, the required
490 Functest environment needs to be prepared. A custom built **functest**
491 CLI utility is available to perform the needed environment preparation
492 action. Once the environment is prepared, the **functest** CLI utility
493 can be used to run different functional tests. The usage of the
494 **functest** CLI utility to run tests is described further in the
495 Functest User Guide `OPNFV_FuncTestUserGuide`_
497 Prior to commencing the Functest environment preparation, we can check
498 the initial status of the environment. Issue the **functest env status**
499 command at the prompt::
502 Functest environment is not installed.
504 Note: When the Functest environment is prepared, the command will
505 return the status: "Functest environment ready to run tests."
507 To prepare the Functest docker container for test case execution, issue
508 the **functest env prepare** command at the prompt::
512 This script will make sure that the requirements to run the tests are
513 met and will install the needed libraries and tools by all Functest
514 test cases. It should be run only once every time the Functest docker
515 container is started from scratch. If you try to run this command, on
516 an already prepared environment, you will be prompted whether you really
517 want to continue or not::
520 It seems that the environment has been already prepared.
521 Do you want to do it again? [y|n]
523 (Type 'n' to abort the request, or 'y' to repeat the
524 environment preparation)
527 To list some basic information about an already prepared Functest
528 docker container environment, issue the **functest env show** at the
532 +======================================================+
533 | Functest Environment info |
534 +======================================================+
535 | INSTALLER: apex, 192.168.122.89 |
536 | SCENARIO: os-odl_l2-nofeature-ha |
538 | GIT BRANCH: master |
539 | GIT HASH: 5bf1647dec6860464eeb082b2875798f0759aa91 |
540 | DEBUG FLAG: false |
541 +------------------------------------------------------+
543 +------------------------------------------------------+
547 INSTALLER: Displays the INSTALLER_TYPE value
549 and the INSTALLER_IP value
550 - here = "192.168.122.89"
551 SCENARIO: Displays the DEPLOY_SCENARIO value
552 - here = "os-odl_l2-nofeature-ha"
553 POD: Displays the value passed in NODE_NAME
555 GIT BRANCH: Displays the git branch of the OPNFV Functest
556 project repository included in the Functest
559 (In first official colorado release
560 would be "colorado.1.0")
561 GIT HASH: Displays the git hash of the OPNFV Functest
562 project repository included in the Functest
564 - here = "5bf1647dec6860464eeb082b2875798f0759aa91"
565 DEBUG FLAG: Displays the CI_DEBUG value
568 NOTE: In Jenkins CI runs, an additional item "BUILD TAG"
569 would also be listed. The value is set by Jenkins CI.
571 Finally, the **functest** CLI has a **--help** options:
575 functest --help Usage: functest [OPTIONS] COMMAND [ARGS]...
578 --version Show the version and exit.
579 -h, --help Show this message and exit.
588 Usage: functest env [OPTIONS] COMMAND [ARGS]...
591 -h, --help Show this message and exit.
594 prepare Prepares the Functest environment.
595 show Shows information about the current...
596 status Checks if the Functest environment is ready...
598 Checking Openstack and credentials
599 ----------------------------------
600 It is recommended and fairly straightforward to check that Openstack
601 and credentials are working as expected.
603 Once the credentials are there inside the container, they should be
604 sourced before running any Openstack commands::
606 source /home/opnfv/functest/conf/openstack.creds
608 After this, try to run any OpenStack command to see if you get any
609 output, for instance::
613 This will return a list of the actual users in the OpenStack
614 deployment. In any other case, check that the credentials are sourced::
618 This command must show a set of environment variables starting with
621 OS_REGION_NAME=RegionOne
622 OS_DEFAULT_DOMAIN=default
623 OS_PROJECT_NAME=admin
625 OS_AUTH_STRATEGY=keystone
626 OS_AUTH_URL=http://172.30.10.3:5000/v2.0
629 OS_ENDPOINT_TYPE=internalURL
632 If the OpenStack command still does not show anything or complains
633 about connectivity issues, it could be due to an incorrect url given to
634 the OS_AUTH_URL environment variable. Check the deployment settings.
638 If you need to connect to a server that is TLS-enabled (the auth URL
639 begins with "https") and it uses a certificate from a private CA or a
640 self-signed certificate, then you will need to specify the path to an
641 appropriate CA certificate to use, to validate the server certificate
642 with the environment variable OS_CACERT::
645 /etc/ssl/certs/ca.crt
647 However, this certificate does not exist in the container by default.
648 It has to be copied manually from the OpenStack deployment. This can be
651 #. Create manually that file and copy the contents from the OpenStack
653 #. (Recommended) Add the file using a Docker volume when starting the
656 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
658 You might need to export OS_CACERT environment variable inside the
661 export OS_CACERT=/etc/ssl/certs/ca.crt
663 Certificate verification can be turned off using OS_INSECURE=true. For
664 example, Fuel uses self-signed cacerts by default, so an pre step would
667 export OS_INSECURE=true
671 If your Jumphost node is operating behind a http proxy, then there are
672 2 places where some special actions may be needed to make operations
675 #. Initial installation of docker engine First, try following the
676 official Docker documentation for Proxy settings. Some issues were
677 experienced on CentOS 7 based Jumphost. Some tips are documented
678 in section: `Docker Installation on CentOS behind http proxy`_
681 #. Execution of the Functest environment preparation inside the
682 created docker container Functest needs internet access to
683 download some resources for some test cases. This might not
684 work properly if the Jumphost is connecting to internet
685 through a http Proxy.
687 If that is the case, make sure the resolv.conf and the needed
688 http_proxy and https_proxy environment variables, as well as the
689 'no_proxy' environment variable are set correctly::
691 # Make double sure that the 'no_proxy=...' line in the
692 # 'openstack.creds' file is commented out first. Otherwise, the
693 # values set into the 'no_proxy' environment variable below will
694 # be ovewrwritten, each time the command
695 # 'source ~/functest/conf/openstack.creds' is issued.
698 sed -i 's/export no_proxy/#export no_proxy/' openstack.creds
699 source ./openstack.creds
701 # Next calculate some IP addresses for which http_proxy
702 # usage should be excluded:
704 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
706 adminURL_IP=$(openstack catalog show identity | \
707 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
709 export http_proxy="<your http proxy settings>"
710 export https_proxy="<your https proxy settings>"
711 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
713 # Ensure that "git" uses the http_proxy
714 # This may be needed if your firewall forbids SSL based git fetch
715 git config --global http.sslVerify True
716 git config --global http.proxy <Your http proxy settings>
718 Validation check: Before running **'functest env prepare'** CLI command,
719 make sure you can reach http and https sites from inside the Functest
722 For example, try to use the **nc** command from inside the functest
726 Connection to opnfv.org 80 port [tcp/http] succeeded!
729 Connection to opnfv.org 443 port [tcp/https] succeeded!
731 Note: In a Jumphost node based on the CentOS family OS, the **nc**
732 commands might not work. You can use the **curl** command instead.
734 curl http://www.opnfv.org:80
735 <HTML><HEAD><meta http-equiv="content-type"
740 curl https://www.opnfv.org:443
741 <HTML><HEAD><meta http-equiv="content-type"
746 (Ignore the content. If command returns a valid HTML page, it proves
749 Docker Installation on CentOS behind http proxy
750 -----------------------------------------------
751 This section is applicable for CentOS family OS on Jumphost which
752 itself is behind a proxy server. In that case, the instructions below
753 should be followed **before** installing the docker engine::
755 1) # Make a directory '/etc/systemd/system/docker.service.d'
756 # if it does not exist
757 sudo mkdir /etc/systemd/system/docker.service.d
759 2) # Create a file called 'env.conf' in that directory with
760 # the following contents:
762 EnvironmentFile=-/etc/sysconfig/docker
764 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
765 # with the following contents:
766 HTTP_PROXY="<Your http proxy settings>"
767 HTTPS_PROXY="<Your https proxy settings>"
768 http_proxy="${HTTP_PROXY}"
769 https_proxy="${HTTPS_PROXY}"
771 4) # Reload the daemon
772 systemctl daemon-reload
774 5) # Sanity check - check the following docker settings:
775 systemctl show docker | grep -i env
779 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
780 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
782 Now follow the instructions in [`InstallDockerCentOS`_] to download
783 and install the **docker-engine**. The instructions conclude with a
784 "test pull" of a sample "Hello World" docker container. This should now
785 work with the above pre-requisite actions.
787 .. _dockerdocs: https://docs.docker.com/
788 .. _dockerhub: https://hub.docker.com/r/opnfv/functest/
789 .. _Proxy: https://docs.docker.com/engine/admin/systemd/#http-proxy
790 .. _FunctestDockerTags: https://hub.docker.com/r/opnfv/functest/tags/
791 .. _InstallDockerCentOS: https://docs.docker.com/engine/installation/linux/centos/
792 .. _OPNFV_FuncTestUserGuide: http://artifacts.opnfv.org/functest/docs/userguide/index.html