1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. SPDX-License-Identifier: CC-BY-4.0
5 Pulling the Docker image
6 ------------------------
7 Pull the Functest Docker image ('opnfv/functest') from the public
8 dockerhub registry under the OPNFV account: [dockerhub_], with the
9 following docker command::
11 docker pull opnfv/functest:<TagIdentifier>
13 where <TagIdentifier> identifies a release of the Functest docker
14 container image in the public Dockerhub registry. There are many tags
15 created automatically by the CI mechanisms, and you must ensure you
16 pull an image with the **correct tag** to match the OPNFV software
17 release installed in your environment. All available tagged images can
18 be seen from location [FunctestDockerTags_]. For example, when running
19 on the first official release of the OPNFV Danube system platform,
20 tag "danube.1.0" is needed. For the second and third releases, the tag
21 "danube.2.0" and "danube.3.0" can be used respectively.
22 Pulling other tags might cause some problems while running the tests.
23 Docker images pulled without a tag specifier bear the implicitly
24 assigned label "latest". If you need to specifically pull the latest
25 Functest docker image, then omit the tag argument::
27 docker pull opnfv/functest
29 After pulling the Docker image, check that it is available with the
30 following docker command::
32 [functester@jumphost ~]$ docker images
33 REPOSITORY TAG IMAGE ID CREATED SIZE
34 opnfv/functest latest 8cd6683c32ae 2 weeks ago 1.321 GB
35 opnfv/functest danube.2.0 d2c174a91911 7 minutes ago 1.471 GB
36 opnfv/functest danube.1.0 13fa54a1b238 4 weeks ago 1.29 GB
38 The Functest docker container environment can -in principle- be also
39 used with non-OPNFV official installers (e.g. 'devstack'), with the
40 **disclaimer** that support for such environments is outside of the
41 scope and responsibility of the OPNFV project.
43 Accessing the Openstack credentials
44 -----------------------------------
45 OpenStack credentials are mandatory and must be provided to Functest.
46 When running the command "functest env prepare", the framework will
47 automatically look for the Openstack credentials file
48 "/home/opnfv/functest/conf/openstack.creds" and will exit with
49 error if it is not present or it is empty.
51 There are 2 ways to provide that file:
53 * by using a Docker volume with -v option when creating the Docker container.
54 This is referred to in docker documentation as "Bind Mounting".
55 See the usage of this parameter in the following chapter.
56 * or creating manually the file '/home/opnfv/functest/conf/openstack.creds'
57 inside the running container and pasting the credentials in it. Consult
58 your installer guide for further details. This is however not
59 instructed in this document.
61 There is a default environment variable in the Functest container **$creds**
62 that points to the credentials absolute path to help the user with this task.
64 In proxified environment you may need to change the credentials file.
65 There are some tips in chapter: `Proxy support`_
67 Functest Docker parameters
68 --------------------------
69 This chapter explains how to run a container for executing functest
70 test suites. Numbered list below explains some details of the
71 recommended parameters for invoking docker container
73 #. It is a good practice to assign a precise container name through
74 the **--name** option.
76 #. Assign parameter for installer type::
78 -e "INSTALLER_TYPE=<type>"
79 # Use one of following apex, compass, fuel or joid
81 #. Functest needs to know the IP of some installers::
83 -e "INSTALLER_IP=<Specific IP Address>"
85 These two env variables are useful extract some information
86 from the deployment. However, for some test cases like
87 SFC or Barometer they are mandatory since the tests
88 need to access the installer node and the deployment.
90 #. Credentials for accessing the Openstack.
91 Most convenient way of passing them to container is by having a
92 local copy of the credentials file in Jumphost and then using the
93 **-v** option. In the example we have local file by the name of
94 "overcloudrc" and we are using that as an argument::
96 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds
98 The credentials file needs to exist in the Docker container
99 under the path: '/home/opnfv/functest/conf/openstack.creds'.
101 **WARNING:** If you are using the Joid installer, you must pass the
102 credentials using the **-v** option:
103 -v /var/lib/jenkins/admin-openrc:/home/opnfv/functest/conf/openstack.creds.
104 See the section `Accessing the Openstack credentials`_ above.
106 #. Passing deployment scenario
107 When running Functest against any of the supported OPNFV scenarios,
108 it is recommended to include also the environment variable
109 **DEPLOY_SCENARIO**. The **DEPLOY_SCENARIO** environment variable
110 is passed with the format::
112 -e "DEPLOY_SCENARIO=os-<controller>-<nfv_feature>-<ha_mode>"
114 os = OpenStack (No other VIM choices currently available)
115 controller is one of ( nosdn | odl_l2 | odl_l3 | onos | ocl)
116 nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
117 If several features are pertinent then use the underscore
118 character '_' to separate each feature (e.g. ovs_kvm)
119 'nofeature' indicates no NFV feature is deployed
120 ha_mode (high availability) is one of ( ha | noha )
122 **NOTE:** Not all possible combinations of "DEPLOY_SCENARIO" are
123 supported. The name passed in to the Functest Docker container
124 must match the scenario used when the actual OPNFV platform was
125 deployed. See release note to see the list of supported scenarios.
127 **NOTE:** The scenario name is mainly used to automatically detect
128 if a test suite is runnable or not (e.g. it will prevent ONOS test suite
129 to be run on ODL scenarios). If not set, Functest will try to run the
130 default test cases that might not include SDN controller or a specific
133 **NOTE:** A HA scenario means that 3 OpenStack controller nodes are
134 deployed. It does not necessarily mean that the whole system is HA. See
135 installer release notes for details.
138 Putting all above together, when using installer 'fuel' and an invented
139 INSTALLER_IP of '10.20.0.2', the recommended command to create the
140 Functest Docker container is as follows::
142 docker run --name "FunctestContainer" -it \
143 -e "INSTALLER_IP=10.20.0.2" \
144 -e "INSTALLER_TYPE=fuel" \
145 -e "DEPLOY_SCENARIO=os-odl_l2-ovs_kvm-ha" \
146 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
147 opnfv/functest /bin/bash
149 After the *run* command, a new prompt appears which means that we are inside
150 the container and ready to move to the next step.
152 For tips on how to set up container with installer Apex, see chapter
153 `Apex Installer Tips`_.
155 Finally, three additional environment variables can also be passed in
156 to the Functest Docker Container, using the -e
157 "<EnvironmentVariable>=<Value>" mechanism. The first two of these are
158 only relevant to Jenkins CI invoked testing and **should not be used**
159 when performing manual test scenarios::
161 -e "NODE_NAME=<Test POD Name>" \
162 -e "BUILD_TAG=<Jenkins Build Tag>" \
163 -e "CI_DEBUG=<DebugTraceValue>"
165 <Test POD Name> = Symbolic name of the POD where the tests are run.
166 Visible in test results files, which are stored
167 to the database. This option is only used when
168 tests are activated under Jenkins CI control.
169 It indicates the POD/hardware where the test has
170 been run. If not specified, then the POD name is
171 defined as "Unknown" by default.
172 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
173 <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
174 Visible in test results files, which are stored
175 to the database. This option is only set when
176 tests are activated under Jenkins CI control.
177 It enables the correlation of test results,
179 are independently pushed to the results database
180 from different Jenkins jobs.
181 DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
182 <DebugTraceValue> = "true" or "false"
183 Default = "false", if not specified
184 If "true" is specified, then additional debug trace
185 text can be sent to the test results file / log files
186 and also to the standard console output.
190 Some specific tips are useful for the Apex Installer case. If not using
191 Apex Installer; ignore this section.
193 In case of Triple-O based installer (like Apex) the docker container
194 needs to connect to the installer VM, so it is then required that some
195 known SSH keys are present in docker container. Since the Jumphost root
196 SSH keys are already known, easiest way is to use those using the
197 'Bind mount' method. See below for sample parameter::
199 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa
201 NOTE: You need the "sudo" when creating the container to access root
202 users ssh credentials even the docker command itself might not
205 HINT! In case of Triple-O installers you can find value for the
206 INSTALLER_IP parameter by executing command and note the returned IP
209 inst=$(sudo virsh list | grep -iEo "undercloud|instack")
210 sudo virsh domifaddr ${inst}
212 NOTE: In releases prior to Colorado, the name 'instack' was
213 used. Currently the name 'undercloud' is used.
215 You can copy the credentials file from the "stack" users home directory
216 in installer VM to Jumphost. Please check the correct IP from the
217 command above. In the example below we are using invented IP address
220 scp stack@192.168.122.89:overcloudrc .
222 Here is an example of the full docker command invocation for an Apex
223 installed system, using latest Functest docker container, for
224 illustration purposes::
226 sudo docker run -it --name "ApexFuncTestODL" \
227 -e "INSTALLER_IP=192.168.122.89" \
228 -e "INSTALLER_TYPE=apex" \
229 -e "DEPLOY_SCENARIO=os-odl_l2-nofeature-ha" \
230 -v /root/.ssh/id_rsa:/root/.ssh/id_rsa \
231 -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
232 opnfv/functest /bin/bash
234 Compass installer local development env usage Tips
235 --------------------------------------------------
236 In the compass-functest local test case check and development environment,
237 in order to get openstack service inside the functest container, some
238 parameters should be configured during container creation, which are
239 hard to guess for freshman. This section will provide the guideline, the
240 parameters values are defaults here, which should be adjusted according
241 to the settings, the complete steps are given here so as not to appear
244 1, Pull Functest docker image from public dockerhub::
246 docker pull opnfv/functest:<Tag>
248 <Tag> here can be "brahmaputra.1.0", "colorado.1.0", etc.
249 Tag omitted means the latest docker image::
251 docker pull opnfv/functest
253 2, Functest Docker container creation
255 To make a file used for the environment, such as 'functest-docker-env'::
257 OS_AUTH_URL=http://172.16.1.222:35357/v2.0
261 OS_VOLUME_API_VERSION=2
262 OS_PROJECT_NAME=admin
263 INSTALLER_TYPE=compass
264 INSTALLER_IP=192.168.200.2
265 EXTERNAL_NETWORK=ext-net
267 Note: please adjust the content according to the environment, such as
268 'TENANT_ID' maybe used for some special cases.
270 Then to create the Functest docker::
272 docker run --privileged=true --rm -t \
273 --env-file functest-docker-env \
274 --name <Functest_Container_Name> \
275 opnfv/functest:<Tag> /bin/bash
277 3, To attach Functest container
279 Before trying to attach the Functest container, the status can be checked by::
283 to attach the 'Up' status Functest container and start bash mode::
285 docker exec -it <Functest_Container_Name> bash
287 4, Functest environment preparation and check
289 To see the Section below `Preparing the Functest environment`_.
291 Functest docker container directory structure
292 ---------------------------------------------
293 Inside the Functest docker container, the following directory structure
294 should now be in place::
321 Underneath the '/home/opnfv/' directory, the Functest docker container
322 includes two main directories:
324 * The **functest** directory stores configuration files (e.g. the
325 OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
326 the **data** directory stores a 'cirros' test image used in some
327 functional tests and the **results** directory stores some temporary
329 * The **repos** directory holds various repositories. The directories
330 are used for the installation of the needed tooling (e.g. rally) or
331 for the retrieval of feature projects scenarios (e.g. promise)
333 The structure under the **functest** repository can be described as
339 |-- run_unit_tests.sh
341 |-- test-requirements.txt
345 | `--traffic-profile-guidelines.rst
348 | |-- config_install_env.sh
349 | `-- docker_remote_api
363 | |-- check_deployment.py
364 | |-- config_functest.yaml
365 | |-- config_patch.yaml
366 | |-- generate_report.py
370 | |-- tier_builder.py
371 | `-- tier_handler.py
376 | |-- functest-complete.sh
381 | |-- pytest_suite_runner.py
400 |-- functest_logger.py
401 |-- functest_utils.py
403 |-- openstack_clean.py
404 |-- openstack_snapshot.py
405 |-- openstack_tacker.py
406 `-- openstack_utils.py
409 (Note: All *.pyc files removed from above list for brevity...)
411 We may distinguish several directories, the first level has 4 directories:
413 * **commons**: This directory is dedicated for storage of traffic
414 profile or any other test inputs that could be reused by any test
416 * **docker**: This directory includes the needed files and tools to
417 build the Functest Docker image.
418 * **docs**: This directory includes documentation: Release Notes,
419 User Guide, Configuration Guide and Developer Guide.
420 * **functest**: This directory contains all the code needed to run
421 functest internal cases and OPNFV onboarded feature or VNF test cases.
423 Functest directory has 6 directories:
424 * **ci**: This directory contains test structure definition files
425 (e.g <filename>.yaml) and bash shell/python scripts used to
426 configure and execute Functional tests. The test execution script
427 can be executed under the control of Jenkins CI jobs.
428 * **cli**: This directory holds the python based Functest CLI utility
429 source code, which is based on the Python 'click' framework.
430 * **core**: This directory holds the python based Functest core
431 source code. Three abstraction classes have been created to ease
432 the integration of internal, feature or vnf cases.
433 * **opnfv_tests**: This directory includes the scripts required by
434 Functest internal test cases and other feature projects test cases.
435 * **tests**: This directory includes the functest unit tests
436 * **utils**: this directory holds Python source code for some general
437 purpose helper utilities, which testers can also re-use in their
438 own test code. See for an example the Openstack helper utility:
439 'openstack_utils.py'.
441 Useful Docker commands
442 ----------------------
443 When typing **exit** in the container prompt, this will cause exiting
444 the container and probably stopping it. When stopping a running Docker
445 container all the changes will be lost, there is a keyboard shortcut
446 to quit the container without stopping it: <CTRL>-P + <CTRL>-Q. To
447 reconnect to the running container **DO NOT** use the *run* command
448 again (since it will create a new container), use the *exec* or *attach*
451 docker ps # <check the container ID from the output>
452 docker exec -ti <CONTAINER_ID> /bin/bash
454 There are other useful Docker commands that might be needed to manage possible
455 issues with the containers.
457 List the running containers::
461 List all the containers including the stopped ones::
465 Start a stopped container named "FunTest"::
469 Attach to a running container named "StrikeTwo"::
471 docker attach StrikeTwo
473 It is useful sometimes to remove a container if there are some problems::
475 docker rm <CONTAINER_ID>
477 Use the *-f* option if the container is still running, it will force to
480 docker rm -f <CONTAINER_ID>
482 Check the Docker documentation dockerdocs_ for more information.
484 Preparing the Functest environment
485 ----------------------------------
486 Once the Functest docker container is up and running, the required
487 Functest environment needs to be prepared. A custom built **functest**
488 CLI utility is available to perform the needed environment preparation
489 action. Once the environment is prepared, the **functest** CLI utility
490 can be used to run different functional tests. The usage of the
491 **functest** CLI utility to run tests is described further in the
492 Functest User Guide `OPNFV_FuncTestUserGuide`_
494 Prior to commencing the Functest environment preparation, we can check
495 the initial status of the environment. Issue the **functest env status**
496 command at the prompt::
499 Functest environment is not installed.
501 Note: When the Functest environment is prepared, the command will
502 return the status: "Functest environment ready to run tests."
504 To prepare the Functest docker container for test case execution, issue
505 the **functest env prepare** command at the prompt::
509 This script will make sure that the requirements to run the tests are
510 met and will install the needed libraries and tools by all Functest
511 test cases. It should be run only once every time the Functest docker
512 container is started from scratch. If you try to run this command, on
513 an already prepared environment, you will be prompted whether you really
514 want to continue or not::
517 It seems that the environment has been already prepared.
518 Do you want to do it again? [y|n]
520 (Type 'n' to abort the request, or 'y' to repeat the
521 environment preparation)
524 To list some basic information about an already prepared Functest
525 docker container environment, issue the **functest env show** at the
529 +======================================================+
530 | Functest Environment info |
531 +======================================================+
532 | INSTALLER: apex, 192.168.122.89 |
533 | SCENARIO: os-odl_l2-nofeature-ha |
535 | GIT BRANCH: master |
536 | GIT HASH: 5bf1647dec6860464eeb082b2875798f0759aa91 |
537 | DEBUG FLAG: false |
538 +------------------------------------------------------+
540 +------------------------------------------------------+
544 INSTALLER: Displays the INSTALLER_TYPE value
546 and the INSTALLER_IP value
547 - here = "192.168.122.89"
548 SCENARIO: Displays the DEPLOY_SCENARIO value
549 - here = "os-odl_l2-nofeature-ha"
550 POD: Displays the value passed in NODE_NAME
552 GIT BRANCH: Displays the git branch of the OPNFV Functest
553 project repository included in the Functest
556 (In first official colorado release
557 would be "colorado.1.0")
558 GIT HASH: Displays the git hash of the OPNFV Functest
559 project repository included in the Functest
561 - here = "5bf1647dec6860464eeb082b2875798f0759aa91"
562 DEBUG FLAG: Displays the CI_DEBUG value
565 NOTE: In Jenkins CI runs, an additional item "BUILD TAG"
566 would also be listed. The value is set by Jenkins CI.
568 Finally, the **functest** CLI has a **--help** options:
572 functest --help Usage: functest [OPTIONS] COMMAND [ARGS]...
575 --version Show the version and exit.
576 -h, --help Show this message and exit.
585 Usage: functest env [OPTIONS] COMMAND [ARGS]...
588 -h, --help Show this message and exit.
591 prepare Prepares the Functest environment.
592 show Shows information about the current...
593 status Checks if the Functest environment is ready...
595 Checking Openstack and credentials
596 ----------------------------------
597 It is recommended and fairly straightforward to check that Openstack
598 and credentials are working as expected.
600 Once the credentials are there inside the container, they should be
601 sourced before running any Openstack commands::
603 source /home/opnfv/functest/conf/openstack.creds
605 After this, try to run any OpenStack command to see if you get any
606 output, for instance::
610 This will return a list of the actual users in the OpenStack
611 deployment. In any other case, check that the credentials are sourced::
615 This command must show a set of environment variables starting with
618 OS_REGION_NAME=RegionOne
619 OS_DEFAULT_DOMAIN=default
620 OS_PROJECT_NAME=admin
622 OS_AUTH_STRATEGY=keystone
623 OS_AUTH_URL=http://172.30.10.3:5000/v2.0
626 OS_ENDPOINT_TYPE=internalURL
629 If the OpenStack command still does not show anything or complains
630 about connectivity issues, it could be due to an incorrect url given to
631 the OS_AUTH_URL environment variable. Check the deployment settings.
635 If you need to connect to a server that is TLS-enabled (the auth URL
636 begins with "https") and it uses a certificate from a private CA or a
637 self-signed certificate, then you will need to specify the path to an
638 appropriate CA certificate to use, to validate the server certificate
639 with the environment variable OS_CACERT::
642 /etc/ssl/certs/ca.crt
644 However, this certificate does not exist in the container by default.
645 It has to be copied manually from the OpenStack deployment. This can be
648 #. Create manually that file and copy the contents from the OpenStack
650 #. (Recommended) Add the file using a Docker volume when starting the
653 -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
655 You might need to export OS_CACERT environment variable inside the
658 export OS_CACERT=/etc/ssl/certs/ca.crt
660 Certificate verification can be turned off using OS_INSECURE=true. For
661 example, Fuel uses self-signed cacerts by default, so an pre step would
664 export OS_INSECURE=true
668 If your Jumphost node is operating behind a http proxy, then there are
669 2 places where some special actions may be needed to make operations
672 #. Initial installation of docker engine First, try following the
673 official Docker documentation for Proxy settings. Some issues were
674 experienced on CentOS 7 based Jumphost. Some tips are documented
675 in section: `Docker Installation on CentOS behind http proxy`_
678 #. Execution of the Functest environment preparation inside the
679 created docker container Functest needs internet access to
680 download some resources for some test cases. This might not
681 work properly if the Jumphost is connecting to internet
682 through a http Proxy.
684 If that is the case, make sure the resolv.conf and the needed
685 http_proxy and https_proxy environment variables, as well as the
686 'no_proxy' environment variable are set correctly::
688 # Make double sure that the 'no_proxy=...' line in the
689 # 'openstack.creds' file is commented out first. Otherwise, the
690 # values set into the 'no_proxy' environment variable below will
691 # be ovewrwritten, each time the command
692 # 'source ~/functest/conf/openstack.creds' is issued.
695 sed -i 's/export no_proxy/#export no_proxy/' openstack.creds
696 source ./openstack.creds
698 # Next calculate some IP addresses for which http_proxy
699 # usage should be excluded:
701 publicURL_IP=$(echo $OS_AUTH_URL | grep -Eo "([0-9]+\.){3}[0-9]+")
703 adminURL_IP=$(openstack catalog show identity | \
704 grep adminURL | grep -Eo "([0-9]+\.){3}[0-9]+")
706 export http_proxy="<your http proxy settings>"
707 export https_proxy="<your https proxy settings>"
708 export no_proxy="127.0.0.1,localhost,$publicURL_IP,$adminURL_IP"
710 # Ensure that "git" uses the http_proxy
711 # This may be needed if your firewall forbids SSL based git fetch
712 git config --global http.sslVerify True
713 git config --global http.proxy <Your http proxy settings>
715 Validation check: Before running **'functest env prepare'** CLI command,
716 make sure you can reach http and https sites from inside the Functest
719 For example, try to use the **nc** command from inside the functest
723 Connection to opnfv.org 80 port [tcp/http] succeeded!
726 Connection to opnfv.org 443 port [tcp/https] succeeded!
728 Note: In a Jumphost node based on the CentOS family OS, the **nc**
729 commands might not work. You can use the **curl** command instead.
731 curl http://www.opnfv.org:80
732 <HTML><HEAD><meta http-equiv="content-type"
737 curl https://www.opnfv.org:443
738 <HTML><HEAD><meta http-equiv="content-type"
743 (Ignore the content. If command returns a valid HTML page, it proves
746 Docker Installation on CentOS behind http proxy
747 -----------------------------------------------
748 This section is applicable for CentOS family OS on Jumphost which
749 itself is behind a proxy server. In that case, the instructions below
750 should be followed **before** installing the docker engine::
752 1) # Make a directory '/etc/systemd/system/docker.service.d'
753 # if it does not exist
754 sudo mkdir /etc/systemd/system/docker.service.d
756 2) # Create a file called 'env.conf' in that directory with
757 # the following contents:
759 EnvironmentFile=-/etc/sysconfig/docker
761 3) # Set up a file called 'docker' in directory '/etc/sysconfig'
762 # with the following contents:
763 HTTP_PROXY="<Your http proxy settings>"
764 HTTPS_PROXY="<Your https proxy settings>"
765 http_proxy="${HTTP_PROXY}"
766 https_proxy="${HTTPS_PROXY}"
768 4) # Reload the daemon
769 systemctl daemon-reload
771 5) # Sanity check - check the following docker settings:
772 systemctl show docker | grep -i env
776 EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
777 DropInPaths=/etc/systemd/system/docker.service.d/env.conf
779 Now follow the instructions in [`InstallDockerCentOS`_] to download
780 and install the **docker-engine**. The instructions conclude with a
781 "test pull" of a sample "Hello World" docker container. This should now
782 work with the above pre-requisite actions.
784 .. _dockerdocs: https://docs.docker.com/
785 .. _dockerhub: https://hub.docker.com/r/opnfv/functest/
786 .. _Proxy: https://docs.docker.com/engine/admin/systemd/#http-proxy
787 .. _FunctestDockerTags: https://hub.docker.com/r/opnfv/functest/tags/
788 .. _InstallDockerCentOS: https://docs.docker.com/engine/installation/linux/centos/
789 .. _OPNFV_FuncTestUserGuide: http://artifacts.opnfv.org/functest/docs/userguide/index.html