.. This work is licensed under a Creative Commons Attribution 4.0 International License.
-.. http://creativecommons.org/licenses/by/4.0
+.. SPDX-License-Identifier: CC-BY-4.0
+Installation and configuration
+==============================
-Pulling the Docker image
-------------------------
-Pull the Functest Docker image ('opnfv/functest') from the public
-dockerhub registry under the OPNFV account: [dockerhub_], with the
-following docker command::
-
- docker pull opnfv/functest:<TagIdentifier>
-
-where <TagIdentifier> identifies a release of the Functest docker
-container image in the public dockerhub registry. There are many tags
-created automatically by the CI mechanisms, and you must ensure you
-pull an image with the **correct tag** to match the OPNFV software
-release installed in your environment. All available tagged images can
-be seen from location [FunctestDockerTags_]. For example, when running
-on the first official release of the OPNFV Colorado system platform,
-tag "colorado.1.0" is needed. Pulling other tags might cause some
-problems while running the tests.
-Docker images pulled without a tag specifier bear the implicitly
-assigned label "latest". If you need to specifically pull the latest
-Functest docker image, then omit the tag argument::
-
- docker pull opnfv/functest
-
-After pulling the Docker image, check that it is available with the
-following docker command::
-
- [functester@jumphost ~]$ docker images
- REPOSITORY TAG IMAGE ID CREATED SIZE
- opnfv/functest latest 8cd6683c32ae 2 weeks ago 1.321 GB
- opnfv/functest danube.1.0 13fa54a1b238 4 weeks ago 1.29 GB
- opnfv/functest colorado.1.0 94b78faa94f7 9 weeks ago 968 MB
+Alpine containers have been introduced in Euphrates.
+Alpine allows Functest testing in several very light containers and thanks to
+the refactoring on dependency management should allow the creation of light and
+fully customized docker images.
-The Functest docker container environment can -in principle- be also
-used with non-OPNFV official installers (e.g. 'devstack'), with the
-**disclaimer** that support for such environments is outside of the
-scope and responsibility of the OPNFV project.
+It is still possible to use the monolithic Ubuntu image opnfv/functest especially
+for tests on Aarch64 architecture.
-Accessing the Openstack credentials
------------------------------------
-OpenStack credentials are mandatory and can be retrieved in different
-ways. From inside the running Functest docker container the
-"functest env prepare" command will automatically look for the
-Openstack credentials file "/home/opnfv/functest/conf/openstack.creds"
-and retrieve it unless the file already exists. This Functest
-environment preparation step is described later in this document.
-
-WARNING: When the installer type is "joid" you have to have the
-credentials file inside the running container **before** initiating the
-functest environment preparation. For that reason you have to choose
-either one of the options below, since the automated copying does not
-work for "joid".
-
-You can also specifically pass in the needed file prior to running the
-environment preparation either:
-
- * by using the -v option when creating the Docker container. This is
- referred to in docker documentation as "Bind Mounting". See the
- usage of this parameter in the following chapter.
- * or creating a local file '/home/opnfv/functest/conf/openstack.creds'
- inside the running container with the credentials in it. Consult
- your installer guide for further details. This is however not
- instructed in this document.
+Functest Dockers
+----------------
+Docker images are available on the dockerhub:
-NOTE: When the installer type is "fuel" and virtualized deployment
-is used, there you have to explicitly fetch the credentials file
-executing the following sequence
+ * opnfv/functest-core
+ * opnfv/functest-healthcheck
+ * opnfv/functest-smoke
+ * opnfv/functest-features
+ * opnfv/functest-components
+ * opnfv/functest-vnf
+ * opnfv/functest-parser
+ * opnfv/functest-restapi
- #. Create a container as described in next chapter but do not
- "Bind Mount" the credentials
- #. Log in to container and execute the following command. Replace
- the IP with installer address after the "-a" parameter::
+The tag "opnfv-5.0.0" is the official release image in Euphrates, but you can also pull "euphrates"
+tag as it is being maintained by Functest team and might include bugfixes.
- $REPOS_DIR/releng/utils/fetch_os_creds.sh \
- -d /home/opnfv/functest/conf/openstack.creds \
- -i fuel \
- -a 10.20.0.2 \
- -v
- ( -d specifies the full path to the Openstack credential file
- -i specifies the INSTALLER_TYPE
- -a specifies the INSTALLER_IP
- -v indicates a virtualized environment and takes no arguments )
-
- #. Continue with your testing, initiate functest environment
- preparation, run tests etc.
+The Functest docker container environment can -in principle- be also
+used with non-OPNFV official installers (e.g. 'devstack'), with the
+**disclaimer** that support for such environments is outside of the
+scope and responsibility of the OPNFV project.
-In proxified environment you may need to change the credentials file.
-There are some tips in chapter: `Proxy support`_
-Functest Docker parameters
+Preparing your environment
--------------------------
-This chapter explains how to run a container for executing functest
-test suites. Numbered list below explains some details of the
-recommended parameters for invoking docker container
- #. It is a good practice to assign a precise container name through
- the **--name** option.
+cat env::
- #. Assign parameter for installer type::
+ INSTALLER_TYPE=XXX
+ INSTALLER_IP=XXX
+ EXTERNAL_NETWORK=XXX
+ DEPLOY_SCENARIO=XXX
- -e "INSTALLER_TYPE=<type>"
- # Use one of following apex, compass, fuel or joid
+See section on environment variables for details.
- #. Functest needs to know the IP of some installers::
+cat openstack.creds::
- -e "INSTALLER_IP=<Specific IP Address>"
+ export OS_AUTH_URL=XXX
+ export OS_USER_DOMAIN_NAME=XXX
+ export OS_PROJECT_DOMAIN_NAME=XXX
+ export OS_USERNAME=XXX
+ export OS_TENANT_NAME=XXX
+ export OS_PROJECT_NAME=XXX
+ export OS_PASSWORD=XXX
+ export OS_VOLUME_API_VERSION=XXX
+ export OS_IDENTITY_API_VERSION=XXX
+ export OS_IMAGE_API_VERSION=XXX
- This IP is needed to fetch RC file from deployment, fetch logs, ...
- If not provided, there is no way to fetch the RC file. It must be
- provided manually as a volume
+See section on OpenStack credentials for details.
- #. Credentials for accessing the Openstack.
- Most convenient way of passing them to container is by having a
- local copy of the credentials file in Jumphost and then using the
- **-v** option. In the example we have local file by the name of
- "overcloudrc" and we are using that as an argument::
+Create a directory for the different images (attached as a Docker volume)::
- -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds
+ mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/euphrates | bash -s -- images && ls -1 images/*
- The credentials file needs to exist in the Docker container
- under the path: '/home/opnfv/functest/conf/openstack.creds'.
+ images/CentOS-7-aarch64-GenericCloud.qcow2
+ images/CentOS-7-aarch64-GenericCloud.qcow2.xz
+ images/CentOS-7-x86_64-GenericCloud.qcow2
+ images/cirros-0.3.5-x86_64-disk.img
+ images/cirros-0.3.5-x86_64-lxc.tar.gz
+ images/cirros-d161201-aarch64-disk.img
+ images/cirros-d161201-aarch64-initramfs
+ images/cirros-d161201-aarch64-kernel
+ images/cloudify-manager-premium-4.0.1.qcow2
+ images/img
+ images/trusty-server-cloudimg-amd64-disk1.img
+ images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
+ images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
+ images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
+ images/vyos-1.1.7.img
- **WARNING:** If you are using the Joid installer, you must pass the
- credentials using the **-v** option:
- -v /var/lib/jenkins/admin-openrc:/home/opnfv/functest/conf/openstack.creds.
- See the section `Accessing the Openstack credentials`_ above.
- #. Passing deployment scenario
- When running Functest against any of the supported OPNFV scenarios,
- it is recommended to include also the environment variable
- **DEPLOY_SCENARIO**. The **DEPLOY_SCENARIO** environment variable
- is passed with the format::
+Testing healthcheck suite
+--------------------------
- -e "DEPLOY_SCENARIO=os-<controller>-<nfv_feature>-<ha_mode>"
- where:
- os = OpenStack (No other VIM choices currently available)
- controller is one of ( nosdn | odl_l2 | odl_l3 | onos | ocl)
- nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
- If several features are pertinent then use the underscore
- character '_' to separate each feature (e.g. ovs_kvm)
- 'nofeature' indicates no NFV feature is deployed
- ha_mode (high availability) is one of ( ha | noha )
+Run healthcheck suite::
- **NOTE:** Not all possible combinations of "DEPLOY_SCENARIO" are
- supported. The name passed in to the Functest Docker container
- must match the scenario used when the actual OPNFV platform was
- deployed. See release note to see the list of supported scenarios.
+ sudo docker run --env-file env \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
+ -v $(pwd)/images:/home/opnfv/functest/images \
+ opnfv/functest-healthcheck
- **NOTE:** The scenario name is mainly used to automatically detect
- if a test suite is runnable or not (e.g. it will prevent ONOS test suite
- to be run on ODL scenarios). If not set, Functest will try to run the
- default test cases that might not include SDN controller or a specific
- feature
+Results shall be displayed as follows::
- **NOTE:** A HA scenario means that 3 OpenStack controller nodes are
- deployed. It does not necessarily mean that the whole system is HA. See
- installer release notes for details.
+ +----------------------------+------------------+---------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +----------------------------+------------------+---------------------+------------------+----------------+
+ | connection_check | functest | healthcheck | 00:02 | PASS |
+ | api_check | functest | healthcheck | 04:57 | PASS |
+ | snaps_health_check | functest | healthcheck | 00:51 | PASS |
+ +----------------------------+------------------+---------------------+------------------+----------------+
+ NOTE: the duration is a reference and it might vary depending on your SUT.
+Testing smoke suite
+-------------------
-Putting all above together, when using installer 'fuel' and an invented
-INSTALLER_IP of '10.20.0.2', the recommended command to create the
-Functest Docker container is as follows::
+Run smoke suite::
+
+ sudo docker run --env-file env \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
+ -v $(pwd)/images:/home/opnfv/functest/images \
+ opnfv/functest-smoke
+
+Results shall be displayed as follows::
+
+ +------------------------------+------------------+---------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +------------------------------+------------------+---------------+------------------+----------------+
+ | vping_ssh | functest | smoke | 01:19 | PASS |
+ | vping_userdata | functest | smoke | 01:56 | PASS |
+ | tempest_smoke_serial | functest | smoke | 26:30 | PASS |
+ | rally_sanity | functest | smoke | 19:42 | PASS |
+ | refstack_defcore | functest | smoke | 22:00 | PASS |
+ | snaps_smoke | functest | smoke | 41:14 | PASS |
+ | odl | functest | smoke | 00:16 | PASS |
+ | odl_netvirt | functest | smoke | 00:00 | SKIP |
+ | fds | functest | smoke | 00:00 | SKIP |
+ +------------------------------+------------------+---------------+------------------+----------------+
+ Note: if the scenario does not support some tests, they are indicated as SKIP.
+ See User guide for details.
+
+Testing features suite
+----------------------
- docker run --name "FunctestContainer" -it \
- -e "INSTALLER_IP=10.20.0.2" \
- -e "INSTALLER_TYPE=fuel" \
- -e "DEPLOY_SCENARIO=os-odl_l2-ovs_kvm-ha" \
- -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
- opnfv/functest /bin/bash
+Run features suite::
+
+ sudo docker run --env-file env \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
+ -v $(pwd)/images:/home/opnfv/functest/images \
+ opnfv/functest-features
+
+Results shall be displayed as follows::
+
+ +---------------------------+--------------------------+------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +---------------------------+--------------------------+------------------+------------------+----------------+
+ | promise | promise | features | 00:00 | SKIP |
+ | bgpvpn | sdnvpn | features | 00:00 | SKIP |
+ | security_scan | securityscanning | features | 00:00 | SKIP |
+ | functest-odl-sfc | sfc | features | 00:00 | SKIP |
+ | domino-multinode | domino | features | 00:00 | SKIP |
+ | barometercollectd | barometer | features | 00:00 | SKIP |
+ +---------------------------+--------------------------+------------------+------------------+----------------+
+ Note: if the scenario does not support some tests, they are indicated as SKIP.
+ See User guide for details.
+
+Testing components suite
+------------------------
-After the *run* command, a new prompt appears which means that we are inside
-the container and ready to move to the next step.
+Run components suite::
-For tips on how to set up container with installer Apex, see chapter
-`Apex Installer Tips`_.
+ sudo docker run --env-file env \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
+ -v $(pwd)/images:/home/opnfv/functest/images \
+ opnfv/functest-components
-Finally, three additional environment variables can also be passed in
-to the Functest Docker Container, using the -e
-"<EnvironmentVariable>=<Value>" mechanism. The first two of these are
-only relevant to Jenkins CI invoked testing and **should not be used**
-when performing manual test scenarios::
-
- -e "NODE_NAME=<Test POD Name>" \
- -e "BUILD_TAG=<Jenkins Build Tag>" \
- -e "CI_DEBUG=<DebugTraceValue>"
- where:
- <Test POD Name> = Symbolic name of the POD where the tests are run.
- Visible in test results files, which are stored
- to the database. This option is only used when
- tests are activated under Jenkins CI control.
- It indicates the POD/hardware where the test has
- been run. If not specified, then the POD name is
- defined as "Unknown" by default.
- DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
- <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
- Visible in test results files, which are stored
- to the database. This option is only set when
- tests are activated under Jenkins CI control.
- It enables the correlation of test results,
- which
- are independently pushed to the results datbase
- from different Jenkins jobs.
- DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
- <DebugTraceValue> = "true" or "false"
- Default = "false", if not specified
- If "true" is specified, then additional debug trace
- text can be sent to the test results file / log files
- and also to the standard console output.
-
-Apex Installer Tips
--------------------
-Some specific tips are useful for the Apex Installer case. If not using
-Apex Installer; ignore this section.
-
-In case of Triple-O based installer (like Apex) the docker container
-needs to connect to the installer VM, so it is then required that some
-known SSH keys are present in docker container. Since the Jumphost root
-SSH keys are already known, easiest way is to use those using the
-'Bind mount' method. See below for sample parameter::
+Results shall be displayed as follows::
- -v /root/.ssh/id_rsa:/root/.ssh/id_rsa
+ +-------------------------------+------------------+--------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +-------------------------------+------------------+--------------------+------------------+----------------+
+ | tempest_full_parallel | functest | components | 102:48 | PASS |
+ | rally_full | functest | components | 160:58 | PASS |
+ +-------------------------------+------------------+--------------------+------------------+----------------+
- NOTE: You need the "sudo" when creating the container to access root
- users ssh credentials even the docker command itself might not
- require that.
+Testing vnf suite
+-----------------
-HINT! In case of Triple-O installers you can find value for the
-INSTALLER_IP parameter by executing command and note the returned IP
-address::
+Run vnf suite::
- inst=$(sudo virsh list | grep -iEo "undercloud|instack")
- sudo virsh domifaddr ${inst}
+sudo docker run --env-file env \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
+ -v $(pwd)/images:/home/opnfv/functest/images \
+ opnfv/functest-vnf
- NOTE: In releases prior to Colorado, the name 'instack' was
- used. Currently the name 'undercloud' is used.
+Results shall be displayed as follows::
-You can copy the credentials file from the "stack" users home directory
-in installer VM to Jumphost. Please check the correct IP from the
-command above. In the example below we are using invented IP address
-"192.168.122.89"::
+ +---------------------------------+------------------+--------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +---------------------------------+------------------+--------------+------------------+----------------+
+ | cloudify_ims | functest | vnf | 21:25 | PASS |
+ | orchestra_openims | functest | vnf | 11:02 | FAIL |
+ | orchestra_clearwaterims | functest | vnf | 09:13 | FAIL |
+ +---------------------------------+------------------+--------------+------------------+----------------+
- scp stack@192.168.122.89:overcloudrc .
-Here is an example of the full docker command invocation for an Apex
-installed system, using latest Functest docker container, for
-illustration purposes::
+Environment variables
+=====================
- sudo docker run -it --name "ApexFuncTestODL" \
- -e "INSTALLER_IP=192.168.122.89" \
- -e "INSTALLER_TYPE=apex" \
- -e "DEPLOY_SCENARIO=os-odl_l2-nofeature-ha" \
- -v /root/.ssh/id_rsa:/root/.ssh/id_rsa \
- -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \
- opnfv/functest /bin/bash
+Several environement variables may be specified:
+ * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid)
+ * INSTALLER_IP=<Specific IP Address>
+ * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
-Compass installer local development env usage Tips
---------------------------------------------------
-In the compass-functest local test case check and development environment,
-in order to get openstack service inside the functest container, some
-parameters should be configured during container creation, which are
-hard to guess for freshman. This section will provide the guideline, the
-parameters values are defaults here, which should be adjusted according
-to the settings, the complete steps are given here so as not to appear
-too abruptly.
-1, Pull Functest docker image from public dockerhub::
+INSTALLER IP may be required by some test cases like SFC or Barometer in order
+to access the installer node and the deployment.
- docker pull opnfv/functest:<Tag>
+The format for the DEPLOY_SCENARIO env variable can be described as follows:
+ * vim: (os|k8s) = OpenStack or Kubernetes
+ * controller is one of ( nosdn | odl )
+ * nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature )
+ * ha_mode (high availability) is one of ( ha | noha )
-<Tag> here can be "brahmaputra.1.0", "colorado.1.0", etc.
-Tag omitted means the latest docker image::
+If several features are pertinent then use the underscore character '_' to
+separate each feature (e.g. ovs_kvm). 'nofeature' indicates that no OPNFV
+feature is deployed.
- docker pull opnfv/functest
+The list of supported scenarios per release/installer is indicated in the
+release note.
-2, Functest Docker container creation
+**NOTE:** The scenario name is mainly used to automatically detect
+if a test suite is runnable or not (e.g. it will prevent ODL test suite to be
+run on 'nosdn' scenarios). If not set, Functest will try to run the default test
+cases that might not include SDN controller or a specific feature
-To make a file used for the environment, such as 'functest-docker-env'::
+**NOTE:** An HA scenario means that 3 OpenStack controller nodes are
+deployed. It does not necessarily mean that the whole system is HA. See
+installer release notes for details.
- OS_AUTH_URL=http://172.16.1.222:35357/v2.0
- OS_USERNAME=admin
- OS_PASSWORD=console
- OS_TENANT_NAME=admin
- OS_VOLUME_API_VERSION=2
- OS_PROJECT_NAME=admin
- INSTALLER_TYPE=compass
- INSTALLER_IP=192.168.200.2
- EXTERNAL_NETWORK=ext-net
+Finally, three additional environment variables can also be passed in
+to the Functest Docker Container, using the -e
+"<EnvironmentVariable>=<Value>" mechanism. The first two parameters are
+only relevant to Jenkins CI invoked testing and **should not be used**
+when performing manual test scenarios:
+
+ * NODE_NAME = <Test POD Name>
+ * BUILD_TAG = <Jenkins Build Tag>
+ * CI_DEBUG = <DebugTraceValue>
+
+where:
+
+ * <Test POD Name> = Symbolic name of the POD where the tests are run.
+ Visible in test results files, which are stored
+ to the database. This option is only used when
+ tests are activated under Jenkins CI control.
+ It indicates the POD/hardware where the test has
+ been run. If not specified, then the POD name is
+ defined as "Unknown" by default.
+ DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
+ * <Jenkins Build tag> = Symbolic name of the Jenkins Build Job.
+ Visible in test results files, which are stored
+ to the database. This option is only set when
+ tests are activated under Jenkins CI control.
+ It enables the correlation of test results,
+ which are independently pushed to the results database
+ from different Jenkins jobs.
+ DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
+ * <DebugTraceValue> = "true" or "false"
+ Default = "false", if not specified
+ If "true" is specified, then additional debug trace
+ text can be sent to the test results file / log files
+ and also to the standard console output.
+
+
+Openstack credentials
+=====================
+OpenStack credentials are mandatory and must be provided to Functest.
+When running the command "functest env prepare", the framework will
+automatically look for the Openstack credentials file
+"/home/opnfv/functest/conf/openstack.creds" and will exit with
+error if it is not present or is empty.
+
+There are 2 ways to provide that file:
+
+ * by using a Docker volume with -v option when creating the Docker container.
+ This is referred to in docker documentation as "Bind Mounting".
+ See the usage of this parameter in the following chapter.
+ * or creating manually the file '/home/opnfv/functest/conf/openstack.creds'
+ inside the running container and pasting the credentials in it. Consult
+ your installer guide for further details. This is however not
+ instructed in this document.
-Note: please adjust the content according to the environment, such as
-'TENANT_ID' maybe used for some special cases.
+In proxified environment you may need to change the credentials file.
+There are some tips in chapter: `Proxy support`_
-Then to create the Functest docker::
+SSL Support
+-----------
+If you need to connect to a server that is TLS-enabled (the auth URL
+begins with "https") and it uses a certificate from a private CA or a
+self-signed certificate, then you will need to specify the path to an
+appropriate CA certificate to use, to validate the server certificate
+with the environment variable OS_CACERT::
- docker run --privileged=true --rm -t \
- --env-file functest-docker-env \
- --name <Functest_Container_Name> \
- opnfv/functest:<Tag> /bin/bash
+ echo $OS_CACERT
+ /etc/ssl/certs/ca.crt
-3, To attach Functest container
+However, this certificate does not exist in the container by default.
+It has to be copied manually from the OpenStack deployment. This can be
+done in 2 ways:
-Before trying to attach the Functest container, the status can be checked by::
+ #. Create manually that file and copy the contents from the OpenStack
+ controller.
+ #. (Recommended) Add the file using a Docker volume when starting the
+ container::
- docker ps -a
+ -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
-to attach the 'Up' status Functest container and start bash mode::
+You might need to export OS_CACERT environment variable inside the
+credentials file::
- docker exec -it <Functest_Container_Name> bash
+ export OS_CACERT=/etc/ssl/certs/ca.crt
-4, Functest environemnt preparation and check
+Certificate verification can be turned off using OS_INSECURE=true. For
+example, Fuel uses self-signed cacerts by default, so an pre step would
+be::
-To see the Section below `Preparing the Functest environment`_.
+ export OS_INSECURE=true
Functest docker container directory structure
----------------------------------------------
+=============================================
Inside the Functest docker container, the following directory structure
should now be in place::
- `-- home
- `-- opnfv
- |-- functest
- | |-- conf
- | |-- data
- | `-- results
- `-- repos
- |-- bgpvpn
- |-- copper
- |-- doctor
- |-- domino
- |-- functest
- |-- kingbird
- |-- odl_test
- |-- onos
- |-- ovno
- |-- parser
- |-- promise
- |-- rally
- |-- releng
- |-- sdnvpn
- |-- securityscanning
- |-- sfc
- |-- tempest
- |-- vims_test
- `-- vnfs
-
-Underneath the '/home/opnfv/' directory, the Functest docker container
+ `--
+ |- home
+ | |-- opnfv
+ | | `- functest
+ | | |-- conf
+ | | `-- results
+ | `-- repos
+ | `-- vnfs
+ |- src
+ | |-- tempest
+ | |-- vims-test
+ | |-- odl_test
+ | `-- fds
+ `- usr
+ `- lib
+ `- python2.7
+ `- site-packages
+ `- functest
+ |-- ...
+
+Underneath the '/home/opnfv/functest' directory, the Functest docker container
includes two main directories:
- * The **functest** directory stores configuration files (e.g. the
+ * The **conf** directory stores configuration files (e.g. the
OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
- the **data** directory stores a 'cirros' test image used in some
- functional tests and the **results** directory stores some temporary
- result log files
- * The **repos** directory holds various repositories. The directory
- '/home/opnfv/repos/functest' is used to prepare the needed Functest
- environment and to run the tests. The other repository directories
- are used for the installation of the needed tooling (e.g. rally) or
- for the retrieval of feature projects scenarios (e.g. promise)
-
-The structure under the **functest** repository can be described as
-follows::
-
- . |-- INFO
- |-- LICENSE
- |-- requirements.txt
- |-- run_unit_tests.sh
- |-- setup.py
- |-- test-requirements.txt
- |-- commons
- | |-- ims
- | |-- mobile
- | `--traffic-profile-guidelines.rst
- |-- docker
- | |-- Dockerfile
- | |-- config_install_env.sh
- | `-- docker_remote_api
- |-- docs
- | |-- com
- | |-- configguide
- | |-- devguide
- | |-- images
- | |-- internship
- | |-- release-notes
- | |-- results
- | `--userguide
- |-- functest
- |-- __init__.py
- |-- ci
- | |-- __init__.py
- | |-- check_os.sh
- | |-- config_functest.yaml
- | |-- config_patch.yaml
- | |-- generate_report.py
- | |-- prepare_env.py
- | |-- run_tests.py
- | |-- testcases.yaml
- | |-- tier_builder.py
- | `-- tier_handler.py
- |-- cli
- | |-- __init__.py
- | |-- cli_base.py
- | |-- commands
- | |-- functest-complete.sh
- | `-- setup.py
- |-- core
- | |-- __init__.py
- | |-- feature_base.py
- | |-- pytest_suite_runner.py
- | |-- testcase_base.py
- | |-- vnf_base.py
- |-- opnfv_tests
- | |-- __init__.py
- | |-- features
- | |-- mano
- | |-- openstack
- | |-- sdn
- | |-- security_scan
- | `-- vnf
- |-- tests
- | |-- __init__.py
- | `-- unit
- `-- utils
- |-- __init__.py
- |-- config.py
- |-- constants.py
- |-- env.py
- |-- functest_logger.py
- |-- functest_utils.py
- |-- openstack
- |-- openstack_clean.py
- |-- openstack_snapshot.py
- |-- openstack_tacker.py
- `-- openstack_utils.py
-
-
- (Note: All *.pyc files removed from above list for brevity...)
-
-We may distinguish several directories, the first level has 4 directories:
-
+ * the **results** directory stores some temporary result log files
+
+src and repos directories are used to host third party code used for the tests.
+
+The functest code is under /usr/lib/python2.7/site-packages/functest
+The structure can be described as follows::
+
+ |-- INFO
+ |-- LICENSE
+ |-- api
+ | `-- apidoc
+ |-- build.sh
+ |-- commons
+ | |-- docker
+ | |-- Dockerfile
+ | |-- Dockerfile.aarch64.patch
+ | |-- components
+ | |-- config_install_env.sh
+ | |-- core
+ | |-- docker_remote_api
+ | |-- features
+ | |-- healthcheck
+ | |-- smoke
+ | |-- vnf
+ | `-- thirdparty-requirements.txt
+ |-- docs
+ | |-- com
+ | |-- images
+ | |-- release
+ | | `-- release-notes
+ | |-- results
+ | | testing
+ | | |-- developer
+ | | `-- user
+ | | |-- configguide
+ | | `-- userguide
+ `-- functest
+ |-- api
+ | |-- base.py
+ | |-- server.py
+ | |-- urls.py
+ | |-- common
+ | | |-- api_utils.py
+ | | `-- error.py
+ | `-- resources
+ | `-- v1
+ | |-- creds.py
+ | |-- envs.py
+ | |-- testcases.py
+ | `-- tiers.py
+ |-- ci
+ │ |-- check_deployment.py
+ │ |-- config_aarch64_patch.yaml
+ │ |-- config_functest.yaml
+ │ |-- config_patch.yaml
+ │ |-- download_images.sh
+ │ |-- installer_params.yaml
+ │ |-- logging.ini
+ │ |-- prepare_env.py
+ │ |-- rally_aarch64_patch.conf
+ │ |-- run_tests.py
+ │ |-- testcases.yaml
+ │ |-- tier_builder.py
+ │ `-- tier_handler.py
+ |-- cli
+ │ |-- cli_base.py
+ │ |-- commands
+ │ │ |-- cli_env.py
+ │ │ |-- cli_os.py
+ │ │ |-- cli_testcase.py
+ │ │ `-- cli_tier.py
+ │ |-- functest-complete.sh
+ |-- core
+ │ |-- feature.py
+ │ |-- testcase.py
+ │ |-- unit.py
+ │ `-- vnf.py
+ |-- energy
+ │ |-- energy.py
+ │ `-- energy.pyc
+ |-- opnfv_tests
+ │ |-- mano
+ │ │ |-- orchestra.py
+ │ |-- openstack
+ │ │ |-- rally
+ │ │ |-- refstack_client
+ │ │ |-- snaps
+ │ │ |-- tempest
+ │ │ `-- vping
+ │ |-- sdn
+ │ │ `-- odl
+ │ `-- vnf
+ │ |-- aaa
+ │ |-- ims
+ │ `-- router
+ |-- tests
+ │ `-- unit
+ │ |-- ci
+ │ |-- cli
+ │ |-- core
+ │ |-- energy
+ │ |-- features
+ │ |-- odl
+ │ |-- openstack
+ │ |-- opnfv_tests
+ │ |-- test_utils.py
+ │ |-- utils
+ │ `-- vnf
+ |-- utils
+ | |-- config.py
+ | |-- constants.py
+ | |-- decorators.py
+ | |-- env.py
+ | |-- functest_utils.py
+ | |-- functest_vacation.py
+ | |-- openstack_clean.py
+ | |-- openstack_snapshot.py
+ | |-- openstack_tacker.py
+ | `-- openstack_utils.py
+ |-- requirements.txt
+ |-- setup.cfg
+ |-- setup.py
+ |-- test-requirements.txt
+ |-- tox.ini
+ |-- upper-constraints.txt
+
+ (Note: All *.pyc files removed from above list for brevity...)
+
+We may distinguish several directories, the first level has 5 directories:
+
+* **api**: This directory is dedicated to the API (framework) documentations.
* **commons**: This directory is dedicated for storage of traffic
profile or any other test inputs that could be reused by any test
project.
* **docker**: This directory includes the needed files and tools to
- build the Funtest Docker image.
+ build the Functest Docker image.
* **docs**: This directory includes documentation: Release Notes,
- User Guide, Configuration Guide and Developer Guide. Test results
- are also located in a sub--directory called 'results'.
+ User Guide, Configuration Guide and Developer Guide.
* **functest**: This directory contains all the code needed to run
functest internal cases and OPNFV onboarded feature or VNF test cases.
-Functest directory has 6 directories:
+Functest directory has 7 sub-directories:
+ * **api**: This directory is dedicated for the internal Functest API and the
+ API (framework) documentations.
* **ci**: This directory contains test structure definition files
(e.g <filename>.yaml) and bash shell/python scripts used to
configure and execute Functional tests. The test execution script
own test code. See for an example the Openstack helper utility:
'openstack_utils.py'.
-Useful Docker commands
-----------------------
+
+Logs
+====
+By default all the logs are put un /home/opnfv/functest/results/functest.log.
+If you want to have more logs in console, you may edit the logging.ini file
+manually.
+Connect on the docker then edit the file located in
+/usr/lib/python2.7/site-packages/functest/ci/logging.ini
+
+Change wconsole to console in the desired module to get more traces.
+
+
+Configuration
+=============
+
+You may also directly modify the python code or the configuration file (e.g.
+testcases.yaml used to declare test constraints) under
+/usr/lib/python2.7/site-packages/functest
+
+
+Tips
+====
+
+Docker
+------
When typing **exit** in the container prompt, this will cause exiting
the container and probably stopping it. When stopping a running Docker
container all the changes will be lost, there is a keyboard shortcut
docker rm -f <CONTAINER_ID>
-Check the Docker documentation dockerdocs_ for more information.
+Check the Docker documentation [`dockerdocs`_] for more information.
-Preparing the Functest environment
-----------------------------------
-Once the Functest docker container is up and running, the required
-Functest environment needs to be prepared. A custom built **functest**
-CLI utility is available to perform the needed environment preparation
-action. Once the environment is prepared, the **functest** CLI utility
-can be used to run different functional tests. The usage of the
-**functest** CLI utility to run tests is described further in the
-Functest User Guide `OPNFV_FuncTestUserGuide`_
-
-Prior to commencing the Functest environment preparation, we can check
-the initial status of the environment. Issue the **functest env status**
-command at the prompt::
-
- functest env status
- Functest environment is not installed.
-
- Note: When the Functest environment is prepared, the command will
- return the status: "Functest environment ready to run tests."
-
-To prepare the Functest docker container for test case execution, issue
-the **functest env prepare** command at the prompt::
-
- functest env prepare
-
-This script will make sure that the requirements to run the tests are
-met and will install the needed libraries and tools by all Functest
-test cases. It should be run only once every time the Functest docker
-container is started from scratch. If you try to run this command, on
-an already prepared enviroment, you will be prompted whether you really
-want to continue or not::
-
- functest env prepare
- It seems that the environment has been already prepared.
- Do you want to do it again? [y|n]
-
- (Type 'n' to abort the request, or 'y' to repeat the
- environment preparation)
-
-
-To list some basic information about an already prepared Functest
-docker container environment, issue the **functest env show** at the
-prompt::
-
- functest env show
- +======================================================+
- | Functest Environment info |
- +======================================================+
- | INSTALLER: apex, 192.168.122.89 |
- | SCENARIO: os-odl_l2-nofeature-ha |
- | POD: localhost |
- | GIT BRANCH: master |
- | GIT HASH: 5bf1647dec6860464eeb082b2875798f0759aa91 |
- | DEBUG FLAG: false |
- +------------------------------------------------------+
- | STATUS: ready |
- +------------------------------------------------------+
-
- Where:
-
- INSTALLER: Displays the INSTALLER_TYPE value
- - here = "apex"
- and the INSTALLER_IP value
- - here = "192.168.122.89"
- SCENARIO: Displays the DEPLOY_SCENARIO value
- - here = "os-odl_l2-nofeature-ha"
- POD: Displays the value passed in NODE_NAME
- - here = "localhost"
- GIT BRANCH: Displays the git branch of the OPNFV Functest
- project repository included in the Functest
- Docker Container.
- - here = "master"
- (In first official colorado release
- would be "colorado.1.0")
- GIT HASH: Displays the git hash of the OPNFV Functest
- project repository included in the Functest
- Docker Container.
- - here = "5bf1647dec6860464eeb082b2875798f0759aa91"
- DEBUG FLAG: Displays the CI_DEBUG value
- - here = "false"
-
- NOTE: In Jenkins CI runs, an additional item "BUILD TAG"
- would also be listed. The valaue is set by Jenkins CI.
-
-Finally, the **functest** CLI has a **--help** options:
-
-Some examples::
-
- functest --help Usage: functest [OPTIONS] COMMAND [ARGS]...
-
- Options:
- --version Show the version and exit.
- -h, --help Show this message and exit.
-
- Commands:
- env
- openstack
- testcase
- tier
-
- functest env --help
- Usage: functest env [OPTIONS] COMMAND [ARGS]...
-
- Options:
- -h, --help Show this message and exit.
-
- Commands:
- prepare Prepares the Functest environment.
- show Shows information about the current...
- status Checks if the Functest environment is ready...
Checking Openstack and credentials
----------------------------------
*OS_*, for example::
OS_REGION_NAME=RegionOne
- OS_DEFAULT_DOMAIN=default
+ OS_USER_DOMAIN_NAME=Default
OS_PROJECT_NAME=admin
- OS_PASSWORD=admin
- OS_AUTH_STRATEGY=keystone
- OS_AUTH_URL=http://172.30.10.3:5000/v2.0
+ OS_AUTH_VERSION=3
+ OS_IDENTITY_API_VERSION=3
+ OS_PASSWORD=da54c27ae0d10dfae5297e6f0d6be54ebdb9f58d0f9dfc
+ OS_AUTH_URL=http://10.1.0.9:5000/v3
OS_USERNAME=admin
OS_TENANT_NAME=admin
OS_ENDPOINT_TYPE=internalURL
- OS_NO_CACHE=true
+ OS_INTERFACE=internalURL
+ OS_NO_CACHE=1
+ OS_PROJECT_DOMAIN_NAME=Default
+
If the OpenStack command still does not show anything or complains
about connectivity issues, it could be due to an incorrect url given to
the OS_AUTH_URL environment variable. Check the deployment settings.
-SSL Support
------------
-If you need to connect to a server that is TLS-enabled (the auth URL
-begins with "https") and it uses a certificate from a private CA or a
-self-signed certificate, then you will need to specify the path to an
-appropriate CA certificate to use, to validate the server certificate
-with the environment variable OS_CACERT::
-
- echo $OS_CACERT
- /etc/ssl/certs/ca.crt
-
-However, this certificate does not exist in the container by default.
-It has to be copied manually from the OpenStack deployment. This can be
-done in 2 ways:
-
- #. Create manually that file and copy the contents from the OpenStack
- controller.
- #. (Recommended) Add the file using a Docker volume when starting the
- container::
-
- -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
-
-You might need to export OS_CACERT environment variable inside the
-container::
-
- export OS_CACERT=/etc/ssl/certs/ca.crt
-
-Certificate verification can be turned off using OS_INSECURE=true. For
-example, Fuel uses self-signed cacerts by default, so an pre step would
-be::
-
- export OS_INSECURE=true
-
Proxy support
-------------
If your Jumphost node is operating behind a http proxy, then there are
in section: `Docker Installation on CentOS behind http proxy`_
below.
- #. Execution of the Functest environment preparation inside the
- created docker container Functest needs internet access to
- download some resources for some test cases. This might not
- work properly if the Jumphost is connecting to internet
- through a http Proxy.
-
If that is the case, make sure the resolv.conf and the needed
http_proxy and https_proxy environment variables, as well as the
'no_proxy' environment variable are set correctly::
git config --global http.sslVerify True
git config --global http.proxy <Your http proxy settings>
-Validation check: Before running **'functest env prepare'** CLI command,
-make sure you can reach http and https sites from inside the Functest
-docker container.
-
For example, try to use the **nc** command from inside the functest
docker container::
commands might not work. You can use the **curl** command instead.
curl http://www.opnfv.org:80
+
<HTML><HEAD><meta http-equiv="content-type"
.
.
</BODY></HTML>
curl https://www.opnfv.org:443
+
<HTML><HEAD><meta http-equiv="content-type"
.
.
EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes)
DropInPaths=/etc/systemd/system/docker.service.d/env.conf
-Now follow the instructions in [`InstallDockerCentOS`_] to download
+Now follow the instructions in [`Install Docker on CentOS`_] to download
and install the **docker-engine**. The instructions conclude with a
"test pull" of a sample "Hello World" docker container. This should now
work with the above pre-requisite actions.
-.. _dockerdocs: https://docs.docker.com/
-.. _dockerhub: https://hub.docker.com/r/opnfv/functest/
-.. _Proxy: https://docs.docker.com/engine/admin/systemd/#http-proxy
-.. _FunctestDockerTags: https://hub.docker.com/r/opnfv/functest/tags/
-.. _InstallDockerCentOS: https://docs.docker.com/engine/installation/linux/centos/
-.. _OPNFV_FuncTestUserGuide: http://artifacts.opnfv.org/functest/docs/userguide/index.html
+
+.. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
+.. _`dockerdocs`: https://docs.docker.com/
+.. _`dockerhub`: https://hub.docker.com/r/opnfv/functest/
+.. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy
+.. _`FunctestDockerTags`: https://hub.docker.com/r/opnfv/functest/tags/
+.. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
+.. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
+.. _`images/CentOS-7-x86_64-GenericCloud.qcow2` http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
+.. _`images/cirros-0.3.5-x86_64-disk.img` https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img
+.. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img` https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
Code Review / functest.git / blobdiff