Alpine containers have been introduced in Euphrates.
Alpine allows Functest testing in several very light containers and thanks to
the refactoring on dependency management should allow the creation of light and
-fully customized docker files.
+fully customized docker images.
-It is still possible to use the monolithic functest opnfv/functest especially
-for tests on Aarch64 architecture.
Functest Dockers
----------------
-Docker files are available on the dockerhub:
+Docker images are available on the dockerhub:
* opnfv/functest-core
* opnfv/functest-healthcheck
* opnfv/functest-parser
* opnfv/functest-restapi
-By default, we use the docker tag latest, but you may pull a tagged docker
-image.
+The tag "opnfv-5.0.0" is the official release image in Euphrates, but you can also pull "euphrates"
+tag as it is being maintained by Functest team and might include bugfixes.
The Functest docker container environment can -in principle- be also
used with non-OPNFV official installers (e.g. 'devstack'), with the
See section on environment variables for details.
-cat openstack.creds::
+cat env_file::
export OS_AUTH_URL=XXX
export OS_USER_DOMAIN_NAME=XXX
See section on OpenStack credentials for details.
-Create a directory for the different images (included as volume)::
+Create a directory for the different images (attached as a Docker volume)::
- mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/*
+ mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/euphrates | bash -s -- images && ls -1 images/*
images/CentOS-7-aarch64-GenericCloud.qcow2
images/CentOS-7-aarch64-GenericCloud.qcow2.xz
images/CentOS-7-x86_64-GenericCloud.qcow2
- images/cirros-0.3.5-x86_64-disk.img
- images/cirros-0.3.5-x86_64-lxc.tar.gz
+ images/cirros-0.4.0-x86_64-disk.img
+ images/cirros-0.4.0-x86_64-lxc.tar.gz
images/cirros-d161201-aarch64-disk.img
images/cirros-d161201-aarch64-initramfs
images/cirros-d161201-aarch64-kernel
images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
images/vyos-1.1.7.img
-
Testing healthcheck suite
--------------------------
Run healthcheck suite::
sudo docker run --env-file env \
- -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
- -v $(pwd)/images:/home/opnfv/functest/images \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
+ -v $(pwd)/images:/home/opnfv/functest/images \
opnfv/functest-healthcheck
Results shall be displayed as follows::
+----------------------------+------------------+---------------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+----------------------------+------------------+---------------------+------------------+----------------+
- | connection_check | functest | healthcheck | 00:02 | PASS |
- | api_check | functest | healthcheck | 04:57 | PASS |
- | snaps_health_check | functest | healthcheck | 00:51 | PASS |
+ | connection_check | functest | healthcheck | 00:06 | PASS |
+ | api_check | functest | healthcheck | 06:52 | PASS |
+ | snaps_health_check | functest | healthcheck | 00:42 | PASS |
+----------------------------+------------------+---------------------+------------------+----------------+
+ NOTE: the duration is a reference and it might vary depending on your SUT.
Testing smoke suite
-------------------
Run smoke suite::
sudo docker run --env-file env \
- -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
- -v $(pwd)/images:/home/opnfv/functest/images \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
+ -v $(pwd)/images:/home/opnfv/functest/images \
opnfv/functest-smoke
Results shall be displayed as follows::
+------------------------------+------------------+---------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+------------------------------+------------------+---------------+------------------+----------------+
- | vping_ssh | functest | smoke | 01:19 | PASS |
- | vping_userdata | functest | smoke | 01:56 | PASS |
- | tempest_smoke_serial | functest | smoke | 26:30 | PASS |
- | rally_sanity | functest | smoke | 19:42 | PASS |
- | refstack_defcore | functest | smoke | 22:00 | PASS |
- | snaps_smoke | functest | smoke | 41:14 | PASS |
- | odl | functest | smoke | 00:16 | PASS |
+ | vping_ssh | functest | smoke | 00:45 | PASS |
+ | vping_userdata | functest | smoke | 00:36 | PASS |
+ | tempest_smoke_serial | functest | smoke | 14:04 | PASS |
+ | rally_sanity | functest | smoke | 23:59 | PASS |
+ | refstack_defcore | functest | smoke | 14:37 | FAIL |
+ | snaps_smoke | functest | smoke | 42:09 | PASS |
+ | odl | functest | smoke | 00:00 | SKIP |
| odl_netvirt | functest | smoke | 00:00 | SKIP |
- | fds | functest | smoke | 00:00 | SKIP |
+------------------------------+------------------+---------------+------------------+----------------+
Note: if the scenario does not support some tests, they are indicated as SKIP.
See User guide for details.
Run features suite::
sudo docker run --env-file env \
- -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
- -v $(pwd)/images:/home/opnfv/functest/images \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
+ -v $(pwd)/images:/home/opnfv/functest/images \
opnfv/functest-features
Results shall be displayed as follows::
- +---------------------------+--------------------------+------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +---------------------------+--------------------------+------------------+------------------+----------------+
- | promise | promise | features | 00:00 | SKIP |
- | bgpvpn | sdnvpn | features | 00:00 | SKIP |
- | security_scan | securityscanning | features | 00:00 | SKIP |
- | functest-odl-sfc | sfc | features | 00:00 | SKIP |
- | domino-multinode | domino | features | 00:00 | SKIP |
- | barometercollectd | barometer | features | 00:00 | SKIP |
- +---------------------------+--------------------------+------------------+------------------+----------------+
+ +-----------------------------+------------------------+------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +-----------------------------+------------------------+------------------+------------------+----------------+
+ | doctor-notification | doctor | features | 00:00 | SKIP |
+ | bgpvpn | sdnvpn | features | 00:00 | SKIP |
+ | functest-odl-sfc | sfc | features | 00:00 | SKIP |
+ | barometercollectd | barometer | features | 00:00 | SKIP |
+ | fds | fastdatastacks | features | 00:00 | SKIP |
+ +-----------------------------+------------------------+------------------+------------------+----------------+
Note: if the scenario does not support some tests, they are indicated as SKIP.
See User guide for details.
Run components suite::
sudo docker run --env-file env \
- -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
- -v $(pwd)/images:/home/opnfv/functest/images \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
+ -v $(pwd)/images:/home/opnfv/functest/images \
opnfv/functest-components
Results shall be displayed as follows::
+-------------------------------+------------------+--------------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+-------------------------------+------------------+--------------------+------------------+----------------+
- | tempest_full_parallel | functest | components | 102:48 | PASS |
- | rally_full | functest | components | 160:58 | PASS |
- | tempest_custom | functest | components | 00:00 | SKIP |
+ | tempest_full_parallel | functest | components | 52:42 | PASS |
+ | rally_full | functest | components | 114:22 | PASS |
+-------------------------------+------------------+--------------------+------------------+----------------+
Testing vnf suite
Run vnf suite::
sudo docker run --env-file env \
- -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
- -v $(pwd)/images:/home/opnfv/functest/images \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
+ -v $(pwd)/images:/home/opnfv/functest/images \
opnfv/functest-vnf
Results shall be displayed as follows::
- +---------------------------------+------------------+--------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +---------------------------------+------------------+--------------+------------------+----------------+
- | cloudify_ims | functest | vnf | 21:25 | PASS |
- | orchestra_openims | functest | vnf | 11:02 | FAIL |
- | orchestra_clearwaterims | functest | vnf | 09:13 | FAIL |
- | vyos_vrouter | functest | vnf | 00:00 | SKIP |
- +---------------------------------+------------------+--------------+------------------+----------------+
+ +---------------------------------+-------------------+--------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +---------------------------------+-------------------+--------------+------------------+----------------+
+ | cloudify_ims | functest | vnf | 28:49 | FAIL |
+ | vyos_vrouter | functest | vnf | 27:57 | FAIL |
+ | juju_epc | functest | vnf | 55:03 | PASS |
+ | orchestra_openims | orchestra | vnf | 00:00 | SKIP |
+ | orchestra_clearwaterims | orchestra | vnf | 00:00 | SKIP |
+ +---------------------------------+-------------------+--------------+------------------+----------------+
Environment variables
=====================
Several environement variables may be specified:
- * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid|osa)
+ * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid)
* INSTALLER_IP=<Specific IP Address>
* DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
* ha_mode (high availability) is one of ( ha | noha )
If several features are pertinent then use the underscore character '_' to
-separate each feature (e.g. ovs_kvm) 'nofeature' indicates no OPNFV feature is
-deployed
+separate each feature (e.g. ovs_kvm). 'nofeature' indicates that no OPNFV
+feature is deployed.
The list of supported scenarios per release/installer is indicated in the
release note.
**NOTE:** The scenario name is mainly used to automatically detect
-if a test suite is runnable or not (e.g. it will prevent ONOS test suite to be
-run on ODL scenarios). If not set, Functest will try to run the default test
-cases that might not include SDN controller or a specific
-feature
+if a test suite is runnable or not (e.g. it will prevent ODL test suite to be
+run on 'nosdn' scenarios). If not set, Functest will try to run the default test
+cases that might not include SDN controller or a specific feature
-**NOTE:** A HA scenario means that 3 OpenStack controller nodes are
+**NOTE:** An HA scenario means that 3 OpenStack controller nodes are
deployed. It does not necessarily mean that the whole system is HA. See
installer release notes for details.
* NODE_NAME = <Test POD Name>
* BUILD_TAG = <Jenkins Build Tag>
- * CI_DEBUG = <DebugTraceValue>
where:
to the database. This option is only set when
tests are activated under Jenkins CI control.
It enables the correlation of test results,
- which
- are independently pushed to the results database
+ which are independently pushed to the results database
from different Jenkins jobs.
DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS.
- * <DebugTraceValue> = "true" or "false"
- Default = "false", if not specified
- If "true" is specified, then additional debug trace
- text can be sent to the test results file / log files
- and also to the standard console output.
Openstack credentials
OpenStack credentials are mandatory and must be provided to Functest.
When running the command "functest env prepare", the framework will
automatically look for the Openstack credentials file
-"/home/opnfv/functest/conf/openstack.creds" and will exit with
+"/home/opnfv/functest/conf/env_file" and will exit with
error if it is not present or is empty.
There are 2 ways to provide that file:
* by using a Docker volume with -v option when creating the Docker container.
This is referred to in docker documentation as "Bind Mounting".
See the usage of this parameter in the following chapter.
- * or creating manually the file '/home/opnfv/functest/conf/openstack.creds'
+ * or creating manually the file '/home/opnfv/functest/conf/env_file'
inside the running container and pasting the credentials in it. Consult
your installer guide for further details. This is however not
instructed in this document.
-There is a default environment variable in the Functest container **$creds**
-that points to the credentials absolute path to help the user with this task.
-
In proxified environment you may need to change the credentials file.
There are some tips in chapter: `Proxy support`_
-v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert
You might need to export OS_CACERT environment variable inside the
-container::
+credentials file::
export OS_CACERT=/etc/ssl/certs/ca.crt
includes two main directories:
* The **conf** directory stores configuration files (e.g. the
- OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'),
+ OpenStack creds are stored in path '/home/opnfv/functest/conf/env_file'),
* the **results** directory stores some temporary result log files
src and repos directories are used to host third party code used for the tests.
-The functest code is under /usr/lib/python2.7/site-packages/functest
-The structure can be described as follows::
+The structure of functest repo can be described as follows::
|-- INFO
|-- LICENSE
| `-- apidoc
|-- build.sh
|-- commons
- | |-- docker
- | |-- Dockerfile
- | |-- Dockerfile.aarch64.patch
+ |-- docker
| |-- components
- | |-- config_install_env.sh
| |-- core
- | |-- docker_remote_api
| |-- features
| |-- healthcheck
| |-- smoke
| |-- vnf
- | `-- thirdparty-requirements.txt
+ | |-- parser
+ | |-- restapi
+ | |-- thirdparty-requirements.txt
|-- docs
| |-- com
| |-- images
| |-- urls.py
| |-- common
| | |-- api_utils.py
- | | `-- error.py
+ | | |-- thread.py
| `-- resources
| `-- v1
| |-- creds.py
| |-- envs.py
| |-- testcases.py
- | `-- tiers.py
+ | |-- tiers.py
+ | |-- tasks.py
+ | `-- database
+ | |-- db.py
+ | `-- v1
+ | |-- handlers.py
+ | |-- models.py
+ | `-- swagger
|-- ci
- │ |-- check_deployment.py
- │ |-- config_aarch64_patch.yaml
- │ |-- config_functest.yaml
- │ |-- config_patch.yaml
- │ |-- download_images.sh
- │ |-- installer_params.yaml
- │ |-- logging.ini
- │ |-- prepare_env.py
- │ |-- rally_aarch64_patch.conf
- │ |-- run_tests.py
- │ |-- testcases.yaml
- │ |-- tier_builder.py
- │ `-- tier_handler.py
+ │ |-- check_deployment.py
+ │ |-- config_aarch64_patch.yaml
+ │ |-- config_functest.yaml
+ │ |-- config_patch.yaml
+ │ |-- download_images.sh
+ │ |-- logging.ini
+ │ |-- rally_aarch64_patch.conf
+ │ |-- run_tests.py
+ │ |-- testcases.yaml
+ │ |-- tier_builder.py
+ │ |-- tier_handler.py
|-- cli
- │ |-- cli_base.py
- │ |-- commands
- │ │ |-- cli_env.py
- │ │ |-- cli_os.py
- │ │ |-- cli_testcase.py
- │ │ `-- cli_tier.py
- │ |-- functest-complete.sh
+ │ |-- cli_base.py
+ │ |-- commands
+ │ │ |-- cli_env.py
+ │ │ |-- cli_os.py
+ │ │ |-- cli_testcase.py
+ │ │ |-- cli_tier.py
+ │ |-- functest-complete.sh
|-- core
- │ |-- feature.py
- │ |-- testcase.py
- │ |-- unit.py
- │ `-- vnf.py
+ │ |-- feature.py
+ │ |-- robotframework.py
+ │ |-- testcase.py
+ │ |-- unit.py
+ │ |-- vnf.py
|-- energy
- │ |-- energy.py
- │ `-- energy.pyc
+ │ |-- energy.py
|-- opnfv_tests
- │ |-- mano
- │ │ |-- orchestra.py
- │ |-- openstack
- │ │ |-- rally
- │ │ |-- refstack_client
- │ │ |-- snaps
- │ │ |-- tempest
- │ │ `-- vping
- │ |-- sdn
- │ │ `-- odl
- │ `-- vnf
- │ |-- aaa
- │ |-- ims
- │ `-- router
+ │ `-- openstack
+ │ |-- rally
+ │ |-- refstack_client
+ │ |-- snaps
+ │ |-- tempest
+ │ |-- vping
+ │ `-- sdn
+ │ │ `-- odl
+ │ `-- vnf
+ │ |-- ims
+ │ `-- router
|-- tests
- │ `-- unit
- │ |-- ci
- │ |-- cli
- │ |-- core
- │ |-- energy
- │ |-- features
- │ |-- odl
- │ |-- openstack
- │ |-- opnfv_tests
- │ |-- test_utils.py
- │ |-- utils
- │ `-- vnf
+ │ `-- unit
+ │ |-- ci
+ │ |-- cli
+ │ |-- core
+ │ |-- energy
+ │ |-- features
+ │ |-- odl
+ │ |-- openstack
+ │ |-- opnfv_tests
+ │ |-- test_utils.py
+ │ |-- utils
+ │ `-- vnf
|-- utils
| |-- config.py
| |-- constants.py
| |-- functest_utils.py
| |-- functest_vacation.py
| |-- openstack_clean.py
- | |-- openstack_snapshot.py
| |-- openstack_tacker.py
| `-- openstack_utils.py
|-- requirements.txt
We may distinguish several directories, the first level has 5 directories:
* **api**: This directory is dedicated to the API (framework) documentations.
-* **commons**: This directory is dedicated for storage of traffic
- profile or any other test inputs that could be reused by any test
- project.
+* **commons**: This directory is dedicated for storage of traffic profile or
+ any other test inputs that could be reused by any test project.
* **docker**: This directory includes the needed files and tools to
- build the Functest Docker image.
+ build the Functest Docker images.
* **docs**: This directory includes documentation: Release Notes,
User Guide, Configuration Guide and Developer Guide.
* **functest**: This directory contains all the code needed to run
functest internal cases and OPNFV onboarded feature or VNF test cases.
-Functest directory has 7 sub-directories:
- * **api**: This directory is dedicated for the internal Functest API and the
- API (framework) documentations.
+Functest directory has 7 sub-directories, which is located under
+/usr/lib/python2.7/site-packages/functest:
+ * **api**: This directory is dedicated for the internal Functest API.
* **ci**: This directory contains test structure definition files
(e.g <filename>.yaml) and bash shell/python scripts used to
configure and execute Functional tests. The test execution script
the integration of internal, feature or vnf cases.
* **opnfv_tests**: This directory includes the scripts required by
Functest internal test cases and other feature projects test cases.
- * **tests**: This directory includes the functest unit tests
+ * **tests**: This directory includes the functest unit tests.
* **utils**: this directory holds Python source code for some general
purpose helper utilities, which testers can also re-use in their
own test code. See for an example the Openstack helper utility:
Once the credentials are there inside the container, they should be
sourced before running any Openstack commands::
- source /home/opnfv/functest/conf/openstack.creds
+ source /home/opnfv/functest/conf/env_file
After this, try to run any OpenStack command to see if you get any
output, for instance::
'no_proxy' environment variable are set correctly::
# Make double sure that the 'no_proxy=...' line in the
- # 'openstack.creds' file is commented out first. Otherwise, the
+ # 'env_file' file is commented out first. Otherwise, the
# values set into the 'no_proxy' environment variable below will
# be ovewrwritten, each time the command
- # 'source ~/functest/conf/openstack.creds' is issued.
+ # 'source ~/functest/conf/env_file' is issued.
cd ~/functest/conf/
- sed -i 's/export no_proxy/#export no_proxy/' openstack.creds
- source ./openstack.creds
+ sed -i 's/export no_proxy/#export no_proxy/' env_file
+ source ./env_file
# Next calculate some IP addresses for which http_proxy
# usage should be excluded:
.. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
.. _`dockerdocs`: https://docs.docker.com/
-.. _`dockerhub`: https://hub.docker.com/r/opnfv/functest/
.. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy
-.. _`FunctestDockerTags`: https://hub.docker.com/r/opnfv/functest/tags/
.. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
.. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
-.. _`images/CentOS-7-x86_64-GenericCloud.qcow2` http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
-.. _`images/cirros-0.3.5-x86_64-disk.img` https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img
-.. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img` https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
+.. _`images/CentOS-7-x86_64-GenericCloud.qcow2`: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
+.. _`images/cirros-0.4.0-x86_64-disk.img`: http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img
+.. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img`: https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img
Code Review / functest.git / blobdiff