-===============================================
-OPNFV FUNCTEST configuration/installation guide
-===============================================
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+==================================
+OPNFV FUNCTEST Configuration Guide
+==================================
.. toctree::
:numbered:
:maxdepth: 2
+Version history
+===============
+
++------------+----------+------------------+----------------------------------+
+| **Date** | **Ver.** | **Author** | **Comment** |
+| | | | |
++------------+----------+------------------+----------------------------------+
+| 2016-08-17 | 1.0.0 | Juha Haapavirta | Colorado release |
+| | | Column Gaynor | |
++------------+----------+------------------+----------------------------------+
+| 2017-01-19 | 1.0.1 | Morgan Richomme | Adaptations for Danube |
+| | | | * update testcase list |
+| | | | * update docker command |
++------------+----------+------------------+----------------------------------+
+
Introduction
============
+This document describes how to install and configure Functest in OPNFV.
+The Functest CLI is utilized during the Functest environment preparation
+step. The given example commands should work in both virtual and bare
+metal cases alike.
-** DOCUMENT IS IN PROGRESS FOR BRAHMAPUTRA **
+High level architecture
+-----------------------
+
+The high level architecture of Functest within OPNFV can be described as
+follows::
+
+ CIMC/Lights+out management Admin Mgmt/API Public Storage Private
+ PXE
+ + + + + + +
+ | | | | | |
+ | +----------------------------+ | | | | |
+ | | | | | | | |
+ +-----+ Jumphost | | | | | |
+ | | +--------+ | | | |
+ | | | | | | | |
+ | | +--------------------+ | | | | | |
+ | | | | | | | | | |
+ | | | Tools | +----------------+ | | |
+ | | | - Rally | | | | | | |
+ | | | - Robot | | | | | | |
+ | | | - TestON | | | | | | |
+ | | | | |-------------------------+ | |
+ | | | Testcases | | | | | | |
+ | | | - VIM | | | | | | |
+ | | | -- healthcheck | | | | | | |
+ | | | -- vPing_ssh | | | | | | |
+ | | | -- vPing_userdata | | | | | | |
+ | | | -- SNAPS_cases | | | | | | |
+ | | | -- Tempest_smoke | | | | | | |
+ | | | -- Rally_sanity | | | | | | |
+ | | | -- Tempest_full | | | | | | |
+ | | | -- Rally_full | | | | | | |
+ | | | | | | | | | |
+ | | | - SDN Controller | | | | | | |
+ | | | -- odl | | | | | | |
+ | | | -- onos | | | | | | |
+ | | | | | | | | | |
+ | | | - Features | | | | | | |
+ | | | | | | | | | |
+ | | | - VNF | | | | | | |
+ | | | | | | | | | |
+ | | +--------------------+ | | | | | |
+ | | Functest Docker + | | | | |
+ | | | | | | | |
+ | | | | | | | |
+ | | | | | | | |
+ | +----------------------------+ | | | | |
+ | | | | | |
+ | +----------------+ | | | | |
+ | | 1 | | | | | |
+ +----+ +--------------+-+ | | | | |
+ | | | 2 | | | | | |
+ | | | +--------------+-+ | | | | |
+ | | | | 3 | | | | | |
+ | | | | +--------------+-+ | | | | |
+ | | | | | 4 | | | | | |
+ | +-+ | | +--------------+-+ | | | | |
+ | | | | | 5 +-------------+ | | | |
+ | +-+ | | nodes for | | | | | |
+ | | | | deploying +---------------------+ | | |
+ | +-+ | OPNFV | | | | | |
+ | | | +------------------------------+ | |
+ | +-+ SUT | | | | | |
+ | | +--------------------------------------+ |
+ | | | | | | | |
+ | | +----------------------------------------------+
+ | +----------------+ | | | | |
+ | | | | | |
+ + + + + + +
+ SUT = System Under Test
+
+All the libraries and dependencies needed by all of the Functest tools
+are pre-installed into the Docker image. This allows running Functest
+on any platform on any Operating System.
+
+The automated mechanisms inside the Functest Docker container will:
+
+ * Retrieve OpenStack credentials
+ * Prepare the environment according to the SUT
+ * Perform the appropriate functional tests
+ * Push the test results into the OPNFV test result database
+
+This Docker image can be integrated into CI or deployed independently.
+
+Please note that the Functest Docker container has been designed for
+OPNFV, however, it would be possible to adapt it to any VIM + controller
+environment, since most of the test cases are integrated from upstream
+communities.
+
+The functional test cases are described in the Functest User Guide `[2]`_
-This document describes how to install and configure Functest in OPNFV.
Prerequisites
=============
-
-The OPNFV deployment is out of the scope of this document but it can be found in
-XXX. The OPNFV platform is considered as the System Under Test (SUT) in this
+The OPNFV deployment is out of the scope of this document but it can be
+found in http://docs.opnfv.org.
+The OPNFV platform is considered as the System Under Test (SUT) in this
document.
-Several prerequisites are needed for functest:
+Several prerequisites are needed for Functest:
#. A Jumphost to run Functest on
- #. Docker daemon shall be installed on the Jumphost
+ #. A Docker daemon shall be installed on the Jumphost
#. A public/external network created on the SUT
+ #. An admin/management network created on the SUT
#. Connectivity from the Jumphost to the SUT public/external network
- #. Connectivity from the Jumphost to the SUT management network
+ #. Connectivity from the Jumphost to the SUT admin/management network
+
+WARNING: Connectivity from Jumphost is essential and it is of paramount
+importance to make sure it is working before even considering to install
+and run Functest. Make also sure you understand how your networking is
+designed to work.
-NOTE: “Jumphost” refers to any server which meets the previous requirements.
-Normally it is the same server from where the OPNFV deployment has been
-triggered.
+NOTE: **Jumphost** refers to any server which meets the previous
+requirements. Normally it is the same server from where the OPNFV
+deployment has been triggered previously.
+
+NOTE: If your Jumphost is operating behind a company http proxy and/or
+Firewall, please consult first the section `Proxy Support`_, towards
+the end of this document. The section details some tips/tricks which
+*may* be of help in a proxified environment.
Docker installation
-------------------
+Docker installation and configuration is only needed to be done once
+through the life cycle of Jumphost.
+
+If your Jumphost is based on Ubuntu, RHEL or CentOS linux, please
+consult the references below for more detailed instructions. The
+commands below are offered as a short reference.
-.. _Ubuntu: https://docs.docker.com/installation/ubuntulinux/
-.. _RHEL: https://docs.docker.com/installation/rhel/
+*Tip:* For running docker containers behind the proxy, you need first
+some extra configuration which is described in section
+`Docker Installation on CentOS behind http proxy`_. You should follow
+that section before installing the docker engine.
-Log on your jumphost then install docker (e.g. for Ubuntu)::
+Docker installation needs to be done as root user. You may use other
+userid's to create and run the actual containers later if so desired.
+Log on to your Jumphost as root user and install the Docker Engine
+(e.g. for CentOS family)::
curl -sSL https://get.docker.com/ | sh
+ systemctl start docker
+
+ *Tip:* If you are working through proxy, please set the https_proxy
+ environment variable first before executing the curl command.
Add your user to docker group to be able to run commands without sudo::
sudo usermod -aG docker <your_user>
+A reconnect is needed. There are 2 ways for this:
+ #. Re-login to your account
+ #. su - <username>
-References:
-
+References - Installing Docker Engine on different Linux Operating Systems:
* Ubuntu_
* RHEL_
-
-Connectivity to OPNFV management network
-----------------------------------------
-
-Some of the Functest tools need to have access to the OpenStack management
-network of the controllers `[1]`_.
-
-For this reason, an interface shall be configured in the Jumphost in the
-OpenStack management network range.
-
-Example::
-
- The OPNFV Fuel installation uses VLAN tagged 300 and subnet 192.168.1.0/24 as
- Openstack Management network.
- .
- Supposing that eth1 is the physical interface with access to that subnet:
- $ ip link add name eth1.300 link eth1 type vlan id 300
- $ ip link set eth1.300 up
- $ ip addr add 192.168.1.66/24 dev eth1.300
-
-
-External network on SUT
------------------------
-
-Some of the tests against the VIM (Virtual Infrastructure Manager) need an
-existing public network to succeed. This is needed, for example, to create
-floating IPs to access instances from the public network (i.e. Jumphost).
-
-By default, any of the four OPNFV installers provide a fresh installation with
-an external network created along with a router.
-
-
-
-High level architecture
-=======================
-
-The high level architecture of Functest within OPNFV can be described as follow::
-
- CIMC/Lights+out management Admin Private Public Storage
- PXE
- +
- + + IP_PRIV/24 | |
- | | + + |
- | | | IP_PUB/24 |
- | +----------------------------+ | | + |
- | | | | | | |
- +-----+ Jumphost | | | | |
- | | +----------+ | | |
- | | | | | | |
- | | +--------------------+ | | | | |
- | | | | | | | | |
- | | | Tools | +--------------------+ | |
- | | | - Rally | | | | | |
- | | | - Robot | | | | | |
- | | | - TestON | | | | | |
- | | | | | | | | |
- | | | Testcases | | | | | |
- | | | - VIM | | | | | |
- | | | -- vPing | | | | | |
- | | | -- vPing_userdata | | | | | |
- | | | -- Tempest | | | | | |
- | | | -- Rally | | | | | |
- | | | - Controller | | | | | |
- | | | -- odl | | | | | |
- | | | -- onos | | | | | |
- | | | -- opencontrail | | | | | |
- | | | | | | | | |
- | | | Features | | | | | |
- | | | - vIMS | | | | | |
- | | | | | | | | |
- | | +--------------------+ | | | | |
- | | +------------------------------+ |
- | | Functest Docker | | | | |
- | | +---------------------------------------+
- | | | | | | |
- | +----------------------------+ | | | |
- | | | | |
- | +----------------+ | | | |
- | | 1 | | | | |
- +----+ +--------------+-+ | | | |
- | | | 2 | | | | |
- | | | +--------------+-+ | | | |
- | | | | 3 | | | | |
- | | | | +--------------+-+ | | | |
- | | | | | 4 | | | | |
- | +-+ | | +--------------+-+ | | | |
- | | | | | 5 +---------------+ | | |
- | +-+ | | nodes for | | | | |
- | | | | deploying +-------------------------+ | |
- | +-+ | opnfv | | | | |
- | | | SUT +-----------------------------------+ |
- | +-+ | | | | |
- | | +--------------------------------------------+
- | +----------------+ | | | |
- | | | | |
- | + + + +
-
-All the libraries and dependencies needed by all the Functest tools are
-pre-installed in the Docker image.
-This allows running Functest on any platform with any Operating System.
-
-The Docker image will:
-
- * retrieve OpenStack credentials
- * prepare the environment according to the SUT
- * perform the appropriate tests
- * push the results into the OPNFV test result database
-
-
-
-This Docker image can be integrated into CI or deployed **independently** of the CI.
-A description of the Brahmaputra testcases can be retrieved in the Functest user
-guide `[2]`_
-
-Please note that the Functest container has been designed for OPNFV, however, it
-would be possible to adapt it to any VIM+controller environment as most of the
-testcases are upstream testcases.
-
-Manual Installation
-===================
-
-Pull the Functest Docker image from the Docker hub::
-
- $ docker pull opnfv/functest:brahmaputra.1.0
-
-
-Check that the image is available::
-
- $ docker images
-
-Run the docker container giving the environment variables::
-
- - INSTALLER_TYPE. Possible values are "apex", "compass", "fuel" or "joid".
- - INSTALLER_IP. each installer has its installation strategy.
-
-Functest may need to know the IP of the installer to retrieve the credentials
-(e.g. usually "10.20.0.2" for fuel, not neede for joid...).
-
-The minimum command to create the Functest docker file can be described as
-follows::
-
- docker run -it -e "INSTALLER_IP=10.20.0.2" -e "INSTALLER_TYPE=fuel" opnfv/functest:brahmaputra.1.0 /bin/bash
-
-Optionally, it is possible to precise the container name through the option
---name::
-
- docker run --name "CONTAINER_NAME" -it -e "INSTALLER_IP=10.20.0.2" -e "INSTALLER_TYPE=fuel" opnfv/functest:brahmaputra.1.0 /bin/bash
-
-It is also possible to to indicate the path of the OpenStack creds using -v::
-
- docker run -it -e "INSTALLER_IP=10.20.0.2" -e "INSTALLER_TYPE=fuel" -v <path_to_your_local_creds_file>:/home/opnfv/functest/conf/openstack.creds opnfv/functest:brahmaputra.1.0 /bin/bash
-
-The local file will be mounted in the container under
-/home/opnfv/functest/conf/openstack.creds
-
-After the run command the prompt appears which means that we are inside the
-container and ready to run Functest.
-
-Inside the container, you must have the following directory structure::
-
- `-- home
- `-- opnfv
- |-- functest
- | |-- conf
- | |-- data
- | `-- results
- `-- repos
- |-- bgpvpn
- |-- functest
- |-- odl_integration
- |-- rally
- |-- releng
- `-- vims-test
-
-
-Basically the container includes:
-
- * Functest directory to store the configuration (the OpenStack creds are paste
- in /home/opngb/functest/conf), the data (images neede for test for offline
- testing), results (some temporary artifacts may be stored here)
- * Repositories: the functest repository will be used to prepare the
- environment, run the tests. Other repositories are used for the installation
- of the tooling (e.g. rally) and/or the retrieval of feature projects
- scenarios (e.g. bgpvpn)
-
-The arborescence under the functest repo can be described as follow::
-
- .
- |-- INFO
- |-- LICENSE
- |-- commons
- | |-- ims
- | |-- mobile
- | `-- traffic-profile-guidelines.rst
- |-- docker
- | |-- Dockerfile
- | |-- common.sh
- | |-- prepare_env.sh
- | |-- requirements.pip
- | `-- run_tests.sh
- |-- docs
- | |-- configguide
- | |-- functest.rst
- | |-- images
- | `-- userguide
- `-- testcases
- |-- Controllers
- |-- VIM
- |-- __init__.py
- |-- config_functest.py
- |-- config_functest.yaml
- |-- functest_utils.py
- |-- functest_utils.pyc
- |-- vIMS
- `-- vPing
-
-We may distinguish 4 different folders:
-
- * commons: it is a folder dedicated to store traffic profile or any test
- inputs that could be reused by any test project
- * docker: this folder includes the scripts that will be used to setup the
- environment and run the tests
- * docs: this folder includes the user and installation/configuration guide
- * testcases: this folder includes the scripts required by Functest internal
- test cases
-
-
-Firstly run the script to install functest environment::
-
- $ ${repos_dir}/functest/docker/prepare_env.sh
-
-NOTE: ${repos_dir} is a default environment variable inside the docker
-container, which points to /home/opnfv/repos
-
-Run the script to start the tests::
-
- $ ${repos_dir}/functest/docker/run_tests.sh
-
-NOTE: This will run ALL the tests by default, see `[2]`_ for details
-
-Focus on the OpenStack credentials
-----------------------------------
-
-The OpenStack credentials are needed to test the VIM. There are 3 ways to
-provide them to Functest:
-
- * using the -v option when running the Docker container
- * create an empty file in /home/opnfv/functest/conf/openstack.creds and paste
- the needed info in it.
- * automatically retrieved using the following script::
- $repos_dir/releng/utils/fetch_os_creds.sh
-
-Once the credentials are there, they shall be sourced before running the tests::
-
- source /home/opnfv/functest/conf/openstack.creds
-
-Additional Options
-------------------
-
-In case you need to provide different configuration parameters to Functest (e.g.
- commit IDs or branches for the repositories, ...) copy the config_functest.yaml
- from the repository to your current directory and run docker with a volume::
-
- $ wget https://git.opnfv.org/cgit/functest/plain/testcases/config_functest.yaml
- $ cmd1 = "/home/opnfv/repos/functest/docker/prepare_env.sh"
- $ cmd2 = "/home/opnfv/repos/functest/docker/run_tests.sh"
- $ docker run -t -e "INSTALLER_TYPE=fuel" -e "INSTALLER_IP=10.20.0.2" opnfv/functest \
- -v $(pwd)/config_functest.yaml:/home/opnfv/functest/conf/config_functest.yaml \
- "${cmd1} && ${cmd2}"
-
+ * CentOS_
+
+.. _Ubuntu: https://docs.docker.com/engine/installation/linux/ubuntulinux/
+.. _RHEL: https://docs.docker.com/engine/installation/linux/rhel/
+.. _CentOS: https://docs.docker.com/engine/installation/linux/centos/
+
+Public/External network on SUT
+------------------------------
+Some of the tests against the VIM (Virtual Infrastructure Manager) need
+connectivity through an existing public/external network in order to
+succeed. This is needed, for example, to create floating IPs to access
+VM instances through the public/external network (i.e. from the Docker
+container).
+
+By default, the four OPNFV installers provide a fresh installation with
+a public/external network created along with a router. Make sure that
+the public/external subnet is reachable from the Jumphost.
+
+*Hint:* For the given OPNFV Installer in use, the IP sub-net address
+used for the public/external network is usually a planning item and
+should thus be known. Consult the OPNFV Configuration guide `[4]`_, and
+ensure you can reach each node in the SUT, from the Jumphost using the
+'ping' command using the respective IP address on the public/external
+network for each node in the SUT. The details of how to determine the
+needed IP addresses for each node in the SUT may vary according to the
+used installer and are therefore ommitted here.
+
+Connectivity to Admin/Management network on SUT
+-----------------------------------------------
+Some of the Functest tools need to have access to the OpenStack
+admin/management network of the controllers `[1]`_.
+
+For this reason, check the connectivity from the Jumphost to all the
+controllers in cluster in the OpenStack admin/management network range.
+
+Installation and configuration
+==============================
+
+.. include:: ./configguide.rst
Integration in CI
=================
-
-In CI we use the docker file and execute commande within the container from
-Jenkins.
+In CI we use the Docker image and execute the appropriate commands within the
+container from Jenkins.
Docker creation in set-functest-env builder `[3]`_::
- envs="INSTALLER_TYPE=${INSTALLER_TYPE} -e INSTALLER_IP=${INSTALLER_IP} -e NODE_NAME=${NODE_NAME}"
+ envs="-e INSTALLER_TYPE=${INSTALLER_TYPE} -e INSTALLER_IP=${INSTALLER_IP} -e NODE_NAME=${NODE_NAME}"
[...]
- docker pull opnfv/functest:latest_stable
- cmd="docker run -id -e $envs ${labconfig} ${sshkey} ${res_volume} opnfv/functest:latest_stable /bin/bash"
+ docker pull opnfv/functest:$DOCKER_TAG >/dev/null
+ cmd="sudo docker run -id ${envs} ${volumes} ${custom_params} ${TESTCASE_OPTIONS} opnfv/functest:${DOCKER_TAG} /bin/bash"
echo "Functest: Running docker run command: ${cmd}"
- ${cmd}
- docker ps -a
+ ${cmd} >${redirect}
sleep 5
- container_id=$(docker ps | grep 'opnfv/functest:latest_stable' | awk '{print $1}' | head -1)
+ container_id=$(docker ps | grep "opnfv/functest:${DOCKER_TAG}" | awk '{print $1}' | head -1)
echo "Container ID=${container_id}"
if [ -z ${container_id} ]; then
echo "Cannot find opnfv/functest container ID ${container_id}. Please check if it is existing."
echo "Starting the container: docker start ${container_id}"
docker start ${container_id}
sleep 5
- docker ps
- if [ $(docker ps | grep 'opnfv/functest:latest_stable' | wc -l) == 0 ]; then
+ docker ps >${redirect}
+ if [ $(docker ps | grep "opnfv/functest:${DOCKER_TAG}" | wc -l) == 0 ]; then
echo "The container opnfv/functest with ID=${container_id} has not been properly started. Exiting..."
exit 1
fi
- cmd="${FUNCTEST_REPO_DIR}/docker/prepare_env.sh"
+
+ cmd="python ${FUNCTEST_REPO_DIR}/functest/ci/prepare_env.py start"
echo "Executing command inside the docker: ${cmd}"
docker exec ${container_id} ${cmd}
Test execution in functest-all builder `[3]`_::
- echo "Functest: run $FUNCTEST_SUITE_NAME"
- cmd="${FUNCTEST_REPO_DIR}/docker/run_tests.sh --test $FUNCTEST_SUITE_NAME ${flag}"
- container_id=$(docker ps -a | grep opnfv/functest | awk '{print $1}' | head -1)
- docker exec $container_id $cmd
-
-Docker clean in functest-cleanup builder `[3]`_::
-
- echo "Cleaning up docker containers/images..."
- # Remove previous running containers if exist
- if [[ ! -z $(docker ps -a | grep opnfv/functest) ]]; then
- echo "Removing existing opnfv/functest containers..."
- docker ps | grep opnfv/functest | awk '{print $1}' | xargs docker stop
- docker ps -a | grep opnfv/functest | awk '{print $1}' | xargs docker rm
- fi
-
- # Remove existing images if exist
- if [[ ! -z $(docker images | grep opnfv/functest) ]]; then
- echo "Docker images to remove:"
- docker images | head -1 && docker images | grep opnfv/functest
- image_tags=($(docker images | grep opnfv/functest | awk '{print $2}'))
- for tag in "${image_tags[@]}"; do
- echo "Removing docker image opnfv/functest:$tag..."
- docker rmi opnfv/functest:$tag
- done
+ branch=${GIT_BRANCH##*/}
+ echo "Functest: run $FUNCTEST_SUITE_NAME on branch ${branch}"
+ cmd="functest testcase run $FUNCTEST_SUITE_NAME"
fi
+ container_id=$(docker ps -a | grep opnfv/functest | awk '{print $1}' | head -1)
+ docker exec $container_id $cmd
+ ret_value=$?
+ exit $ret_value
-
-Configuration
-=============
-
-Everything is preconfigured in the docker file.
-It is however possible to customize the list of tests, see `[2]` for details.
-
-Errors
-======
-
-
-
+Docker clean in functest-cleanup builder `[3]`_ calling docker rm and docker rmi
References
==========
.. _`[1]`: https://ask.openstack.org/en/question/68144/keystone-unable-to-use-the-public-endpoint/
-.. _`[2]`: url functest user guide
+.. _`[2]`: http://artifacts.opnfv.org/functest/docs/userguide/index.html
.. _`[3]`: https://git.opnfv.org/cgit/releng/tree/jjb/functest/functest-ci-jobs.yml
+.. _`[4]`: http://artifacts.opnfv.org/functest/danube/docs/configguide/index.html
OPNFV main site: opnfvmain_.
OPNFV functional test page: opnfvfunctest_.
-IRC support channel: #opnfv-testperf
+IRC support channel: #opnfv-functest
.. _opnfvmain: http://www.opnfv.org
-.. _opnfvfunctest: https://wiki.opnfv.org/opnfv_functional_testing
-.. _`OpenRC`: http://docs.openstack.org/user-guide/common/cli_set_environment_variables_using_openstack_rc.html
-.. _`Rally installation procedure`: https://rally.readthedocs.org/en/latest/tutorial/step_0_installation.html
-.. _`config_test.py` : https://git.opnfv.org/cgit/functest/tree/testcases/config_functest.py
-.. _`config_functest.yaml` : https://git.opnfv.org/cgit/functest/tree/testcases/config_functest.yaml
+.. _opnfvfunctest: https://wiki.opnfv.org/functest
Code Review / functest.git / blobdiff