Abstract
--------
-Yardstick currently supports installation on Ubuntu 14.04 or by using a Docker
-image. Detailed steps about installing Yardstick using both of these options
-can be found below.
+Yardstick supports installation on Ubuntu 14.04 or via a Docker image. The
+installation procedure on Ubuntu 14.04 or via the docker image are detailed in
+the section below.
-To use Yardstick you should have access to an OpenStack environment,
-with at least Nova, Neutron, Glance, Keystone and Heat installed.
+To use Yardstick you should have access to an OpenStack environment, with at
+least Nova, Neutron, Glance, Keystone and Heat installed.
The steps needed to run Yardstick are:
-1. Install Yardstick and create the test configuration .yaml file.
-2. Build a guest image and load the image into the OpenStack environment.
-3. Create a Neutron external network and load OpenStack environment variables.
-4. Run the test case.
+1. Install Yardstick.
+2. Load OpenStack environment variables.
+3. Create a Neutron external network.
+4. Build Yardstick flavor and a guest image.
+5. Load the guest image into the OpenStack environment.
+6. Create the test configuration .yaml file.
+7. Run the test case.
+
+
+Prerequisites
+-------------
+
+The OPNFV deployment is out of the scope of this document but it can be
+found in http://artifacts.opnfv.org/opnfvdocs/colorado/docs/configguide/index.html.
+The OPNFV platform is considered as the System Under Test (SUT) in this
+document.
+
+Several prerequisites are needed for Yardstick:
+
+ #. A Jumphost to run Yardstick on
+ #. A Docker daemon shall be installed on the Jumphost
+ #. A public/external network created on the SUT
+ #. Connectivity from the Jumphost to the SUT public/external network
+
+WARNING: Connectivity from Jumphost is essential and it is of paramount
+importance to make sure it is working before even considering to install
+and run Yardstick. Make also sure you understand how your networking is
+designed to work.
+
+NOTE: **Jumphost** refers to any server which meets the previous
+requirements. Normally it is the same server from where the OPNFV
+deployment has been triggered previously.
+
+NOTE: If your Jumphost is operating behind a company http proxy and/or
+Firewall, please consult first the section `Proxy Support`_, towards
+the end of this document. The section details some tips/tricks which
+*may* be of help in a proxified environment.
Installing Yardstick on Ubuntu 14.04
.. _install-framework:
-Installing Yardstick framework
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Install dependencies:
-::
+You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu
+14.04 Docker image. No matter which way you choose to install Yardstick
+framework, the following installation steps are identical.
+
+If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu
+14.04 Docker image from Docker hub:
- sudo apt-get update && sudo apt-get install -y \
- wget \
- git \
- sshpass \
- qemu-utils \
- kpartx \
- libffi-dev \
- libssl-dev \
- python \
- python-dev \
- python-virtualenv \
- libxml2-dev \
- libxslt1-dev \
- python-setuptools
-
-Create a python virtual environment, source it and update setuptools:
::
- virtualenv ~/yardstick_venv
- source ~/yardstick_venv/bin/activate
- easy_install -U setuptools
+ docker pull ubuntu:14.04
+Installing Yardstick framework
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Download source code and install python dependencies:
+
::
git clone https://gerrit.opnfv.org/gerrit/yardstick
cd yardstick
- python setup.py install
+ ./install.sh
-There is also a YouTube video, showing the above steps:
-.. image:: http://img.youtube.com/vi/4S4izNolmR0/0.jpg
- :alt: http://www.youtube.com/watch?v=4S4izNolmR0
- :target: http://www.youtube.com/watch?v=4S4izNolmR0
+Installing Yardstick using Docker
+---------------------------------
-Installing extra tools
-^^^^^^^^^^^^^^^^^^^^^^
-yardstick-plot
-""""""""""""""
-Yardstick has an internal plotting tool ``yardstick-plot``, which can be installed
-using the following command:
-::
+Yardstick has a Docker image, this Docker image (**Yardstick-stable**)
+serves as a replacement for installing the Yardstick framework in a virtual
+environment (for example as done in :ref:`install-framework`).
+It is recommended to use this Docker image to run Yardstick test.
- sudo apt-get install -y g++ libfreetype6-dev libpng-dev pkg-config
- python setup.py develop easy_install yardstick[plot]
+Pulling the Yardstick Docker image
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-.. _guest-image:
+.. _dockerhub: https://hub.docker.com/r/opnfv/yardstick/
-Building a guest image
-^^^^^^^^^^^^^^^^^^^^^^
-Yardstick has a tool for building an Ubuntu Cloud Server image containing all
-the required tools to run test cases supported by Yardstick. It is necessary to
-have sudo rights to use this tool.
+Pull the Yardstick Docker image ('opnfv/yardstick') from the public dockerhub
+registry under the OPNFV account: [dockerhub_], with the following docker
+command::
-Also you may need install several additional packages to use this tool, by
-follwing the commands below:
-::
+ docker pull opnfv/yardstick:stable
- apt-get update && apt-get install -y \
- qemu-utils \
- kpartx
+After pulling the Docker image, check that it is available with the
+following docker command::
+
+ [yardsticker@jumphost ~]$ docker images
+ REPOSITORY TAG IMAGE ID CREATED SIZE
+ opnfv/yardstick stable a4501714757a 1 day ago 915.4 MB
+
+Run the Docker image:
-This image can be built using the following command while in the directory where
-Yardstick is installed (``~/yardstick`` if the framework is installed
-by following the commands above):
::
- sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh
+ docker run --privileged=true -it opnfv/yardstick:stable /bin/bash
-**Warning:** the script will create files by default in:
-``/tmp/workspace/yardstick`` and the files will be owned by root!
+In the container the Yardstick repository is located in the /home/opnfv/repos
+directory.
-The created image can be added to OpenStack using the ``glance image-create`` or
-via the OpenStack Dashboard.
-Example command:
-::
+OpenStack parameters and credentials
+------------------------------------
- glance --os-image-api-version 1 image-create \
- --name yardstick-trusty-server --is-public true \
- --disk-format qcow2 --container-format bare \
- --file /tmp/workspace/yardstick/yardstick-trusty-server.img
+Environment variables
+^^^^^^^^^^^^^^^^^^^^^
+Before running Yardstick it is necessary to export OpenStack environment variables
+from the OpenStack *openrc* file (using the ``source`` command) and export the
+external network name ``export EXTERNAL_NETWORK="external-network-name"``,
+the default name for the external network is ``net04_ext``.
+Credential environment variables in the *openrc* file have to include at least:
-Installing Yardstick using Docker
+* OS_AUTH_URL
+* OS_USERNAME
+* OS_PASSWORD
+* OS_TENANT_NAME
+
+A sample openrc file may look like this:
+
+* export OS_PASSWORD=console
+* export OS_TENANT_NAME=admin
+* export OS_AUTH_URL=http://172.16.1.222:35357/v2.0
+* export OS_USERNAME=admin
+* export OS_VOLUME_API_VERSION=2
+* export EXTERNAL_NETWORK=net04_ext
+
+
+Yardstick falvor and guest images
---------------------------------
-Yardstick has two Docker images, first one (**Yardstick-framework**) serves as a
-replacement for installing the Yardstick framework in a virtual environment (for
-example as done in :ref:`install-framework`), while the other image is mostly for
-CI purposes (**Yardstick-CI**).
+Before executing Yardstick test cases, make sure that yardstick guest image and
+yardstick flavor are available in OpenStack.
+Detailed steps about creating yardstick flavor and building yardstick-trusty-server
+image can be found below.
-Yardstick-framework image
-^^^^^^^^^^^^^^^^^^^^^^^^^
-Download the source code:
+Yardstick-flavor
+^^^^^^^^^^^^^^^^
+Most of the sample test cases in Yardstick are using an OpenStack flavor called
+*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the
+disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny.
+
+Create yardstick-flavor:
::
- git clone https://gerrit.opnfv.org/gerrit/yardstick
+ nova flavor-create yardstick-flavor 100 512 3 1
-Build the Docker image and tag it as *yardstick-framework*:
-::
+.. _guest-image:
- cd yardstick
- docker build -t yardstick-framework .
+Building a guest image
+^^^^^^^^^^^^^^^^^^^^^^
+Most of the sample test cases in Yardstick are using a guest image called
+*yardstick-trusty-server* which deviates from an Ubuntu Cloud Server image
+containing all the required tools to run test cases supported by Yardstick.
+Yardstick has a tool for building this custom image. It is necessary to have
+sudo rights to use this tool.
-Run the Docker instance:
+Also you may need install several additional packages to use this tool, by
+follwing the commands below:
::
- docker run --name yardstick_instance -i -t yardstick-framework
-
-To build a guest image for Yardstick, see :ref:`guest-image`.
+ apt-get update && apt-get install -y \
+ qemu-utils \
+ kpartx
-Yardstick-CI image
-^^^^^^^^^^^^^^^^^^
-Pull the Yardstick-CI Docker image from Docker hub:
+This image can be built using the following command while in the directory where
+Yardstick is installed (``~/yardstick`` if the framework is installed
+by following the commands above):
::
- docker pull opnfv/yardstick:$DOCKER_TAG
+ export YARD_IMG_ARCH="amd64"
+ sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers
+ sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh
-Where ``$DOCKER_TAG`` is latest for master branch, as for the release branches,
-this coincides with its release name, such as brahmaputra.1.0.
+**Warning:** the script will create files by default in:
+``/tmp/workspace/yardstick`` and the files will be owned by root!
-Run the Docker image:
+If you are building this guest image in inside a docker container make sure the
+container is granted with privilege.
+
+The created image can be added to OpenStack using the ``glance image-create`` or
+via the OpenStack Dashboard.
+
+Example command:
::
- docker run \
- --privileged=true \
- --rm \
- -t \
- -e "INSTALLER_TYPE=${INSTALLER_TYPE}" \
- -e "INSTALLER_IP=${INSTALLER_IP}" \
- opnfv/yardstick \
- exec_tests.sh ${YARDSTICK_DB_BACKEND} ${YARDSTICK_SUITE_NAME}
-
-Where ``${INSTALLER_TYPE}`` can be apex, compass, fuel or joid, ``${INSTALLER_IP}``
-is the installer master node IP address (i.e. 10.20.0.2 is default for fuel). ``${YARDSTICK_DB_BACKEND}``
-is the IP and port number of DB, ``${YARDSTICK_SUITE_NAME}`` is the test suite you want to run.
-For more details, please refer to the Jenkins job defined in Releng project, labconfig information
-and sshkey are required. See the link
-https://git.opnfv.org/cgit/releng/tree/jjb/yardstick/yardstick-ci-jobs.yml.
-
-Note: exec_tests.sh is used for executing test suite here, furthermore, if someone wants to execute the
-test suite manually, it can be used as long as the parameters are configured correct. Another script
-called run_tests.sh is used for unittest in Jenkins verify job, in local manaul environment,
-it is recommended to run before test suite execuation.
-
-Basic steps performed by the **Yardstick-CI** container:
-
-1. clone yardstick and releng repos
-2. setup OS credentials (releng scripts)
-3. install yardstick and dependencies
-4. build yardstick cloud image and upload it to glance
-5. upload cirros-0.3.3 cloud image to glance
-6. run yardstick test scenarios
-7. cleanup
+ glance --os-image-api-version 1 image-create \
+ --name yardstick-image --is-public true \
+ --disk-format qcow2 --container-format bare \
+ --file /tmp/workspace/yardstick/yardstick-image.img
+Some Yardstick test cases use a Cirros image, you can find one at
+http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-disk.img
-OpenStack parameters and credentials
-------------------------------------
-Yardstick-flavor
-^^^^^^^^^^^^^^^^
-Most of the sample test cases in Yardstick are using an OpenStack flavor called
-*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the
-disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny.
+Automatic flavor and image creation
+-----------------------------------
+Yardstick has a script for automatic creating yardstick flavor and building
+guest images. This script is mainly used in CI, but you can still use it in
+your local environment.
-Environment variables
-^^^^^^^^^^^^^^^^^^^^^
-Before running Yardstick it is necessary to export OpenStack environment variables
-from the OpenStack *openrc* file (using the ``source`` command) and export the
-external network name ``export EXTERNAL_NETWORK="external-network-name"``,
-the default name for the external network is ``net04_ext``.
+Example command:
-Credential environment variables in the *openrc* file have to include at least:
+::
+
+ export YARD_IMG_ARCH="amd64"
+ sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers
+ source $YARDSTICK_REPO_DIR/tests/ci/load_images.sh
-* OS_AUTH_URL
-* OS_USERNAME
-* OS_PASSWORD
-* OS_TENANT_NAME
Yardstick default key pair
^^^^^^^^^^^^^^^^^^^^^^^^^^
----------------------------------
It is recommended to verify that Yardstick was installed successfully
-by executing some simple commands and test samples. Below is an example invocation
-of yardstick help command and ping.py test sample:
+by executing some simple commands and test samples. Before executing yardstick
+test cases make sure yardstick flavor and building yardstick-trusty-server
+image can be found in glance and openrc file is sourced. Below is an example
+invocation of yardstick help command and ping.py test sample:
::
yardstick –h
Each testing tool supported by Yardstick has a sample configuration file.
These configuration files can be found in the **samples** directory.
-Example invocation of ``yardstick-plot`` tool:
-::
-
- yardstick-plot -i /tmp/yardstick.out -o /tmp/plots/
-
Default location for the output is ``/tmp/yardstick.out``.
-More info about the tool can be found by executing:
-::
-
- yardstick-plot -h
-
Deploy InfluxDB and Grafana locally
------------------------------------
.. pull docker images
Pull docker images
+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
::
Run influxdb
::
- docker run -d --name influxdb -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 tutum/influxdb
+ docker run -d --name influxdb \
+ -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 \
+ tutum/influxdb
docker exec -it influxdb bash
Config influxdb
log on using admin/admin and config database resource to be {YOUR_IP_HERE}:8086
.. image:: images/Grafana_config.png
+ :width: 800px
+ :alt: Grafana data source configration
Config yardstick conf
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
username = root
password = root
-Now you can run yardstick test case and store the results in influxdb
+Now you can run yardstick test cases and store the results in influxdb
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+
+Create a test suite for yardstick
+------------------------------------
+
+A test suite in yardstick is a yaml file which include one or more test cases.
+Yardstick is able to support running test suite task, so you can customize you
+own test suite and run it in one task.
+
+"tests/opnfv/test_suites" is where yardstick put ci test-suite. A typical test
+suite is like below:
+
+fuel_test_suite.yaml
+
+::
+
+ ---
+ # Fuel integration test task suite
+
+ schema: "yardstick:suite:0.1"
+
+ name: "fuel_test_suite"
+ test_cases_dir: "samples/"
+ test_cases:
+ -
+ file_name: ping.yaml
+ -
+ file_name: iperf3.yaml
+
+As you can see, there are two test cases in fuel_test_suite, the syntax is simple
+here, you must specify the schema and the name, then you just need to list the
+test cases in the tag "test_cases" and also mark their relative directory in the
+tag "test_cases_dir".
+
+Yardstick test suite also support constraints and task args for each test case.
+Here is another sample to show this, which is digested from one big test suite.
+
+os-nosdn-nofeature-ha.yaml
+
+::
+
+ ---
+
+ schema: "yardstick:suite:0.1"
+
+ name: "os-nosdn-nofeature-ha"
+ test_cases_dir: "tests/opnfv/test_cases/"
+ test_cases:
+ -
+ file_name: opnfv_yardstick_tc002.yaml
+ -
+ file_name: opnfv_yardstick_tc005.yaml
+ -
+ file_name: opnfv_yardstick_tc043.yaml
+ constraint:
+ installer: compass
+ pod: huawei-pod1
+ task_args:
+ huawei-pod1: '{"pod_info": "etc/yardstick/.../pod.yaml",
+ "host": "node4.LF","target": "node5.LF"}'
+
+As you can see in test case "opnfv_yardstick_tc043.yaml", there are two tags, "constraint" and
+"task_args". "constraint" is where you can specify which installer or pod it can be run in
+the ci environment. "task_args" is where you can specify the task arguments for each pod.
+
+All in all, to create a test suite in yardstick, you just need to create a suite yaml file
+and add test cases and constraint or task arguments if necessary.
+
Code Review / yardstick.git / blobdiff