1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others.
7 Convention for heading levels in Yardstick documentation:
9 ======= Heading 0 (reserved for the title in a document)
15 Avoid deeper levels because they do not render well.
17 ======================
18 Yardstick Installation
19 ======================
22 Yardstick supports installation by Docker or directly in Ubuntu. The
23 installation procedure for Docker and direct installation are detailed in
26 To use Yardstick you should have access to an OpenStack environment, with at
27 least Nova, Neutron, Glance, Keystone and Heat installed.
29 The steps needed to run Yardstick are:
32 2. Load OpenStack environment variables.
33 3. Create Yardstick flavor.
34 4. Build a guest image and load it into the OpenStack environment.
35 5. Create the test configuration ``.yaml`` file and run the test case/suite.
41 The OPNFV deployment is out of the scope of this document and can be found in
42 `User Guide & Configuration Guide`_. The OPNFV platform is considered as the
43 System Under Test (SUT) in this document.
45 Several prerequisites are needed for Yardstick:
47 1. A Jumphost to run Yardstick on
48 2. A Docker daemon or a virtual environment installed on the Jumphost
49 3. A public/external network created on the SUT
50 4. Connectivity from the Jumphost to the SUT public/external network
52 .. note:: *Jumphost* refers to any server which meets the previous
53 requirements. Normally it is the same server from where the OPNFV
54 deployment has been triggered.
56 .. warning:: Connectivity from Jumphost is essential and it is of paramount
57 importance to make sure it is working before even considering to install
58 and run Yardstick. Make also sure you understand how your networking is
61 .. note:: If your Jumphost is operating behind a company http proxy and/or
62 Firewall, please first consult `Proxy Support`_ section which is towards
63 the end of this document. That section details some tips/tricks which *may*
64 be of help in a proxified environment.
67 Install Yardstick using Docker (first option) (**recommended**)
68 ---------------------------------------------------------------
70 Yardstick has a Docker image. It is recommended to use this Docker image to run
73 Prepare the Yardstick container
74 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
76 Install docker on your guest system with the following command, if not done
79 wget -qO- https://get.docker.com/ | sh
81 Pull the Yardstick Docker image (``opnfv/yardstick``) from the public dockerhub
82 registry under the OPNFV account in dockerhub_, with the following docker
85 sudo -EH docker pull opnfv/yardstick:stable
87 After pulling the Docker image, check that it is available with the
88 following docker command::
90 [yardsticker@jumphost ~]$ docker images
91 REPOSITORY TAG IMAGE ID CREATED SIZE
92 opnfv/yardstick stable a4501714757a 1 day ago 915.4 MB
94 Run the Docker image to get a Yardstick container::
96 docker run -itd --privileged -v /var/run/docker.sock:/var/run/docker.sock \
97 -p 8888:5000 --name yardstick opnfv/yardstick:stable
99 Description of the parameters used with ``docker run`` command
101 +------------------------+--------------------------------------------------+
102 | Parameters | Detail |
103 +========================+==================================================+
104 | -itd | -i: interactive, Keep STDIN open even if not |
106 | +--------------------------------------------------+
107 | | -t: allocate a pseudo-TTY detached mode, in the |
109 +------------------------+--------------------------------------------------+
110 | --privileged | If you want to build ``yardstick-image`` in |
111 | | Yardstick container, this parameter is needed |
112 +------------------------+--------------------------------------------------+
113 | -p 8888:5000 | Redirect the a host port (8888) to a container |
115 +------------------------+--------------------------------------------------+
116 | -v /var/run/docker.sock| If you want to use yardstick env |
117 | :/var/run/docker.sock | grafana/influxdb to create a grafana/influxdb |
118 | | container out of Yardstick container |
119 +------------------------+--------------------------------------------------+
120 | --name yardstick | The name for this container |
121 +------------------------+--------------------------------------------------+
124 If the host is restarted
125 ^^^^^^^^^^^^^^^^^^^^^^^^
127 The yardstick container must be started if the host is rebooted::
129 docker start yardstick
131 Configure the Yardstick container environment
132 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
134 There are three ways to configure environments for running Yardstick, explained
135 in the following sections. Before that, access the Yardstick container::
137 docker exec -it yardstick /bin/bash
139 and then configure Yardstick environments in the Yardstick container.
141 Using the CLI command ``env prepare`` (first way) (**recommended**)
142 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
144 In the Yardstick container, the Yardstick repository is located in the
145 ``/home/opnfv/repos`` directory. Yardstick provides a CLI to prepare OpenStack
146 environment variables and create Yardstick flavor and guest images
149 yardstick env prepare
151 .. note:: Since Euphrates release, the above command will not be able to
152 automatically configure the ``/etc/yardstick/openstack.creds`` file. So before
153 running the above command, it is necessary to create the
154 ``/etc/yardstick/openstack.creds`` file and save OpenStack environment
155 variables into it manually. If you have the openstack credential file saved
156 outside the Yardstick Docker container, you can do this easily by mapping the
157 credential file into Yardstick container using::
159 '-v /path/to/credential_file:/etc/yardstick/openstack.creds'
161 when running the Yardstick container. For details of the required OpenStack
162 environment variables please refer to section `Export OpenStack environment
165 The ``env prepare`` command may take up to 6-8 minutes to finish building
166 yardstick-image and other environment preparation. Meanwhile if you wish to
167 monitor the env prepare process, you can enter the Yardstick container in a new
168 terminal window and execute the following command::
170 tail -f /var/log/yardstick/uwsgi.log
173 Manually exporting the env variables and initializing OpenStack (second way)
174 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
176 Export OpenStack environment variables
177 ######################################
179 Before running Yardstick it is necessary to export OpenStack environment
184 Environment variables in the ``openrc`` file have to include at least::
192 A sample ``openrc`` file may look like this::
194 export OS_PASSWORD=console
195 export OS_PROJECT_NAME=admin
196 export OS_AUTH_URL=http://172.16.1.222:35357/v2.0
197 export OS_USERNAME=admin
198 export OS_VOLUME_API_VERSION=2
199 export EXTERNAL_NETWORK=net04_ext
202 Manual creation of Yardstick flavor and guest images
203 ####################################################
205 Before executing Yardstick test cases, make sure that Yardstick flavor and
206 guest image are available in OpenStack. Detailed steps about creating the
207 Yardstick flavor and building the Yardstick guest image can be found below.
209 Most of the sample test cases in Yardstick are using an OpenStack flavor called
210 ``yardstick-flavor`` which deviates from the OpenStack standard ``m1.tiny``
211 flavor by the disk size; instead of 1GB it has 3GB. Other parameters are the
212 same as in ``m1.tiny``.
214 Create ``yardstick-flavor``::
216 openstack flavor create --disk 3 --vcpus 1 --ram 512 --swap 100 \
219 Most of the sample test cases in Yardstick are using a guest image called
220 ``yardstick-image`` which deviates from an Ubuntu Cloud Server image
221 containing all the required tools to run test cases supported by Yardstick.
222 Yardstick has a tool for building this custom image. It is necessary to have
223 ``sudo`` rights to use this tool.
225 Also you may need install several additional packages to use this tool, by
226 follwing the commands below::
228 sudo -EH apt-get update && sudo -EH apt-get install -y qemu-utils kpartx
230 This image can be built using the following command in the directory where
231 Yardstick is installed::
233 export YARD_IMG_ARCH='amd64'
234 echo "Defaults env_keep += \'YARD_IMG_ARCH\'" | sudo tee --append \
235 /etc/sudoers > /dev/null
236 sudo -EH tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh
238 .. warning:: Before building the guest image inside the Yardstick container,
239 make sure the container is granted with privilege. The script will create files
240 by default in ``/tmp/workspace/yardstick`` and the files will be owned by root.
242 The created image can be added to OpenStack using the OpenStack client or via
243 the OpenStack Dashboard::
245 openstack image create --disk-format qcow2 --container-format bare \
246 --public --file /tmp/workspace/yardstick/yardstick-image.img \
250 Some Yardstick test cases use a `Cirros 0.3.5`_ image and/or a `Ubuntu 16.04`_
251 image. Add Cirros and Ubuntu images to OpenStack::
253 openstack image create --disk-format qcow2 --container-format bare \
254 --public --file $cirros_image_file cirros-0.3.5
255 openstack image create --disk-format qcow2 --container-format bare \
256 --file $ubuntu_image_file Ubuntu-16.04
259 Automatic initialization of OpenStack (third way)
260 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
262 Similar to the second way, the first step is also to
263 `Export OpenStack environment variables`_. Then the following steps should be
266 Automatic creation of Yardstick flavor and guest images
267 #######################################################
269 Yardstick has a script for automatically creating Yardstick flavor and building
270 Yardstick guest images. This script is mainly used for CI and can be also used
271 in the local environment::
273 source $YARDSTICK_REPO_DIR/tests/ci/load_images.sh
276 The Yardstick container GUI
277 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
279 In Euphrates release, Yardstick implemented a GUI for Yardstick Docker
280 container. After booting up Yardstick container, you can visit the GUI at
281 ``<container_host_ip>:8888/gui/index.html``.
283 For usage of Yardstick GUI, please watch our demo video at
284 `Yardstick GUI demo`_.
286 .. note:: The Yardstick GUI is still in development, the GUI layout and
289 Delete the Yardstick container
290 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
292 If you want to uninstall Yardstick, just delete the Yardstick container::
294 sudo docker stop yardstick && docker rm yardstick
298 Install Yardstick directly in Ubuntu (second option)
299 ----------------------------------------------------
301 .. _install-framework:
303 Alternatively you can install Yardstick framework directly in Ubuntu or in an
304 Ubuntu Docker image. No matter which way you choose to install Yardstick, the
305 following installation steps are identical.
307 If you choose to use the Ubuntu Docker image, you can pull the Ubuntu
308 Docker image from Docker hub::
310 sudo -EH docker pull ubuntu:16.04
316 Prerequisite preparation::
318 sudo -EH apt-get update && sudo -EH apt-get install -y \
319 git python-setuptools python-pip
320 sudo -EH easy_install -U setuptools==30.0.0
321 sudo -EH pip install appdirs==1.4.0
322 sudo -EH pip install virtualenv
324 Download the source code and install Yardstick from it::
326 git clone https://gerrit.opnfv.org/gerrit/yardstick
327 export YARDSTICK_REPO_DIR=~/yardstick
329 sudo -EH ./install.sh
331 If the host is ever restarted, nginx and uwsgi need to be restarted::
333 service nginx restart
334 uwsgi -i /etc/yardstick/yardstick.ini
336 Configure the Yardstick environment (**Todo**)
337 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
339 For installing Yardstick directly in Ubuntu, the ``yardstick env`` command is
340 not available. You need to prepare OpenStack environment variables and create
341 Yardstick flavor and guest images manually.
347 For uninstalling Yardstick, just delete the virtual environment::
349 rm -rf ~/yardstick_venv
352 Install Yardstick directly in OpenSUSE
353 --------------------------------------
355 .. _install-framework:
357 You can install Yardstick framework directly in OpenSUSE.
363 Prerequisite preparation::
365 sudo -EH zypper -n install -y gcc \
378 python-setuptools-git
380 Create a virtual environment::
382 virtualenv ~/yardstick_venv
383 export YARDSTICK_VENV=~/yardstick_venv
384 source ~/yardstick_venv/bin/activate
385 sudo -EH easy_install -U setuptools
387 Download the source code and install Yardstick from it::
389 git clone https://gerrit.opnfv.org/gerrit/yardstick
390 export YARDSTICK_REPO_DIR=~/yardstick
392 sudo -EH python setup.py install
393 sudo -EH pip install -r requirements.txt
395 Install missing python modules::
397 sudo -EH pip install pyyaml \
404 python.glanceclient \
405 python.neutronclient \
410 Configure the Yardstick environment
411 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
413 Source the OpenStack environment variables::
415 source DEVSTACK_DIRECTORY/openrc
417 Export the Openstack external network. The default installation of Devstack
418 names the external network public::
420 export EXTERNAL_NETWORK=public
421 export OS_USERNAME=demo
423 Change the API version used by Yardstick to v2.0 (the devstack openrc sets it
426 export OS_AUTH_URL=http://PUBLIC_IP_ADDRESS:5000/v2.0
432 For unistalling Yardstick, just delete the virtual environment::
434 rm -rf ~/yardstick_venv
437 Verify the installation
438 -----------------------
440 It is recommended to verify that Yardstick was installed successfully
441 by executing some simple commands and test samples. Before executing Yardstick
442 test cases make sure ``yardstick-flavor`` and ``yardstick-image`` can be found
443 in OpenStack and the ``openrc`` file is sourced. Below is an example invocation
444 of Yardstick ``help`` command and ``ping.py`` test sample::
447 yardstick task start samples/ping.yaml
449 .. note:: The above commands could be run in both the Yardstick container and
452 Each testing tool supported by Yardstick has a sample configuration file.
453 These configuration files can be found in the ``samples`` directory.
455 Default location for the output is ``/tmp/yardstick.out``.
458 Automatic installation of Yardstick using ansible
459 -------------------------------------------------
461 Automatic installation can be used as an alternative to the manual.
462 Yardstick can be installed on the bare metal and to the container. Yardstick
463 container can be either pulled or built.
465 Bare metal installation
466 ^^^^^^^^^^^^^^^^^^^^^^^
468 Use ansible script ``install.yaml`` to install Yardstick on Ubuntu server:
470 .. code-block:: console
472 ansible-playbook -i install-inventory.ini install.yaml \
473 -e YARDSTICK_DIR=<path to Yardstick folder>
475 .. note:: By default ``INSTALLATION_MODE`` is ``baremetal``.
477 .. note:: By default Ubuntu 16.04 is chosen (xenial). It can be changed to
478 Ubuntu 18.04 (bionic) by passing ``-e OS_RELEASE=bionic`` parameter.
480 .. note:: To install Yardstick in virtual environment pass parameter
481 ``-e VIRTUAL_ENVIRONMENT=True``.
483 To build Yardstick NSB image pass ``IMG_PROPERTY=nsb`` as input parameter:
485 .. code-block:: console
487 ansible-playbook -i install-inventory.ini install.yaml \
488 -e IMAGE_PROPERTY=nsb \
489 -e YARDSTICK_DIR=<path to Yardstick folder>
491 .. note:: In this ``INSTALLATION_MODE`` mode either Yardstick image or SampleVNF
492 images will be built. Image type is defined by parameter ``IMAGE_PROPERTY``.
493 By default Yardstick image will be built.
495 Container installation
496 ^^^^^^^^^^^^^^^^^^^^^^
498 Use ansible script ``install.yaml`` to pull or build Yardstick
499 container. To pull Yardstick image and start container run:
501 .. code-block:: console
503 ansible-playbook -i install-inventory.ini install.yaml \
504 -e YARDSTICK_DIR=<path to Yardstick folder> \
505 -e INSTALLATION_MODE=container_pull
507 .. note:: In this ``INSTALLATION_MODE`` mode either Yardstick image or SampleVNF
508 images will be built. Image type is defined by variable ``IMG_PROPERTY`` in
509 file ``ansible/group_vars/all.yml``. By default Yardstick image will be
512 .. note:: Open question: How to know if Docker image is built on Ubuntu 16.04 and 18.04?
513 Do we need separate tag to be used?
515 To build Yardstick image run:
517 .. code-block:: console
519 ansible-playbook -i install-inventory.ini install.yaml \
520 -e YARDSTICK_DIR=<path to Yardstick folder> \
521 -e INSTALLATION_MODE=container
523 .. note:: In this ``INSTALLATION_MODE`` mode neither Yardstick image nor SampleVNF
526 .. note:: By default Ubuntu 16.04 is chosen (xenial). It can be changed to
527 Ubuntu 18.04 (bionic) by passing ``-e OS_RELEASE=bionic`` parameter.
529 Parameters for ``install.yaml``
530 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
532 Description of the parameters used with ``install.yaml`` script
534 +-------------------------+-------------------------------------------------+
535 | Parameters | Detail |
536 +=========================+=================================================+
537 | -i install-inventory.ini| Installs package dependency to remote servers |
538 | | Mandatory parameter |
539 | | By default no remote servers are provided |
540 | | Needed packages will be installed on localhost |
541 +-------------------------+-------------------------------------------------+
542 | -e YARDSTICK_DIR | Path to Yardstick folder |
543 | | Mandatory parameter |
544 +-------------------------+-------------------------------------------------+
545 | -e INSTALLATION_MODE | baremetal: Yardstick is installed to the bare |
547 | | Default parameter |
548 | +-------------------------------------------------+
549 | | container: Yardstick is installed in container |
550 | | Container is built from Dockerfile |
551 | +-------------------------------------------------+
552 | | container_pull: Yardstick is installed in |
554 | | Container is pulled from docker hub |
555 +-------------------------+-------------------------------------------------+
556 | -e OS_RELEASE | xenial or bionic: Ubuntu version to be used |
557 | | Default is Ubuntu 16.04 (xenial) |
558 +-------------------------+-------------------------------------------------+
559 | -e IMAGE_PROPERTY | normal or nsb: Type of the VM image to be built |
560 | | Default image is Yardstick |
561 +-------------------------+-------------------------------------------------+
562 | -e VIRTUAL_ENVIRONMENT | False or True: Whether install in virtualenv |
563 | | Default is False |
564 +-------------------------+-------------------------------------------------+
567 Deploy InfluxDB and Grafana using Docker
568 ----------------------------------------
570 Without InfluxDB, Yardstick stores results for running test case in the file
571 ``/tmp/yardstick.out``. However, it's inconvenient to retrieve and display
572 test results. So we will show how to use InfluxDB to store data and use
573 Grafana to display data in the following sections.
575 Automatic deployment of InfluxDB and Grafana containers (**recommended**)
576 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
578 1. Enter the Yardstick container::
580 sudo -EH docker exec -it yardstick /bin/bash
582 2. Create InfluxDB container and configure with the following command::
584 yardstick env influxdb
586 3. Create and configure Grafana container::
588 yardstick env grafana
590 Then you can run a test case and visit http://host_ip:1948
591 (``admin``/``admin``) to see the results.
593 .. note:: Executing ``yardstick env`` command to deploy InfluxDB and Grafana
594 requires Jumphost's docker API version => 1.24. Run the following command to
595 check the docker API version on the Jumphost::
600 Manual deployment of InfluxDB and Grafana containers
601 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
603 You can also deploy influxDB and Grafana containers manually on the Jumphost.
604 The following sections show how to do.
608 sudo -EH docker pull tutum/influxdb
609 sudo -EH docker pull grafana/grafana
613 sudo -EH docker run -d --name influxdb \
614 -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 \
616 docker exec -it influxdb influx
620 > CREATE USER root WITH PASSWORD 'root' WITH ALL PRIVILEGES
621 > CREATE DATABASE yardstick;
628 sudo -EH docker run -d --name grafana -p 1948:3000 grafana/grafana
630 Log on to ``http://{YOUR_IP_HERE}:1948`` using ``admin``/``admin`` and configure
631 database resource to be ``{YOUR_IP_HERE}:8086``.
633 .. image:: images/Grafana_config.png
635 :alt: Grafana data source configuration
637 Configure ``yardstick.conf``::
639 sudo -EH docker exec -it yardstick /bin/bash
640 sudo cp etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf
641 sudo vi /etc/yardstick/yardstick.conf
643 Modify ``yardstick.conf`` to add the ``influxdb`` dispatcher::
647 dispatcher = influxdb
649 [dispatcher_influxdb]
651 target = http://{YOUR_IP_HERE}:8086
656 Now Yardstick will store results in InfluxDB when you run a testcase.
659 Deploy InfluxDB and Grafana directly in Ubuntu (**Todo**)
660 ---------------------------------------------------------
666 To configure the Jumphost to access Internet through a proxy its necessary to
667 export several variables to the environment, contained in the following
671 _proxy=<proxy_address>
672 _proxyport=<proxy_port>
673 _ip=$(hostname -I | awk '{print $1}')
675 export ftp_proxy=http://$_proxy:$_proxyport
676 export FTP_PROXY=http://$_proxy:$_proxyport
677 export http_proxy=http://$_proxy:$_proxyport
678 export HTTP_PROXY=http://$_proxy:$_proxyport
679 export https_proxy=http://$_proxy:$_proxyport
680 export HTTPS_PROXY=http://$_proxy:$_proxyport
681 export no_proxy=127.0.0.1,localhost,$_ip,$(hostname),<.localdomain>
682 export NO_PROXY=127.0.0.1,localhost,$_ip,$(hostname),<.localdomain>
684 To enable Internet access from a container using ``docker``, depends on the OS
685 version. On Ubuntu 14.04 LTS, which uses SysVinit, ``/etc/default/docker`` must
689 # If you need Docker to use an HTTP proxy, it can also be specified here.
690 export http_proxy="http://<proxy_address>:<proxy_port>/"
691 export https_proxy="https://<proxy_address>:<proxy_port>/"
693 Then its necessary to restart the ``docker`` service::
695 sudo -EH service docker restart
697 In Ubuntu 16.04 LTS, which uses Systemd, its necessary to create a drop-in
700 sudo mkdir /etc/systemd/system/docker.service.d
702 Then, the proxy configuration will be stored in the following file::
704 # cat /etc/systemd/system/docker.service.d/http-proxy.conf
706 Environment="HTTP_PROXY=https://<proxy_address>:<proxy_port>/"
707 Environment="HTTPS_PROXY=https://<proxy_address>:<proxy_port>/"
708 Environment="NO_PROXY=localhost,127.0.0.1,<localaddress>,<.localdomain>"
710 The changes need to be flushed and the ``docker`` service restarted::
712 sudo systemctl daemon-reload
713 sudo systemctl restart docker
715 Any container is already created won't contain these modifications. If needed,
716 stop and delete the container::
718 sudo docker stop yardstick
719 sudo docker rm yardstick
721 .. warning:: Be careful, the above ``rm`` command will delete the container
722 completely. Everything on this container will be lost.
724 Then follow the previous instructions `Prepare the Yardstick container`_ to
725 rebuild the Yardstick container.
731 .. _`User Guide & Configuration Guide`: http://docs.opnfv.org/en/latest/release/userguide.introduction.html
732 .. _dockerhub: https://hub.docker.com/r/opnfv/yardstick/
733 .. _`Cirros 0.3.5`: http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
734 .. _`Ubuntu 16.04`: https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img
735 .. _`Yardstick GUI demo`: https://www.youtube.com/watch?v=M3qbJDp6QBk