1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others.
12 Yardstick supports installation on Ubuntu 14.04 or via a Docker image. The
13 installation procedure on Ubuntu 14.04 or via the docker image are detailed in
16 To use Yardstick you should have access to an OpenStack environment, with at
17 least Nova, Neutron, Glance, Keystone and Heat installed.
19 The steps needed to run Yardstick are:
22 2. Load OpenStack environment variables.
23 3. Create a Neutron external network.
24 4. Build Yardstick flavor and a guest image.
25 5. Load the guest image into the OpenStack environment.
26 6. Create the test configuration .yaml file.
33 The OPNFV deployment is out of the scope of this document but it can be
34 found in http://artifacts.opnfv.org/opnfvdocs/colorado/docs/configguide/index.html.
35 The OPNFV platform is considered as the System Under Test (SUT) in this
38 Several prerequisites are needed for Yardstick:
40 #. A Jumphost to run Yardstick on
41 #. A Docker daemon shall be installed on the Jumphost
42 #. A public/external network created on the SUT
43 #. Connectivity from the Jumphost to the SUT public/external network
45 WARNING: Connectivity from Jumphost is essential and it is of paramount
46 importance to make sure it is working before even considering to install
47 and run Yardstick. Make also sure you understand how your networking is
50 NOTE: **Jumphost** refers to any server which meets the previous
51 requirements. Normally it is the same server from where the OPNFV
52 deployment has been triggered previously.
54 NOTE: If your Jumphost is operating behind a company http proxy and/or
55 Firewall, please consult first the section `Proxy Support`_, towards
56 the end of this document. The section details some tips/tricks which
57 *may* be of help in a proxified environment.
60 Installing Yardstick on Ubuntu 14.04
61 ------------------------------------
63 .. _install-framework:
65 You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu
66 14.04 Docker image. No matter which way you choose to install Yardstick
67 framework, the following installation steps are identical.
69 If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu
70 14.04 Docker image from Docker hub:
74 docker pull ubuntu:14.04
76 Installing Yardstick framework
77 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
82 sudo apt-get update && sudo apt-get install -y \
97 Create a python virtual environment, source it and update setuptools:
101 virtualenv ~/yardstick_venv
102 source ~/yardstick_venv/bin/activate
103 easy_install -U setuptools
105 Download source code and install python dependencies:
109 git clone https://gerrit.opnfv.org/gerrit/yardstick
111 python setup.py install
113 A Youtube video tutorial on this installation mode is available
114 at: http://www.youtube.com/watch?v=4S4izNolmR0
116 .. image:: http://img.youtube.com/vi/4S4izNolmR0/0.jpg
117 :alt: http://www.youtube.com/watch?v=4S4izNolmR0
118 :target: http://www.youtube.com/watch?v=4S4izNolmR0
121 Installing Yardstick using Docker
122 ---------------------------------
124 Yardstick has a Docker image, this Docker image (**Yardstick-stable**)
125 serves as a replacement for installing the Yardstick framework in a virtual
126 environment (for example as done in :ref:`install-framework`).
127 It is recommended to use this Docker image to run Yardstick test.
129 Pulling the Yardstick Docker image
130 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
132 .. _dockerhub: https://hub.docker.com/r/opnfv/yardstick/
134 Pull the Yardstick Docker image ('opnfv/yardstick') from the public dockerhub
135 registry under the OPNFV account: [dockerhub_], with the following docker
138 docker pull opnfv/yardstick:stable
140 After pulling the Docker image, check that it is available with the
141 following docker command::
143 [yardsticker@jumphost ~]$ docker images
144 REPOSITORY TAG IMAGE ID CREATED SIZE
145 opnfv/yardstick stable a4501714757a 1 day ago 915.4 MB
147 Run the Docker image:
151 docker run --privileged=true -it opnfv/yardstick /bin/bash
153 In the container the Yardstick repository is located in the /home/opnfv/repos
157 OpenStack parameters and credentials
158 ------------------------------------
160 Environment variables
161 ^^^^^^^^^^^^^^^^^^^^^
162 Before running Yardstick it is necessary to export OpenStack environment variables
163 from the OpenStack *openrc* file (using the ``source`` command) and export the
164 external network name ``export EXTERNAL_NETWORK="external-network-name"``,
165 the default name for the external network is ``net04_ext``.
167 Credential environment variables in the *openrc* file have to include at least:
174 A sample openrc file may look like this:
176 * export OS_PASSWORD=console
177 * export OS_TENANT_NAME=admin
178 * export OS_AUTH_URL=http://172.16.1.222:35357/v2.0
179 * export OS_USERNAME=admin
180 * export OS_VOLUME_API_VERSION=2
181 * export EXTERNAL_NETWORK=net04_ext
184 Yardstick falvor and guest images
185 ---------------------------------
187 Before executing Yardstick test cases, make sure that yardstick guest image and
188 yardstick flavor are available in OpenStack.
189 Detailed steps about creating yardstick flavor and building yardstick-trusty-server
190 image can be found below.
194 Most of the sample test cases in Yardstick are using an OpenStack flavor called
195 *yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the
196 disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny.
198 Create yardstick-flavor:
202 nova flavor-create yardstick-flavor 100 512 3 1
207 Building a guest image
208 ^^^^^^^^^^^^^^^^^^^^^^
209 Most of the sample test cases in Yardstick are using a guest image called
210 *yardstick-trusty-server* which deviates from an Ubuntu Cloud Server image
211 containing all the required tools to run test cases supported by Yardstick.
212 Yardstick has a tool for building this custom image. It is necessary to have
213 sudo rights to use this tool.
215 Also you may need install several additional packages to use this tool, by
216 follwing the commands below:
220 apt-get update && apt-get install -y \
224 This image can be built using the following command while in the directory where
225 Yardstick is installed (``~/yardstick`` if the framework is installed
226 by following the commands above):
230 export YARD_IMG_ARCH="amd64"
231 sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers
232 sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh
234 **Warning:** the script will create files by default in:
235 ``/tmp/workspace/yardstick`` and the files will be owned by root!
237 If you are building this guest image in inside a docker container make sure the
238 container is granted with privilege.
240 The created image can be added to OpenStack using the ``glance image-create`` or
241 via the OpenStack Dashboard.
247 glance --os-image-api-version 1 image-create \
248 --name yardstick-trusty-server --is-public true \
249 --disk-format qcow2 --container-format bare \
250 --file /tmp/workspace/yardstick/yardstick-trusty-server.img
252 Some Yardstick test cases use a Cirros image, you can find one at
253 http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-disk.img
256 Automatic flavor and image creation
257 -----------------------------------
258 Yardstick has a script for automatic creating yardstick flavor and building
259 guest images. This script is mainly used in CI, but you can still use it in
260 your local environment.
266 export YARD_IMG_ARCH="amd64"
267 sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers
268 source $YARDSTICK_REPO_DIR/tests/ci/load_images.sh
271 Yardstick default key pair
272 ^^^^^^^^^^^^^^^^^^^^^^^^^^
273 Yardstick uses a SSH key pair to connect to the guest image. This key pair can
274 be found in the ``resources/files`` directory. To run the ``ping-hot.yaml`` test
275 sample, this key pair needs to be imported to the OpenStack environment.
278 Examples and verifying the install
279 ----------------------------------
281 It is recommended to verify that Yardstick was installed successfully
282 by executing some simple commands and test samples. Before executing yardstick
283 test cases make sure yardstick flavor and building yardstick-trusty-server
284 image can be found in glance and openrc file is sourced. Below is an example
285 invocation of yardstick help command and ping.py test sample:
289 yardstick task start samples/ping.yaml
291 Each testing tool supported by Yardstick has a sample configuration file.
292 These configuration files can be found in the **samples** directory.
294 Default location for the output is ``/tmp/yardstick.out``.
297 Deploy InfluxDB and Grafana locally
298 ------------------------------------
300 .. pull docker images
304 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
308 docker pull tutum/influxdb
309 docker pull grafana/grafana
311 Run influxdb and config
312 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
316 docker run -d --name influxdb \
317 -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 \
319 docker exec -it influxdb bash
325 >CREATE USER root WITH PASSWORD 'root' WITH ALL PRIVILEGES
326 >CREATE DATABASE yardstick;
330 Run grafana and config
331 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
335 docker run -d --name grafana -p 3000:3000 grafana/grafana
340 http://{YOUR_IP_HERE}:3000
341 log on using admin/admin and config database resource to be {YOUR_IP_HERE}:8086
343 .. image:: images/Grafana_config.png
345 :alt: Grafana data source configration
347 Config yardstick conf
348 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
349 cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf
351 vi /etc/yardstick/yardstick.conf
352 Config yardstick.conf
357 dispatcher = influxdb
359 [dispatcher_influxdb]
361 target = http://{YOUR_IP_HERE}:8086
366 Now you can run yardstick test cases and store the results in influxdb
367 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
370 Create a test suite for yardstick
371 ------------------------------------
373 A test suite in yardstick is a yaml file which include one or more test cases.
374 Yardstick is able to support running test suite task, so you can customize you
375 own test suite and run it in one task.
377 "tests/opnfv/test_suites" is where yardstick put ci test-suite. A typical test
385 # Fuel integration test task suite
387 schema: "yardstick:suite:0.1"
389 name: "fuel_test_suite"
390 test_cases_dir: "samples/"
395 file_name: iperf3.yaml
397 As you can see, there are two test cases in fuel_test_suite, the syntax is simple
398 here, you must specify the schema and the name, then you just need to list the
399 test cases in the tag "test_cases" and also mark their relative directory in the
400 tag "test_cases_dir".
402 Yardstick test suite also support constraints and task args for each test suite.
403 Here is another sample to show this, which is digested from one big test suite.
405 os-nosdn-nofeature-ha.yaml
411 schema: "yardstick:suite:0.1"
413 name: "os-nosdn-nofeature-ha"
414 test_cases_dir: "tests/opnfv/test_cases/"
417 file_name: opnfv_yardstick_tc002.yaml
419 file_name: opnfv_yardstick_tc005.yaml
421 file_name: opnfv_yardstick_tc043.yaml
426 huawei-pod1: '{"pod_info": "etc/yardstick/.../pod.yaml",
427 "host": "node4.LF","target": "node5.LF"}'
429 As you can see in test case "opnfv_yardstick_tc043.yaml", there are two tags, "constraint" and
430 "task_args". "constraint" is where you can specify which installer or pod it can be run in
431 the ci environment. "task_args" is where you can specify the task arguments for each pod.
433 All in all, to create a test suite in yardstick, you just need to create a suite yaml file
434 and add test cases and constraint or task arguments if necessary.