1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others.
13 Yardstick supports installation by Docker or directly in Ubuntu. The
14 installation procedure for Docker and direct installation are detailed in
17 To use Yardstick you should have access to an OpenStack environment, with at
18 least Nova, Neutron, Glance, Keystone and Heat installed.
20 The steps needed to run Yardstick are:
23 2. Load OpenStack environment variables.
24 3. Create a Neutron external network.
25 4. Build Yardstick flavor and a guest image.
26 5. Load the guest image into the OpenStack environment.
27 6. Create the test configuration .yaml file.
34 The OPNFV deployment is out of the scope of this document but it can be
35 found in http://artifacts.opnfv.org/opnfvdocs/colorado/docs/configguide/index.html.
36 The OPNFV platform is considered as the System Under Test (SUT) in this
39 Several prerequisites are needed for Yardstick:
41 #. A Jumphost to run Yardstick on
42 #. A Docker daemon shall be installed on the Jumphost
43 #. A public/external network created on the SUT
44 #. Connectivity from the Jumphost to the SUT public/external network
46 WARNING: Connectivity from Jumphost is essential and it is of paramount
47 importance to make sure it is working before even considering to install
48 and run Yardstick. Make also sure you understand how your networking is
51 NOTE: **Jumphost** refers to any server which meets the previous
52 requirements. Normally it is the same server from where the OPNFV
53 deployment has been triggered previously.
55 NOTE: If your Jumphost is operating behind a company http proxy and/or
56 Firewall, please consult first the section `Proxy Support`_, towards
57 the end of this document. The section details some tips/tricks which
58 *may* be of help in a proxified environment.
61 Installing Yardstick using Docker
62 ---------------------------------
64 Yardstick has a Docker image,
65 **It is recommended to use this Docker image to run Yardstick test**.
67 Pulling the Yardstick Docker image
68 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
70 .. _dockerhub: https://hub.docker.com/r/opnfv/yardstick/
72 Pull the Yardstick Docker image (**opnfv/yardstick**) from the public dockerhub
73 registry under the OPNFV account: [dockerhub_], with the following docker
76 docker pull opnfv/yardstick:stable
78 After pulling the Docker image, check that it is available with the
79 following docker command::
81 [yardsticker@jumphost ~]$ docker images
82 REPOSITORY TAG IMAGE ID CREATED SIZE
83 opnfv/yardstick stable a4501714757a 1 day ago 915.4 MB
85 Run the Docker image to get a Yardstick container
86 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
90 docker run -itd --privileged -v /var/run/docker.sock:/var/run/docker.sock -p 8888:5000 -e INSTALLER_IP=192.168.200.2 -e INSTALLER_TYPE=compass --name yardstick opnfv/yardstick:stable
94 +----------------------------------------------+------------------------------+
95 | parameters | Detail |
96 +==============================================+==============================+
97 | -itd | -i: interactive, Keep STDIN |
98 | | open even if not attached. |
99 | | -t: allocate a pseudo-TTY. |
100 | | -d: run container in |
101 | | detached mode, in the |
103 +----------------------------------------------+------------------------------+
104 | --privileged | If you want to build |
105 | | yardstick-image in yardstick |
106 | | container, this parameter is |
108 +----------------------------------------------+------------------------------+
109 | -e INSTALLER_IP=192.168.200.2 | If you want to use yardstick |
110 | | env prepare command(or |
111 | -e INSTALLER_TYPE=compass | related API) to load the |
112 | | images that yardstick needs, |
113 | | these parameters should be |
115 | | The INSTALLER_IP and |
116 | | INSTALLER_TYPE are depending |
117 | | on your OpenStack installer, |
118 | | currently apex, compass, |
119 | | fuel and joid are supported. |
120 | | If you use other installers, |
121 | | such as devstack, these |
122 | | parameters can be ignores. |
123 +----------------------------------------------+------------------------------+
124 | -p 8888:5000 | If you want to call |
125 | | yardstick API out of |
126 | | yardstick container, this |
127 | | parameter is needed. |
128 +----------------------------------------------+------------------------------+
129 | -v /var/run/docker.sock:/var/run/docker.sock | If you want to use yardstick |
130 | | env grafana/influxdb to |
131 | | create a grafana/influxdb |
132 | | container out of yardstick |
133 | | container, this parameter is |
135 +----------------------------------------------+------------------------------+
136 | --name yardstick | The name for this container, |
137 | | not needed and can be |
138 | | defined by the user. |
139 +----------------------------------------------+------------------------------+
141 Enter Yardstick container
142 ^^^^^^^^^^^^^^^^^^^^^^^^^
146 docker exec -it yardstick /bin/bash
148 In the container, the Yardstick repository is located in the /home/opnfv/repos
151 In Danube release, we have improved the Yardstick installation steps.
152 Now Yardstick provides a CLI to prepare openstack environment variables and
153 load yardstick images::
155 yardstick env prepare
157 If you ues this command. you can skip the following sections about how to
158 prepare openstack environment variables, load yardstick images and load
159 yardstick flavor manually.
162 Installing Yardstick directly in Ubuntu
163 ---------------------------------------
165 .. _install-framework:
167 Alternatively you can install Yardstick framework directly in Ubuntu or in an Ubuntu Docker
168 image. No matter which way you choose to install Yardstick framework, the
169 following installation steps are identical.
171 If you choose to use the Ubuntu Docker image, You can pull the Ubuntu
172 Docker image from Docker hub:
176 docker pull ubuntu:16.04
179 Installing Yardstick framework
180 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
181 Download source code and install Yardstick framework:
185 git clone https://gerrit.opnfv.org/gerrit/yardstick
189 For installing yardstick directly in Ubuntu, the **yardstick env command** is not available.
190 You need to prepare openstack environment variables, load yardstick images and load
191 yardstick flavor manually.
194 OpenStack parameters and credentials
195 ------------------------------------
197 Environment variables
198 ^^^^^^^^^^^^^^^^^^^^^
199 Before running Yardstick it is necessary to export OpenStack environment variables
200 from the OpenStack *openrc* file (using the ``source`` command) and export the
201 external network name ``export EXTERNAL_NETWORK="external-network-name"``,
202 the default name for the external network is ``net04_ext``.
204 Credential environment variables in the *openrc* file have to include at least:
211 A sample openrc file may look like this:
213 * export OS_PASSWORD=console
214 * export OS_TENANT_NAME=admin
215 * export OS_AUTH_URL=http://172.16.1.222:35357/v2.0
216 * export OS_USERNAME=admin
217 * export OS_VOLUME_API_VERSION=2
218 * export EXTERNAL_NETWORK=net04_ext
221 Yardstick falvor and guest images
222 ---------------------------------
224 Before executing Yardstick test cases, make sure that yardstick guest image and
225 yardstick flavor are available in OpenStack.
226 Detailed steps about creating yardstick flavor and building yardstick-trusty-server
227 image can be found below.
231 Most of the sample test cases in Yardstick are using an OpenStack flavor called
232 *yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the
233 disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny.
235 Create yardstick-flavor:
239 nova flavor-create yardstick-flavor 100 512 3 1
244 Building a guest image
245 ^^^^^^^^^^^^^^^^^^^^^^
246 Most of the sample test cases in Yardstick are using a guest image called
247 *yardstick-trusty-server* which deviates from an Ubuntu Cloud Server image
248 containing all the required tools to run test cases supported by Yardstick.
249 Yardstick has a tool for building this custom image. It is necessary to have
250 sudo rights to use this tool.
252 Also you may need install several additional packages to use this tool, by
253 follwing the commands below:
257 apt-get update && apt-get install -y \
261 This image can be built using the following command while in the directory where
262 Yardstick is installed (``~/yardstick`` if the framework is installed
263 by following the commands above):
267 sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh
269 **Warning:** the script will create files by default in:
270 ``/tmp/workspace/yardstick`` and the files will be owned by root!
272 If you are building this guest image in inside a docker container make sure the
273 container is granted with privilege.
275 The created image can be added to OpenStack using the ``glance image-create`` or
276 via the OpenStack Dashboard.
282 glance --os-image-api-version 1 image-create \
283 --name yardstick-image --is-public true \
284 --disk-format qcow2 --container-format bare \
285 --file /tmp/workspace/yardstick/yardstick-image.img
287 Some Yardstick test cases use a Cirros image and a Ubuntu 14.04 image, you can find one at
288 http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img, https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
290 Add cirros and ubuntu image to OpenStack:
294 openstack image create \
295 --disk-format qcow2 \
296 --container-format bare \
297 --file $cirros_image_file \
300 openstack image create \
301 --disk-format qcow2 \
302 --container-format bare \
303 --file $ubuntu_image_file \
306 Automatic flavor and image creation
307 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
309 Yardstick has a script for automatic creating yardstick flavor and building
310 guest images. This script is mainly used in CI, but you can still use it in
311 your local environment.
317 source $YARDSTICK_REPO_DIR/tests/ci/load_images.sh
320 Examples and verifying the install
321 ----------------------------------
323 It is recommended to verify that Yardstick was installed successfully
324 by executing some simple commands and test samples. Before executing yardstick
325 test cases make sure yardstick flavor and building yardstick-trusty-server
326 image can be found in glance and openrc file is sourced. Below is an example
327 invocation of yardstick help command and ping.py test sample:
331 yardstick task start samples/ping.yaml
333 Each testing tool supported by Yardstick has a sample configuration file.
334 These configuration files can be found in the **samples** directory.
336 Default location for the output is ``/tmp/yardstick.out``.
339 Deploy InfluxDB and Grafana locally
340 ------------------------------------
342 The 'yardstick env' command can also help you to build influxDB and Grafana in
343 your local environment.
345 Create InfluxDB container and config with the following command::
347 yardstick env influxdb
350 Create Grafana container and config::
352 yardstick env grafana
354 Then you can run a test case and visit http://host_ip:3000(user:admin,passwd:admin) to see the results.
356 note: Using **yardstick env** command to deploy InfluxDB and Grafana requires
357 Jump Server's docker API version => 1.24. You can use the following command to
358 check the docker API version:
364 The following sections describe how to deploy influxDB and Grafana manually.
366 .. pull docker images
370 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
374 docker pull tutum/influxdb
375 docker pull grafana/grafana
377 Run influxdb and config
378 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
382 docker run -d --name influxdb \
383 -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 \
385 docker exec -it influxdb bash
391 >CREATE USER root WITH PASSWORD 'root' WITH ALL PRIVILEGES
392 >CREATE DATABASE yardstick;
396 Run grafana and config
397 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
401 docker run -d --name grafana -p 3000:3000 grafana/grafana
406 http://{YOUR_IP_HERE}:3000
407 log on using admin/admin and config database resource to be {YOUR_IP_HERE}:8086
409 .. image:: images/Grafana_config.png
411 :alt: Grafana data source configration
413 Config yardstick conf
414 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
415 cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf
417 vi /etc/yardstick/yardstick.conf
418 Config yardstick.conf
423 dispatcher = influxdb
425 [dispatcher_influxdb]
427 target = http://{YOUR_IP_HERE}:8086
432 Now you can run yardstick test cases and store the results in influxdb
433 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
436 Create a test suite for yardstick
437 ------------------------------------
439 A test suite in yardstick is a yaml file which include one or more test cases.
440 Yardstick is able to support running test suite task, so you can customize you
441 own test suite and run it in one task.
443 "tests/opnfv/test_suites" is where yardstick put ci test-suite. A typical test
451 # Fuel integration test task suite
453 schema: "yardstick:suite:0.1"
455 name: "fuel_test_suite"
456 test_cases_dir: "samples/"
461 file_name: iperf3.yaml
463 As you can see, there are two test cases in fuel_test_suite, the syntax is simple
464 here, you must specify the schema and the name, then you just need to list the
465 test cases in the tag "test_cases" and also mark their relative directory in the
466 tag "test_cases_dir".
468 Yardstick test suite also support constraints and task args for each test case.
469 Here is another sample to show this, which is digested from one big test suite.
471 os-nosdn-nofeature-ha.yaml
477 schema: "yardstick:suite:0.1"
479 name: "os-nosdn-nofeature-ha"
480 test_cases_dir: "tests/opnfv/test_cases/"
483 file_name: opnfv_yardstick_tc002.yaml
485 file_name: opnfv_yardstick_tc005.yaml
487 file_name: opnfv_yardstick_tc043.yaml
492 huawei-pod1: '{"pod_info": "etc/yardstick/.../pod.yaml",
493 "host": "node4.LF","target": "node5.LF"}'
495 As you can see in test case "opnfv_yardstick_tc043.yaml", there are two tags, "constraint" and
496 "task_args". "constraint" is where you can specify which installer or pod it can be run in
497 the ci environment. "task_args" is where you can specify the task arguments for each pod.
499 All in all, to create a test suite in yardstick, you just need to create a suite yaml file
500 and add test cases and constraint or task arguments if necessary.