1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others.
12 Yardstick supports installation on Ubuntu 14.04 or by using a Docker image.
13 The installation procedure on Ubuntu 14.04 or via the docker image are
14 detailed in the section below
16 To use Yardstick you should have access to an OpenStack environment,
17 with at least Nova, Neutron, Glance, Keystone and Heat installed.
19 The steps needed to run Yardstick are:
22 2. Create the test configuration .yaml file.
23 3. Build a guest image。
24 4 .Load the image into the OpenStack environment.
25 5. Create a Neutron external network.
26 6. Load OpenStack environment variables.
30 Installing Yardstick on Ubuntu 14.04
31 ------------------------------------
33 .. _install-framework:
35 You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu
37 No matter which way you choose to install Yardstick framework, the following
38 installation steps are identical.
39 If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu
40 14.04 Docker image from Docker hub:
44 docker pull ubuntu:14.04
46 Installing Yardstick framework
47 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
52 sudo apt-get update && sudo apt-get install -y \
67 Create a python virtual environment, source it and update setuptools:
71 virtualenv ~/yardstick_venv
72 source ~/yardstick_venv/bin/activate
73 easy_install -U setuptools
75 Download source code and install python dependencies:
79 git clone https://gerrit.opnfv.org/gerrit/yardstick
81 python setup.py install
83 A Youtube video tutorial on this installation mode is available
84 at: http://www.youtube.com/watch?v=4S4izNolmR0
86 .. image:: http://img.youtube.com/vi/4S4izNolmR0/0.jpg
87 :alt: http://www.youtube.com/watch?v=4S4izNolmR0
88 :target: http://www.youtube.com/watch?v=4S4izNolmR0
92 Building a guest image
93 ^^^^^^^^^^^^^^^^^^^^^^
94 Yardstick has a tool for building an Ubuntu Cloud Server image containing all
95 the required tools to run test cases supported by Yardstick. It is necessary to
96 have sudo rights to use this tool.
98 Also you may need install several additional packages to use this tool, by
99 follwing the commands below:
103 apt-get update && apt-get install -y \
107 This image can be built using the following command while in the directory where
108 Yardstick is installed (``~/yardstick`` if the framework is installed
109 by following the commands above):
113 eport YARD_IMG_ARCH="amd64"
114 sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers
115 sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh
117 **Warning:** the script will create files by default in:
118 ``/tmp/workspace/yardstick`` and the files will be owned by root!
120 The created image can be added to OpenStack using the ``glance image-create`` or
121 via the OpenStack Dashboard.
127 glance --os-image-api-version 1 image-create \
128 --name yardstick-trusty-server --is-public true \
129 --disk-format qcow2 --container-format bare \
130 --file /tmp/workspace/yardstick/yardstick-trusty-server.img
133 Installing Yardstick using Docker
134 ---------------------------------
136 Yardstick iteself has a Docker image, this Docker image (**Yardstick-stable**)
137 serves as a replacement for installing the Yardstick framework in a virtual
138 environment (for example as done in :ref:`install-framework`).
139 It is recommended to use this Docker image to run Yardstick test.
141 Yardstick-stable image
142 ^^^^^^^^^^^^^^^^^^^^^^
143 Pull the Yardstick-stable Docker image from Docker hub:
147 docker pull opnfv/yardstick:stable
149 Run the Docker image:
157 -e "INSTALLER_TYPE=${INSTALLER_TYPE}" \
158 -e "INSTALLER_IP=${INSTALLER_IP}" \
160 exec_tests.sh ${YARDSTICK_DB_BACKEND} ${YARDSTICK_SUITE_NAME}
162 Where ``${INSTALLER_TYPE}`` can be apex, compass, fuel or joid, ``${INSTALLER_IP}``
163 is the installer master node IP address (i.e. 10.20.0.2 is default for fuel). ``${YARDSTICK_DB_BACKEND}``
164 is the IP and port number of DB, ``${YARDSTICK_SUITE_NAME}`` is the test suite you want to run.
165 For more details, please refer to the Jenkins job defined in Releng project, labconfig information
166 and sshkey are required. See the link
167 https://git.opnfv.org/cgit/releng/tree/jjb/yardstick/yardstick-ci-jobs.yml.
169 Note: exec_tests.sh is used for executing test suite here, furthermore, if someone
170 wants to execute the test suite manually, it can be used as long as the parameters
171 are configured correct. Another script called run_tests.sh is used for unittest in
172 Jenkins verify job, in local manaul environment, it is recommended to run before
173 test suite execuation.
175 Basic steps performed by the **Yardstick-stable** container:
177 1. clone yardstick and releng repos
178 2. setup OS credentials (releng scripts)
179 3. install yardstick and dependencies
180 4. build yardstick cloud image
181 5. Upload yardstick cloud image to glance
182 6. upload cirros-0.3.3 cloud image and ubuntu-14.04 cloud image to glance
183 7. run yardstick test scenarios
186 If someone only wants to execute a single test case, one can log into the yardstick-stable
187 container first using command:
191 docker run -it openfv/yardstick /bin/bash
193 Then in the container run yardstick task command to execute a single test case.
194 Before executing Yardstick test case, make sure that yardstick-trusty-server
195 image and yardstick flavor is available in OpenStack.
196 Detailed steps about creating yardstick flavor and executing Yardstick test case
200 OpenStack parameters and credentials
201 ------------------------------------
205 Most of the sample test cases in Yardstick are using an OpenStack flavor called
206 *yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the
207 disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny.
209 Create yardstick-flavor:
213 nova flavor-create yardstick-flavor 100 512 3 1
215 Environment variables
216 ^^^^^^^^^^^^^^^^^^^^^
217 Before running Yardstick it is necessary to export OpenStack environment variables
218 from the OpenStack *openrc* file (using the ``source`` command) and export the
219 external network name ``export EXTERNAL_NETWORK="external-network-name"``,
220 the default name for the external network is ``net04_ext``.
222 Credential environment variables in the *openrc* file have to include at least:
229 Yardstick default key pair
230 ^^^^^^^^^^^^^^^^^^^^^^^^^^
231 Yardstick uses a SSH key pair to connect to the guest image. This key pair can
232 be found in the ``resources/files`` directory. To run the ``ping-hot.yaml`` test
233 sample, this key pair needs to be imported to the OpenStack environment.
236 Examples and verifying the install
237 ----------------------------------
239 It is recommended to verify that Yardstick was installed successfully
240 by executing some simple commands and test samples. Below is an example invocation
241 of yardstick help command and ping.py test sample:
245 yardstick task start samples/ping.yaml
247 Each testing tool supported by Yardstick has a sample configuration file.
248 These configuration files can be found in the **samples** directory.
250 Default location for the output is ``/tmp/yardstick.out``.
253 Deploy InfluxDB and Grafana locally
254 ------------------------------------
256 .. pull docker images
260 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
264 docker pull tutum/influxdb
265 docker pull grafana/grafana
267 Run influxdb and config
268 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
272 docker run -d --name influxdb \
273 -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 \
275 docker exec -it influxdb bash
281 >CREATE USER root WITH PASSWORD 'root' WITH ALL PRIVILEGES
282 >CREATE DATABASE yardstick;
286 Run grafana and config
287 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
291 docker run -d --name grafana -p 3000:3000 grafana/grafana
296 http://{YOUR_IP_HERE}:3000
297 log on using admin/admin and config database resource to be {YOUR_IP_HERE}:8086
299 .. image:: images/Grafana_config.png
301 :alt: Grafana data source configration
303 Config yardstick conf
304 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
305 cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf
307 vi /etc/yardstick/yardstick.conf
308 Config yardstick.conf
313 dispatcher = influxdb
315 [dispatcher_influxdb]
317 target = http://{YOUR_IP_HERE}:8086
322 Now you can run yardstick test cases and store the results in influxdb
323 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
326 Create a test suite for yardstick
327 ------------------------------------
329 A test suite in yardstick is a yaml file which include one or more test cases.
330 Yardstick is able to support running test suite task, so you can customize you
331 own test suite and run it in one task.
333 "tests/opnfv/test_suites" is where yardstick put ci test-suite. A typical test
341 # Fuel integration test task suite
343 schema: "yardstick:suite:0.1"
345 name: "fuel_test_suite"
346 test_cases_dir: "samples/"
351 file_name: iperf3.yaml
353 As you can see, there are two test cases in fuel_test_suite, the syntax is simple
354 here, you must specify the schema and the name, then you just need to list the
355 test cases in the tag "test_cases" and also mark their relative directory in the
356 tag "test_cases_dir".
358 Yardstick test suite also support constraints and task args for each test suite.
359 Here is another sample to show this, which is digested from one big test suite.
361 os-nosdn-nofeature-ha.yaml
367 schema: "yardstick:suite:0.1"
369 name: "os-nosdn-nofeature-ha"
370 test_cases_dir: "tests/opnfv/test_cases/"
373 file_name: opnfv_yardstick_tc002.yaml
375 file_name: opnfv_yardstick_tc005.yaml
377 file_name: opnfv_yardstick_tc043.yaml
382 huawei-pod1: '{"pod_info": "etc/yardstick/.../pod.yaml",
383 "host": "node4.LF","target": "node5.LF"}'
385 As you can see in test case "opnfv_yardstick_tc043.yaml", there are two tags, "constraint" and
386 "task_args". "constraint" is where you can specify which installer or pod it can be run in
387 the ci environment. "task_args" is where you can specify the task arguments for each pod.
389 All in all, to create a test suite in yardstick, you just need to create a suite yaml file
390 and add test cases and constraint or task arguments if necessary.