1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, Ericsson AB and others.
13 Yardstick is a framework to test non functional characteristics of an NFV
14 Infrastructure as perceived by an application.
16 An application is a set of virtual machines deployed using the orchestrator of
17 the target cloud, for example OpenStack Heat.
19 Yardstick measures a certain service performance but can also validate the
20 service performance to be within a certain level of agreement.
22 Yardstick is _not_ about testing OpenStack functionality (tempest) or
23 benchmarking OpenStack APIs (rally).
28 Benchmark - assess the relative performance of something
30 Benchmark configuration file - describes a single test case in yaml format
33 - The set of cloud resources used by a benchmark (scenario)
34 – Is a simplified Heat template (context is converted into a Heat template)
37 - Output produced by running a benchmark, written to a file in json format
40 - Logic that determines how the test is run
41 – For example number of iterations, input value stepping, duration etc
44 - Type/class of measurement for example Ping, Pktgen, (Iperf, LmBench, ...)
47 - Some limit to be verified (specific to scenario), for example max_latency
48 – Associated action to automatically take: assert, monitor etc
53 Yardstick is a command line tool written in python inspired by Rally. Yardstick
54 is intended to run on a computer with access and credentials to a cloud. The
55 test case is described in a configuration file given as an argument.
57 How it works: the benchmark task configuration file is parsed and converted into
58 an internal model. The context part of the model is converted into a Heat
59 template and deployed into a stack. Each scenario is run using a runner, either
60 serially or in parallel. Each runner runs in its own subprocess executing
61 commands in a VM using SSH. The output of each command is written as json
77 pktgen test requires a ubuntu server cloud image
80 Development Environment
81 =======================
83 Example setup known to work for development and test:
84 - Development environment: Ubuntu14.04, eclipse, virtual environment
85 - Cloud: Mirantis OpenStack 6.0 deployed using Virtualbox
88 $ sudo apt-get install python-virtualenv python-dev libffi-dev libssl-dev libxml2-dev libxslt1-dev
90 Create a virtual environment:
91 $ virtualenv ~/yardstick_venv
92 $ source ~/yardstick_venv/bin/activate
93 $ easy_install -U setuptools
94 $ python setup.py develop