8 Yardstick is a framework to test non functional characteristics of an NFV
9 Infrastructure as perceived by an application.
11 An application is a set of virtual machines deployed using the orchestrator of
12 the target cloud, for example OpenStack Heat.
14 Yardstick measures a certain service performance but can also validate the
15 service performance to be within a certain level of agreement.
17 Yardstick is _not_ about testing OpenStack functionality (tempest) or
18 benchmarking OpenStack APIs (rally).
23 Benchmark - assess the relative performance of something
25 Benchmark configuration file - describes a single test case in yaml format
28 - The set of cloud resources used by a benchmark (scenario)
29 – Is a simplified Heat template (context is converted into a Heat template)
32 - Output produced by running a benchmark, written to a file in json format
35 - Logic that determines how the test is run
36 – For example number of iterations, input value stepping, duration etc
39 - Type/class of measurement for example Ping, Pktgen, (Iperf, LmBench, ...)
42 - Some limit to be verified (specific to scenario), for example max_latency
43 – Associated action to automatically take: assert, monitor etc
48 Yardstick is a command line tool written in python inspired by Rally. Yardstick
49 is intended to run on a computer with access and credentials to a cloud. The
50 test case is described in a configuration file given as an argument.
52 How it works: the benchmark task configuration file is parsed and converted into
53 an internal model. The context part of the model is converted into a Heat
54 template and deployed into a stack. Each scenario is run using a runner, either
55 serially or in parallel. Each runner runs in its own subprocess executing
56 commands in a VM using SSH. The output of each command is written as json
72 pktgen test requires a ubuntu server cloud image
75 Development Environment
76 =======================
78 Example setup known to work for development and test:
79 - Development environment: Ubuntu14.04, eclipse, virtual environment
80 - Cloud: Mirantis OpenStack 6.0 deployed using Virtualbox
83 $ sudo apt-get install python-virtualenv python-dev libffi-dev libssl-dev libxml2-dev libxslt1-dev
84 $ sudo easy_install -U setuptools
86 Create a virtual environment:
87 $ virtualenv ~/yardstick_venv
88 $ source ~/yardstick_venv/bin/activate
89 $ python setup.py develop