+++ /dev/null
-.. OPNFV Yardstick documentation master file.
- Add chapters and user guides using the
- root `toctree` directive.
-
-=================
-Yardstick Project
-=================
-
-Welcome to Yardstick's documentation !
-
-.. _Yardstick: https://wiki.opnfv.org/yardstick
-
-Yardstick_ is an OPNFV testing project.
-
-The project goal is to verify infrastructure compliance, from the perspective
-of a VNF.
-
-The project scope is the development of a test framework, test cases and test
-stimuli.
-
-This document introduces the methodology applied in Yardstick_ project, which
-decomposes typical VNF work-load performance metrics into a number of
-characteristics/performance vectors, each of them can be represented by
-distinct test-cases.
-
-The user guides and test cases delivered for the release are included.
-
-Contents:
-
-Yardstick Project Description
-=============================
-
-.. toctree::
- :numbered:
- :maxdepth: 2
-
-User Guides
-===========
-
-.. toctree::
- :maxdepth: 1
-
- user_guides/framework/03-installation
-
-Indices
-=======
-
-* :ref:`search`
-
-Revision: _sha1_
-
-Build date: |today|
Basic template syntax
---------------------
-A nice feature of the input task format used in Yardstick is that it supports the template syntax based on Jinja2.
-This turns out to be extremely useful when, say, you have a fixed structure of your task but you want to
-parameterize this task in some way.
-For example, imagine your input task file (task.yaml) runs a set of Ping scenarios:
+A nice feature of the input task format used in Yardstick is that it supports
+the template syntax based on Jinja2.
+This turns out to be extremely useful when, say, you have a fixed structure of
+your task but you want to parameterize this task in some way.
+For example, imagine your input task file (task.yaml) runs a set of Ping
+scenarios:
::
context:
...
-Let's say you want to run the same set of scenarios with the same runner/context/sla,
-but you want to try another packetsize to compare the performance.
-The most elegant solution is then to turn the packetsize name into a template variable:
+Let's say you want to run the same set of scenarios with the same runner/
+context/sla, but you want to try another packetsize to compare the performance.
+The most elegant solution is then to turn the packetsize name into a template
+variable:
::
context:
...
-and then pass the argument value for {{packetsize}} when starting a task with this configuration file.
+and then pass the argument value for {{packetsize}} when starting a task with
+this configuration file.
Yardstick provides you with different ways to do that:
-1.Pass the argument values directly in the command-line interface (with either a JSON or YAML dictionary):
+1.Pass the argument values directly in the command-line interface (with either
+a JSON or YAML dictionary):
::
- yardstick task start samples/ping-template.yaml --task-args '{"packetsize": "200"}'
+ yardstick task start samples/ping-template.yaml
+ --task-args'{"packetsize":"200"}'
2.Refer to a file that specifies the argument values (JSON/YAML):
Using the default values
------------------------
-Note that the Jinja2 template syntax allows you to set the default values for your parameters.
-With default values set, your task file will work even if you don't parameterize it explicitly while starting a task.
-The default values should be set using the {% set ... %} clause (task.yaml).For example:
+Note that the Jinja2 template syntax allows you to set the default values for
+your parameters.
+With default values set, your task file will work even if you don't
+parameterize it explicitly while starting a task.
+The default values should be set using the {% set ... %} clause (task.yaml).
+For example:
::
interval: 1
...
-If you don't pass the value for {{packetsize}} while starting a task, the default one will be used.
+If you don't pass the value for {{packetsize}} while starting a task, the
+default one will be used.
Advanced templates
------------------
-Yardstick makes it possible to use all the power of Jinja2 template syntax, including the mechanism of built-in functions.
-As an example, let us make up a task file that will do a block storage performance test.
-The input task file (fio-template.yaml) below uses the Jinja2 for-endfor construct to accomplish that:
+
+Yardstick makes it possible to use all the power of Jinja2 template syntax,
+including the mechanism of built-in functions.
+As an example, let us make up a task file that will do a block storage
+performance test.
+The input task file (fio-template.yaml) below uses the Jinja2 for-endfor
+construct to accomplish that:
::
+++ /dev/null
-.. Template to be used for test case descriptions in Yardstick Project.
- Write one .rst per test case.
- Upload the .rst for the test case in /docs/source/yardstick directory.
- Review in Gerrit.
-
-.. image:: ../etc/opnfv-logo.png
- :height: 40
- :width: 200
- :alt: OPNFV
- :align: left
-
-******************
-Test Case <slogan>
-******************
-
-.. contents:: Table of Contents
- :depth: 3
-
----------------------
-Test Case Description
----------------------
-
-Yardstick Test Case ID
-----------------------
-
-OPNFV_YARDSTICK_TC<abc>_<slogan>
-
-where:
- - <abc>: check Jira issue for the test case
- - <slogan>: check Jira issue for the test case
-
-
-Purpose
--------
-
-Describe what is the purpose of the test case
-
-Area
-----
-
-State the area and sub-area covered by the test case.
-
-Areas: Compute, Networking, Storage
-
-Sub-areas: Performance, System limit, QoS
-
-Metrics
--------
-
-What will be measured, attribute name or collection of attributes, behavior
-
-References
-----------
-
-Reference documentation
-
---------------
-Pre-requisites
---------------
-
-Tools
------
-
-What tools are used to perform the measurements (e.g. fio, pktgen)
-
-
-Configuration
--------------
-
-State the .yaml file to use.
-
-State default configuration in the tool(s) used to perform the measurements
-(e.g. fio, pktgen).
-
-State what POD-specific configuration is required to enable running the test
-case in different PODs.
-
-
-State SLA, if applicable.
-
-State test duration.
-
--------
-Results
--------
-
-Expected outcome
-----------------
-
-State applicable graphical presentation
-
-State applicable output details
-
-State expected Value, behavior, pass/fail criteria
Upload the .rst for the test case in /docs/source/yardstick directory.
Review in Gerrit.
-.. image:: ../etc/opnfv-logo.png
- :height: 40
- :width: 200
- :alt: OPNFV
- :align: left
-
*************************************
Yardstick Test Case Description TCXXX
*************************************
--- /dev/null
+=================================
+Yardstick Framework Documentation
+=================================
+
+.. toctree::
+ :numbered:
+ :maxdepth: 2
+
+ 03-installation
-=========
-Yardstick
-=========
+==========================
+Virtual Traffic Classifier
+==========================
-Overview of the virtual Traffic Classifier
+Overview
========
+
The virtual Traffic Classifier VNF [1], comprises in the current version of
1 VNFC [2]. The VNFC contains both the Traffic Inspection module, and the
Traffic forwarding module, needed to run the VNF. The exploitation of DPI
methods for traffic classification is built around two basic assumptions:
+
(i) third parties unaffiliated with either source or recipient are able to
inspect each IP packet’s payload and
(ii) the classifier knows the relevant syntax of each application’s packet
payloads (protocol signatures, data patterns, etc.).
-The proposed DPI based approach will only use an indicative, small number of the
-initial packets from each flow in order to identify the content and not inspect
-each packet.
+
+The proposed DPI based approach will only use an indicative, small number of
+the initial packets from each flow in order to identify the content and not
+inspect each packet.
+
In this respect it follows the Packet Based per Flow State (PBFS).
This method uses a table to track each session based on the 5-tuples
(src address,dest address,src port,dest port,transport protocol)
Concepts
========
+
Traffic Inspection: The process of packet analysis and application
identification of network traffic that passes through the vTC.
network interface to a pre-defined outgoing network interface.
Traffic Rule Application: The process of packet tagging, based on a
-predefined set of rules. Packet tagging may include e.g. ToS field modification.
+predefined set of rules. Packet tagging may include e.g. ToS field
+modification.
Architecture
============
The Traffic Inspection module is the most computationally intensive component
of the VNF. It implements filtering and packet matching algorithms in order to
support the enhanced traffic forwarding capability of the VNF. The component
-supports a flow table (exploiting hashing algorithms for fast indexing of flows)
-and an inspection engine for traffic classification. The implementation used for
-these experiments exploits the nDPI library. The packet capturing mechanism is
-implemented using libpcap. When the DPI engine identifies a new flow, the flow
-register is updated with the appropriate information and transmitted across the
-Traffic Forwarding module, which then applies any required policy updates.
+supports a flow table (exploiting hashing algorithms for fast indexing of
+flows) and an inspection engine for traffic classification.
+
+The implementation used for these experiments exploits the nDPI library.
+The packet capturing mechanism is implemented using libpcap. When the DPI
+engine identifies a new flow, the flow register is updated with the
+appropriate information and transmitted across the Traffic Forwarding module,
+which then applies any required policy updates.
+
The Traffic Forwarding moudle is responsible for routing and packet forwarding.
It accepts incoming network traffic, consults the flow table for classification
-information for each incoming flow and then applies pre-defined policies marking
-e.g. type of Service/Differentiated Services Code Point (TOS/DSCP) multimedia
-traffic for QoS enablement on the forwarded traffic. It is assumed that the
-traffic is forwarded using the default policy until it is identified and new
-policies are enforced. The expected response delay is considered to be
-negligible,as only a small number of packets are required to identify each flow.
+information for each incoming flow and then applies pre-defined policies
+marking e.g. type of Service/Differentiated Services Code Point (TOS/DSCP)
+multimedia traffic for QoS enablement on the forwarded traffic.
+It is assumed that the traffic is forwarded using the default policy until it
+is identified and new policies are enforced.
+
+The expected response delay is considered to be negligible,as only a small
+number of packets are required to identify each flow.
Graphical Overview
==================
-+----------------------------+
-| |
-| Virtual Traffic Classifier |
-| |
-| Analysing/Forwarding |
-| +--------> |
-| ethA ethB |
-+------+--------------+------+
- | ^
- | |
- | |
- | |
- v |
-+------+--------------+------+
-| |
-| Virtual Switch |
-| |
-+----------------------------+
-
-
Install
=======
Abbreviations for the virtual Traffic Classifier
-========
+================================================
[1] VNF - Virtual Network Function
[2] VNFC - Virtual Network Function Component
-
--- /dev/null
+======================
+Yardstick Config Guide
+======================
+
+Test Case Descriptions
+======================
+
+.. toctree::
+ :maxdepth: 1
+
+ opnfv_yardstick_tc001.rst
+ opnfv_yardstick_tc002.rst
+
+Templates
+=========
+
+.. toctree::
+ :maxdepth: 1
+
+ ../templates/Yardstick_task_templates
+ ../templates/testcase_description_v2_template
-.. image:: ../../etc/opnfv-logo.png
- :height: 40
- :width: 200
- :alt: OPNFV
- :align: left
-
*************************************
Yardstick Test Case Description TC001
*************************************
+--------------+------+----------------------------------+--------------------+
|test verdict | Fails only if SLA is not passed, or if there is a test case |
| | execution problem. |
-+--------------+--------------------------------------------------------------+
\ No newline at end of file
++--------------+--------------------------------------------------------------+
-.. image:: ../../etc/opnfv-logo.png
- :height: 40
- :width: 200
- :alt: OPNFV
- :align: left
-
*************************************
Yardstick Test Case Description TC002
*************************************
-
*************************************
Yardstick Test Case Description TC012
*************************************
+--------------+--------------------------------------------------------------+
|configuration | File: opnfv_yardstick_tc012.yaml |
| | |
-| | * SLA (optional): 15000 (MBps) |
+| | * SLA (optional): 15000 (MBps) |
| | min_bw: The minimum amount of memory bandwidth that is |
| | accepted. |
| | * Size: 10 240 kB - test allocates twice that size (20 480kB)|
|references | * http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html |
| | |
| | * McVoy, Larry W., and Carl Staelin. "lmbench: Portable Tools|
-| | for Performance Analysis." *USENIX annual technical |
-| | conference*. 1996. |
+| | for Performance Analysis." |
+| | * USENIX annual technical conference. 1996. |
+--------------+--------------------------------------------------------------+
|applicability | Test can be configured with different |
| | * memory sizes; |
| | fcp, bzero, bcopy); |
| | * number of warmup iterations; |
| | * iterations and intervals. |
+| | |
| | There are default values for each above-mentioned option. |
+--------------+--------------------------------------------------------------+
|pre-test | The test case image needs to be installed into Glance |
Code Review / yardstick.git / commitdiff