FROM ubuntu:18.04
-LABEL image=opnfv/yardstick
+LABEL image=opnfv/yardstick-ubuntu-18.04
ARG BRANCH=master
.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Ericsson AB and others.
+ License.
+ http://creativecommons.org/licenses/by/4.0
+ (c) OPNFV, Ericsson AB and others.
+
+ Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
============
Introduction
About This Document
-===================
+-------------------
This document consists of the following chapters:
cases.
Contact Yardstick
-=================
+-----------------
Feedback? `Contact us`_
.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Ericsson AB and others.
+ License.
+ http://creativecommons.org/licenses/by/4.0
+ (c) OPNFV, Ericsson AB and others.
+
+ Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
===========
Methodology
===========
Abstract
-========
+--------
This chapter describes the methodology implemented by the Yardstick project for
verifying the :term:`NFVI` from the perspective of a :term:`VNF`.
ETSI-NFV
-========
+--------
.. _NFV-TST001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf
.. _Yardsticktst: https://wiki.opnfv.org/download/attachments/2925202/opnfv_summit_-_bridging_opnfv_and_etsi.pdf?version=1&modificationDate=1458848320000&api=v2
.. seealso:: Yardsticktst_ for material on alignment ETSI TST001 and Yardstick.
Metrics
-=======
+-------
The metrics, as defined by ETSI GS NFV-TST001, are shown in
:ref:`Table1 <table2_1>`, :ref:`Table2 <table2_2>` and
.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) 2016 Huawei Technologies Co.,Ltd and others
+ License.
+ http://creativecommons.org/licenses/by/4.0
+ (c) 2016 Huawei Technologies Co.,Ltd and others
+
+ Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
============
Architecture
============
Abstract
-========
-This chapter describes the yardstick framework software architecture. We will
-introduce it from Use-Case View, Logical View, Process View and Deployment
+--------
+
+This chapter describes the Yardstick framework software architecture. We will
+introduce it from Use Case View, Logical View, Process View and Deployment
View. More technical details will be introduced in this chapter.
Overview
-========
+--------
Architecture overview
----------------------
+^^^^^^^^^^^^^^^^^^^^^
Yardstick is mainly written in Python, and test configurations are made
in YAML. Documentation is written in reStructuredText format, i.e. .rst
files. Yardstick is inspired by Rally. Yardstick is intended to run on a
Concept
--------
+^^^^^^^
+
**Benchmark** - assess the relative performance of something
**Benchmark** configuration file - describes a single test case in yaml format
Runner types
-------------
+^^^^^^^^^^^^
There exists several predefined runner types to choose between when designing
a test scenario:
Use-Case View
-=============
+-------------
+
Yardstick Use-Case View shows two kinds of users. One is the Tester who will
do testing in cloud, the other is the User who is more concerned with test
result and result analyses.
:alt: Yardstick Use-Case View
Logical View
-============
+------------
+
Yardstick Logical View describes the most important classes, their
organization, and the most important use-case realizations.
:alt: Yardstick framework architecture in Danube
Process View (Test execution flow)
-==================================
+----------------------------------
+
Yardstick process view shows how yardstick runs a test case. Below is the
sequence graph about the test execution flow using heat context, and each
object represents one module in yardstick:
undepoyed, the whole test ends.
Deployment View
-===============
+---------------
+
Yardstick deployment view shows how the yardstick tool can be deployed into the
underlying platform. Generally, yardstick tool is installed on JumpServer(see
`07-installation` for detail installation steps), and JumpServer is
:alt: Yardstick Deployment View
Yardstick Directory structure
-=============================
+-----------------------------
**yardstick/** - Yardstick main directory.
.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others.
+ License.
+ http://creativecommons.org/licenses/by/4.0
+ (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others.
+
+
+ Convention for heading levels in Yardstick documentation:
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
..
Convention for heading levels in Yardstick documentation:
Yardstick Installation
======================
-
Yardstick supports installation by Docker or directly in Ubuntu. The
installation procedure for Docker and direct installation are detailed in
the sections below.
and then configure Yardstick environments in the Yardstick container.
Using the CLI command ``env prepare`` (first way) (**recommended**)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
In the Yardstick container, the Yardstick repository is located in the
``/home/opnfv/repos`` directory. Yardstick provides a CLI to prepare OpenStack
Manually exporting the env variables and initializing OpenStack (second way)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Export OpenStack environment variables
-######################################
+''''''''''''''''''''''''''''''''''''''
Before running Yardstick it is necessary to export OpenStack environment
variables::
Manual creation of Yardstick flavor and guest images
-####################################################
+''''''''''''''''''''''''''''''''''''''''''''''''''''
Before executing Yardstick test cases, make sure that Yardstick flavor and
guest image are available in OpenStack. Detailed steps about creating the
Automatic initialization of OpenStack (third way)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
++++++++++++++++++++++++++++++++++++++++++++++++++
Similar to the second way, the first step is also to
`Export OpenStack environment variables`_. Then the following steps should be
done.
Automatic creation of Yardstick flavor and guest images
-#######################################################
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''
Yardstick has a script for automatically creating Yardstick flavor and building
Yardstick guest images. This script is mainly used for CI and can be also used
not available. You need to prepare OpenStack environment variables and create
Yardstick flavor and guest images manually.
-
Uninstall Yardstick
^^^^^^^^^^^^^^^^^^^
Now Yardstick will store results in InfluxDB when you run a testcase.
-
Deploy InfluxDB and Grafana directly in Ubuntu (**Todo**)
---------------------------------------------------------
-
Proxy Support
-------------
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others.
+.. Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
+
===================================
Installing a plug-in into Yardstick
===================================
Abstract
-========
+--------
Yardstick provides a ``plugin`` CLI command to support integration with other
OPNFV testing projects. Below is an example invocation of Yardstick plugin
Installing Storperf into Yardstick
-==================================
+----------------------------------
Storperf is delivered as a Docker container from
https://hub.docker.com/r/opnfv/storperf/tags/.
Step 0: Environment preparation
--------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Running Storperf on Jump Host
Requirements:
deployed host.
Step 1: Plug-in configuration file preparation
-----------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To install a plug-in, first you need to prepare a plug-in configuration file in
YAML format and store it in the "plugin" directory. The plugin configration
in my local environment.
Step 2: Plug-in install/remove scripts preparation
---------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In ``yardstick/resource/scripts`` directory, there are two folders: an
``install`` folder and a ``remove`` folder. You need to store the plug-in
``storperf.bash``.
Step 3: Install and remove Storperf
------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To install Storperf, simply execute the following command::
# Install Storperf
yardstick plugin install plugin/storperf.yaml
-Removing Storperf from yardstick
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Removing Storperf from Yardstick
+++++++++++++++++++++++++++++++++
To remove Storperf, simply execute the following command::
.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, 2016 Huawei Technologies Co.,Ltd and others.
+ License.
+ http://creativecommons.org/licenses/by/4.0
+ (c) OPNFV, 2016 Huawei Technologies Co.,Ltd and others.
+ Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
==============================================
Store Other Project's Test Results in InfluxDB
==============================================
Abstract
-========
+--------
.. _Framework: https://wiki.opnfv.org/download/attachments/6827660/wiki.png?version=1&modificationDate=1470298075000&api=v2
:alt: Store Other Project's Test Results in InfluxDB
Store Storperf Test Results into Community's InfluxDB
-=====================================================
+-----------------------------------------------------
.. _Influxdb: https://git.opnfv.org/cgit/yardstick/tree/yardstick/dispatcher/influxdb.py
.. _Mingjiang: mailto:limingjiang@huawei.com
.. http://creativecommons.org/licenses/by/4.0
.. (c) 2016 Huawei Technologies Co.,Ltd and others
+.. Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
+
=================
Grafana dashboard
=================
Abstract
-========
+--------
This chapter describes the Yardstick grafana dashboard. The Yardstick grafana
dashboard can be found here: http://testresults.opnfv.org/grafana/
Public access
-=============
+-------------
Yardstick provids a public account for accessing to the dashboard. The username
and password are both set to ‘opnfv’.
Testcase dashboard
-==================
+------------------
For each test case, there is a dedicated dashboard. Shown here is the dashboard
of TC002.
Administration access
-=====================
+---------------------
For a user with administration rights it is easy to update and save any
dashboard configuration. Saved updates immediately take effect and become live.
Any change made by administrator should be careful.
-Add a dashboard into yardstick grafana
-======================================
+Add a dashboard into Yardstick Grafana
+--------------------------------------
Due to security concern, users that using the public opnfv account are not able
-to edit the yardstick grafana directly.It takes a few more steps for a
+to edit the yardstick grafana directly. It takes a few more steps for a
non-yardstick user to add a custom dashboard into yardstick grafana.
There are 6 steps to go.
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+.. Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
=====================
Yardstick Restful API
Abstract
-========
+--------
Yardstick support restful API since Danube.
Available API
-=============
+-------------
/yardstick/env/action
----------------------
+^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to prepare Yardstick test environment.
For Euphrates, it supports:
/yardstick/asynctask
---------------------
+^^^^^^^^^^^^^^^^^^^^
Description: This API is used to get the status of asynchronous tasks
/yardstick/testcases
---------------------
+^^^^^^^^^^^^^^^^^^^^
Description: This API is used to list all released Yardstick test cases.
/yardstick/testcases/release/action
------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to run a Yardstick released test case.
/yardstick/testcases/samples/action
------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to run a Yardstick sample test case.
/yardstick/testcases/<testcase_name>/docs
------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to the documentation of a certain released test
case.
/yardstick/testsuites/action
-----------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to run a Yardstick test suite.
/yardstick/tasks/<task_id>/log
-------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to get the real time log of test case execution.
/yardstick/results
-------------------
+^^^^^^^^^^^^^^^^^^
Description: This API is used to get the test results of tasks. If you call
/yardstick/testcases/samples/action API, it will return a task id. You can use
/api/v2/yardstick/openrcs
--------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API provides functionality of handling OpenStack credential
file (openrc). For Euphrates, it supports:
/api/v2/yardstick/openrcs/<openrc_id>
--------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API provides functionality of handling OpenStack credential file (openrc). For Euphrates, it supports:
/api/v2/yardstick/pods
-----------------------
+^^^^^^^^^^^^^^^^^^^^^^
Description: This API provides functionality of handling Yardstick pod file
(pod.yaml). For Euphrates, it supports:
/api/v2/yardstick/pods/<pod_id>
--------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API provides functionality of handling Yardstick pod file (pod.yaml). For Euphrates, it supports:
/api/v2/yardstick/images
-------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to Yardstick VM images.
For Euphrates, it supports:
/api/v2/yardstick/images/<image_id>
------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to Yardstick VM images. For Euphrates, it supports:
/api/v2/yardstick/tasks
------------------------
+^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to yardstick tasks. For
Euphrates, it supports:
/api/v2/yardstick/tasks/<task_id>
----------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to yardstick tasks. For Euphrates, it supports:
/api/v2/yardstick/testcases
----------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to Yardstick testcases.
For Euphrates, it supports:
/api/v2/yardstick/testcases/<case_name>
----------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to yardstick testcases. For Euphrates, it supports:
/api/v2/yardstick/testsuites
-----------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to yardstick test suites.
For Euphrates, it supports:
/api/v2/yardstick/testsuites
-----------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to yardstick test suites. For Euphrates, it supports:
/api/v2/yardstick/projects
---------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to Yardstick test
projects. For Euphrates, it supports:
/api/v2/yardstick/projects
---------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to yardstick test projects. For Euphrates, it supports:
/api/v2/yardstick/containers
-----------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to Docker containers.
For Euphrates, it supports:
/api/v2/yardstick/containers/<container_id>
--------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Description: This API is used to do some work related to Docker containers. For Euphrates, it supports:
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+
+.. Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
+
========================
Yardstick User Interface
========================
Commands
-========
+--------
To generate the compact HTML report, run::
Description
-===========
+-----------
1. When the command is triggered, the relevant values for the
provided task-id and testcase name are retrieved from the
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, 2016-2017 Intel Corporation.
+.. Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
+
===================================
Network Services Benchmarking (NSB)
===================================
-Abstract
-========
-
.. _Yardstick: https://wiki.opnfv.org/display/yardstick
+.. _`ETSI GS NFV-TST001`: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_nfv-tst001v010101p.pdf
+
+Abstract
+--------
This chapter provides an overview of the NSB, a contribution to OPNFV
Yardstick_ from Intel.
Overview
-========
-
-The goal of NSB is to Extend Yardstick to perform real world VNFs and NFVi
-Characterization and benchmarking with repeatable and deterministic methods.
-
-The Network Service Benchmarking (NSB) extends the yardstick framework to do
-VNF characterization and benchmarking in three different execution
-environments - bare metal i.e. native Linux environment, standalone virtual
-environment and managed virtualized environment (e.g. Open stack etc.).
-It also brings in the capability to interact with external traffic generators
-both hardware & software based for triggering and validating the traffic
-according to user defined profiles.
+--------
+
+Network Services Benchmarking (:term:`NSB`) uses the :term:`Yardstick`
+framework for performing :term:`VNF` and :term:`NFVI` characterisation in an
+:term:`NFV` environment.
+
+For VNF characterisation, NSB will onboard a VNF, source and sink traffic to it
+via traffic generators, and collect a variety of key performance indicators
+(:term:`KPI`) during VNF execution. The stream of KPI data is stored in a
+database, and it is visualized in a performance-visualization dashboard.
+
+For NFVI characterisation, a fixed test VNF, called :term:`PROX` is used.
+PROX implements a suite of test cases and visualizes the output data of the
+test suite. The PROX test cases implement various execution kernels found in
+real-world VNFs, and the output of the test cases provides an indication of
+the fitness of the infrastructure for running NFV services, in addition to
+indicating potential performance optimizations for the NFVI.
+
+NSB extends the Yardstick framework to do VNF characterization in three
+different execution environments - bare metal i.e. native Linux environment,
+standalone virtual environment and managed virtualized environment (e.g.
+OpenStack). It also brings in the capability to interact with external traffic
+generators, both hardware and software based, for triggering and validating the
+traffic according to user defined profiles.
NSB extension includes:
- - Generic data models of Network Services, based on ETSI spec `ETSI GS NFV-TST 001 <http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_nfv-tst001v010101p.pdf>`_
-
- - New Standalone context for VNF testing like SRIOV, OVS, OVS-DPDK etc
-
- - Generic VNF configuration models and metrics implemented with Python
- classes
-
- - Traffic generator features and traffic profiles
-
- - L1-L3 state-less traffic profiles
-
- - L4-L7 state-full traffic profiles
-
- - Tunneling protocol / network overlay support
-
- - Test case samples
+* Generic data models of Network Services, based on ETSI spec
+ `ETSI GS NFV-TST 001`_
+* Standalone :term:`context` for VNF testing with SRIOV, OVS, OVS-DPDK, etc
+* Generic VNF configuration models and metrics implemented with Python
+ classes
+* Traffic generator features and traffic profiles
- - Ping
+ * L1-L3 stateless traffic profiles
+ * L4-L7 state-full traffic profiles
+ * Tunneling protocol/network overlay support
- - Trex
+* Test case samples
- - vPE,vCGNAT, vFirewall etc - ipv4 throughput, latency etc
+ * Ping
+ * Trex
+ * vPE, vCGNAT, vFirewall etc - ipv4 throughput, latency etc
- - Traffic generators like Trex, ab/nginx, ixia, iperf etc
+* Traffic generators i.e. Trex, ab/nginx, ixia, iperf, etc
+* KPIs for a given use case:
- - KPIs for a given use case:
+ * System agent support for collecting NFVi KPI. This includes:
- - System agent support for collecting NFVi KPI. This includes:
+ * CPU statistic
+ * Memory BW
+ * OVS-DPDK Stats
- - CPU statistic
-
- - Memory BW
-
- - OVS-DPDK Stats
-
- - Network KPIs, e.g., inpackets, outpackets, thoughput, latency etc
-
- - VNF KPIs, e.g., packet_in, packet_drop, packet_fwd etc
+ * Network KPIs e.g. inpackets, outpackets, thoughput, latency
+ * VNF KPIs e.g. packet_in, packet_drop, packet_fwd
Architecture
-============
+------------
The Network Service (NS) defines a set of Virtual Network Functions (VNF)
connected together using NFV infrastructure.
configuration parameters, number of connection points used and flavor including
core and memory amount.
-The ETSI defines a Network Service as a set of configurable VNFs working in
-some NFV Infrastructure connecting each other using Virtual Links available
-through Connection Points. The ETSI MANO specification defines a set of
-management entities called Network Service Descriptors (NSD) and
-VNF Descriptors (VNFD) that define real Network Service. The picture below
-makes an example how the real Network Operator use-case can map into ETSI
-Network service definition
-
-Network Service framework performs the necessary test steps. It may involve
-
- - Interacting with traffic generator and providing the inputs on traffic
- type / packet structure to generate the required traffic as per the
- test case. Traffic profiles will be used for this.
-
- - Executing the commands required for the test procedure and analyses the
- command output for confirming whether the command got executed correctly
- or not. E.g. As per the test case, run the traffic for the given
- time period / wait for the necessary time delay
-
- - Verify the test result.
-
- - Validate the traffic flow from SUT
-
- - Fetch the table / data from SUT and verify the value as per the test case
-
- - Upload the logs from SUT onto the Test Harness server
-
- - Read the KPI's provided by particular VNF
+ETSI defines a Network Service as a set of configurable VNFs working in some
+NFV Infrastructure connecting each other using Virtual Links available through
+Connection Points. The ETSI MANO specification defines a set of management
+entities called Network Service Descriptors (NSD) and VNF Descriptors (VNFD)
+that define real Network Service. The picture below makes an example how the
+real Network Operator use-case can map into ETSI Network service definition.
+
+Network Service framework performs the necessary test steps. It may involve:
+
+* Interacting with traffic generator and providing the inputs on traffic
+ type / packet structure to generate the required traffic as per the
+ test case. Traffic profiles will be used for this.
+* Executing the commands required for the test procedure and analyses the
+ command output for confirming whether the command got executed correctly
+ or not e.g. as per the test case, run the traffic for the given
+ time period and wait for the necessary time delay.
+* Verify the test result.
+* Validate the traffic flow from SUT.
+* Fetch the data from SUT and verify the value as per the test case.
+* Upload the logs from SUT onto the Test Harness server
+* Retrieve the KPI's provided by particular VNF
Components of Network Service
------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- * *Models for Network Service benchmarking*: The Network Service benchmarking
- requires the proper modelling approach. The NSB provides models using Python
- files and defining of NSDs and VNFDs.
+* *Models for Network Service benchmarking*: The Network Service benchmarking
+ requires the proper modelling approach. The NSB provides models using Python
+ files and defining of NSDs and VNFDs.
- The benchmark control application being a part of OPNFV yardstick can call
- that python models to instantiate and configure the VNFs. Depending on
- infrastructure type (bare-metal or fully virtualized) that calls could be
- made directly or using MANO system.
+The benchmark control application being a part of OPNFV Yardstick can call
+that Python models to instantiate and configure the VNFs. Depending on
+infrastructure type (bare-metal or fully virtualized) that calls could be
+made directly or using MANO system.
- * *Traffic generators in NSB*: Any benchmark application requires a set of
- traffic generator and traffic profiles defining the method in which traffic
- is generated.
+* *Traffic generators in NSB*: Any benchmark application requires a set of
+ traffic generator and traffic profiles defining the method in which traffic
+ is generated.
- The Network Service benchmarking model extends the Network Service
- definition with a set of Traffic Generators (TG) that are treated
- same way as other VNFs being a part of benchmarked network service.
- Same as other VNFs the traffic generator are instantiated and terminated.
+The Network Service benchmarking model extends the Network Service
+definition with a set of Traffic Generators (TG) that are treated
+same way as other VNFs being a part of benchmarked network service.
+Same as other VNFs the traffic generator are instantiated and terminated.
- Every traffic generator has own configuration defined as a traffic profile
- and a set of KPIs supported. The python models for TG is extended by
- specific calls to listen and generate traffic.
+Every traffic generator has own configuration defined as a traffic profile
+and a set of KPIs supported. The python models for TG is extended by
+specific calls to listen and generate traffic.
- * *The stateless TREX traffic generator*: The main traffic generator used as
- Network Service stimulus is open source TREX tool.
+* *The stateless TREX traffic generator*: The main traffic generator used as
+ Network Service stimulus is open source TREX tool.
- The TREX tool can generate any kind of stateless traffic.
+The TREX tool can generate any kind of stateless traffic.
- .. code-block:: console
-
- +--------+ +-------+ +--------+
- | | | | | |
- | Trex | ---> | VNF | ---> | Trex |
- | | | | | |
- +--------+ +-------+ +--------+
-
- Supported testcases scenarios:
+.. code-block:: console
- - Correlated UDP traffic using TREX traffic generator and replay VNF.
+ +--------+ +-------+ +--------+
+ | | | | | |
+ | Trex | ---> | VNF | ---> | Trex |
+ | | | | | |
+ +--------+ +-------+ +--------+
- - using different IMIX configuration like pure voice, pure video traffic etc
+Supported testcases scenarios:
- - using different number IP flows like 1 flow, 1K, 16K, 64K, 256K, 1M flows
+* Correlated UDP traffic using TREX traffic generator and replay VNF.
- - Using different number of rules configured like 1 rule, 1K, 10K rules
+ * using different IMIX configuration like pure voice, pure video traffic etc
+ * using different number IP flows e.g. 1, 1K, 16K, 64K, 256K, 1M flows
+ * Using different number of rules configured e.g. 1, 1K, 10K rules
- For UDP correlated traffic following Key Performance Indicators are collected
- for every combination of test case parameters:
+For UDP correlated traffic following Key Performance Indicators are collected
+for every combination of test case parameters:
- - RFC2544 throughput for various loss rate defined (1% is a default)
+* RFC2544 throughput for various loss rate defined (1% is a default)
Graphical Overview
-==================
+------------------
-NSB Testing with yardstick framework facilitate performance testing of various
+NSB Testing with Yardstick framework facilitate performance testing of various
VNFs provided.
.. code-block:: console
+-----------+
- | | +-----------+
- | vPE | ->|TGen Port 0|
- | TestCase | | +-----------+
- | | |
- +-----------+ +------------------+ +-------+ |
- | | -- API --> | VNF | <--->
- +-----------+ | Yardstick | +-------+ |
- | Test Case | --> | NSB Testing | |
- +-----------+ | | |
- | | | |
- | +------------------+ |
- +-----------+ | +-----------+
- | Traffic | ->|TGen Port 1|
- | patterns | +-----------+
+ | | +-------------+
+ | vPE | -->| TGen Port 0 |
+ | TestCase | | +-------------+
+ | | |
+ +-----------+ +---------------+ +-------+ |
+ | | ---> | VNF | <--->
+ +-----------+ | Yardstick | +-------+ |
+ | Test Case | --> | NSB Testing | |
+ +-----------+ | | |
+ | | | |
+ | +---------------+ |
+ +-----------+ | +-------------+
+ | Traffic | -->| TGen Port 1 |
+ | patterns | +-------------+
+-----------+
Figure 1: Network Service - 2 server configuration
-VNFs supported for chracterization:
------------------------------------
+VNFs supported for chracterization
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1. CGNAPT - Carrier Grade Network Address and port Translation
2. vFW - Virtual Firewall
3. vACL - Access Control List
-4. Prox - Packet pROcessing eXecution engine:
- - VNF can act as Drop, Basic Forwarding (no touch),
- L2 Forwarding (change MAC), GRE encap/decap, Load balance based on
- packet fields, Symmetric load balancing
- - QinQ encap/decap IPv4/IPv6, ARP, QoS, Routing, Unmpls, Policing, ACL
+4. PROX - Packet pROcessing eXecution engine:
+ * VNF can act as Drop, Basic Forwarding (no touch),
+ L2 Forwarding (change MAC), GRE encap/decap, Load balance based on
+ packet fields, Symmetric load balancing
+ * QinQ encap/decap IPv4/IPv6, ARP, QoS, Routing, Unmpls, Policing, ACL
5. UDP_Replay
======= Heading 0 (reserved for the title in a document)
------- Heading 1
- ~~~~~~~ Heading 2
+ ^^^^^^^ Heading 2
+++++++ Heading 3
''''''' Heading 4
Avoid deeper levels because they do not render well.
-=====================================
-Yardstick - NSB Testing -Installation
-=====================================
+
+================
+NSB Installation
+================
+
+.. _OVS-DPDK: http://docs.openvswitch.org/en/latest/intro/install/dpdk/
+.. _devstack: https://docs.openstack.org/devstack/pike/>
Abstract
--------
-The Network Service Benchmarking (NSB) extends the yardstick framework to do
-VNF characterization and benchmarking in three different execution
-environments viz., bare metal i.e. native Linux environment, standalone virtual
-environment and managed virtualized environment (e.g. Open stack etc.).
-It also brings in the capability to interact with external traffic generators
-both hardware & software based for triggering and validating the traffic
-according to user defined profiles.
-
The steps needed to run Yardstick with NSB testing are:
* Install Yardstick (NSB Testing).
-* Setup/Reference pod.yaml describing Test topology
-* Create/Reference the test configuration yaml file.
+* Setup/reference ``pod.yaml`` describing Test topology
+* Create/reference the test configuration yaml file.
* Run the test case.
-
Prerequisites
-------------
-Refer chapter Yardstick Installation for more information on yardstick
-prerequisites
+Refer to :doc:`04-installation` for more information on Yardstick
+prerequisites.
Several prerequisites are needed for Yardstick (VNF testing):
* intel-cmt-cat
Hardware & Software Ingredients
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SUT requirements:
-
======= ===================
Item Description
======= ===================
Boot and BIOS settings:
-
============= =================================================
Boot settings default_hugepagesz=1G hugepagesz=1G hugepages=16
hugepagesz=2M hugepages=2048 isolcpus=1-11,22-33
Turbo Boost Disabled
============= =================================================
-
-
Install Yardstick (NSB Testing)
-------------------------------
-Download the source code and install Yardstick from it
+Download the source code and check out the latest stable branch::
.. code-block:: console
git clone https://gerrit.opnfv.org/gerrit/yardstick
-
cd yardstick
-
# Switch to latest stable branch
- # git checkout <tag or stable branch>
- git checkout stable/euphrates
+ git checkout stable/gambia
Configure the network proxy, either using the environment variables or setting
-the global environment file:
+the global environment file.
+
+* Set environment
-.. code-block:: ini
+.. code-block::
- cat /etc/environment
http_proxy='http://proxy.company.com:port'
https_proxy='http://proxy.company.com:port'
export http_proxy='http://proxy.company.com:port'
export https_proxy='http://proxy.company.com:port'
-The last step is to modify the Yardstick installation inventory, used by
-Ansible:
-
-.. code-block:: ini
+Modify the Yardstick installation inventory, used by Ansible::
cat ./ansible/install-inventory.ini
[jumphost]
- localhost ansible_connection=local
+ localhost ansible_connection=local
[yardstick-standalone]
yardstick-standalone-node ansible_host=192.168.1.2
.. note::
- SSH access without password needs to be configured for all your nodes defined in
- ``install-inventory.ini`` file.
- If you want to use password authentication you need to install sshpass
-
- .. code-block:: console
+ SSH access without password needs to be configured for all your nodes
+ defined in ``yardstick-install-inventory.ini`` file.
+ If you want to use password authentication you need to install ``sshpass``::
sudo -EH apt-get install sshpass
-To execute an installation for a Bare-Metal or a Standalone context:
-
-.. code-block:: console
+To execute an installation for a BareMetal or a Standalone context::
./nsb_setup.sh
-To execute an installation for an OpenStack context:
-
-.. code-block:: console
+To execute an installation for an OpenStack context::
./nsb_setup.sh <path to admin-openrc.sh>
-Above command setup docker with latest yardstick code. To execute
-
-.. code-block:: console
+The above commands will set up Docker with the latest Yardstick code. To
+execute::
docker exec -it yardstick bash
It will also automatically download all the packages needed for NSB Testing
-setup. Refer chapter :doc:`04-installation` for more on docker
+setup. Refer chapter :doc:`04-installation` for more on Docker
+
**Install Yardstick using Docker (recommended)**
Another way to execute an installation for a Bare-Metal or a Standalone context
Environment parameters and credentials
--------------------------------------
-Config yardstick conf
-~~~~~~~~~~~~~~~~~~~~~
+Configure yardstick.conf
+^^^^^^^^^^^^^^^^^^^^^^^^
-If user did not run 'yardstick env influxdb' inside the container, which will
-generate correct ``yardstick.conf``, then create the config file manually (run
-inside the container):
-::
+If you did not run ``yardstick env influxdb`` inside the container to generate
+ ``yardstick.conf``, then create the config file manually (run inside the
+container)::
cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf
vi /etc/yardstick/yardstick.conf
-Add trex_path, trex_client_lib and bin_path in 'nsb' section.
-
-::
+Add ``trex_path``, ``trex_client_lib`` and ``bin_path`` to the ``nsb``
+section::
[DEFAULT]
debug = True
- dispatcher = file, influxdb
+ dispatcher = influxdb
[dispatcher_influxdb]
timeout = 5
Run Yardstick - Network Service Testcases
-----------------------------------------
-
NS testing - using yardstick CLI
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
See :doc:`04-installation`
-.. code-block:: console
-
+Connect to the Yardstick container::
docker exec -it yardstick /bin/bash
- source /etc/yardstick/openstack.creds (only for heat TC if nsb_setup.sh was NOT used)
- export EXTERNAL_NETWORK="<openstack public network>" (only for heat TC)
+
+If you're running ``heat`` testcases and ``nsb_setup.sh`` was not used::
+ source /etc/yardstick/openstack.creds
+
+In addition to the above, you need to se the ``EXTERNAL_NETWORK`` for
+OpenStack::
+
+ export EXTERNAL_NETWORK="<openstack public network>"
+
+Finally, you should be able to run the testcase::
+
yardstick --debug task start yardstick/samples/vnf_samples/nsut/<vnf>/<test case>
Network Service Benchmarking - Bare-Metal
-----------------------------------------
Bare-Metal Config pod.yaml describing Topology
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Bare-Metal 2-Node setup
+++++++++++++++++++++++
Bare-Metal Config pod.yaml
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-Before executing Yardstick test cases, make sure that pod.yaml reflects the
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+Before executing Yardstick test cases, make sure that ``pod.yaml`` reflects the
topology and update all the required fields.::
cp /etc/yardstick/nodes/pod.yaml.nsb.sample /etc/yardstick/nodes/pod.yaml
if: "xe1"
-Network Service Benchmarking - Standalone Virtualization
---------------------------------------------------------
+Standalone Virtualization
+-------------------------
SR-IOV
-~~~~~~
+^^^^^^
SR-IOV Pre-requisites
+++++++++++++++++++++
On Host, where VM is created:
- a) Create and configure a bridge named ``br-int`` for VM to connect to external network.
- Currently this can be done using VXLAN tunnel.
+ a) Create and configure a bridge named ``br-int`` for VM to connect to
+ external network. Currently this can be done using VXLAN tunnel.
- Execute the following on host, where VM is created:
-
- .. code-block:: console
+ Execute the following on host, where VM is created::
ip link add type vxlan remote <Jumphost IP> local <DUT IP> id <ID: 10> dstport 4789
brctl addbr br-int
ip addr add <IP#1, like: 172.20.2.1/24> dev br-int
ip link set dev br-int up
- .. note:: May be needed to add extra rules to iptable to forward traffic.
+ .. note:: You may need to add extra rules to iptable to forward traffic.
.. code-block:: console
Yardstick has a tool for building this custom image with SampleVNF.
It is necessary to have ``sudo`` rights to use this tool.
- Also you may need to install several additional packages to use this tool, by
- following the commands below::
+ Also you may need to install several additional packages to use this tool, by
+ following the commands below::
- sudo apt-get update && sudo apt-get install -y qemu-utils kpartx
+ sudo apt-get update && sudo apt-get install -y qemu-utils kpartx
- This image can be built using the following command in the directory where Yardstick is installed
+ This image can be built using the following command in the directory where
+ Yardstick is installed::
- .. code-block:: console
+ export YARD_IMG_ARCH='amd64'
+ sudo echo "Defaults env_keep += \'YARD_IMG_ARCH\'" >> /etc/sudoers
- export YARD_IMG_ARCH='amd64'
- sudo echo "Defaults env_keep += \'YARD_IMG_ARCH\'" >> /etc/sudoers
+ For instructions on generating a cloud image using Ansible, refer to
+ :doc:`04-installation`.
- Please use ansible script to generate a cloud image refer to :doc:`04-installation`
+ for more details refer to chapter :doc:`04-installation`
- for more details refer to chapter :doc:`04-installation`
-
- .. note:: VM should be build with static IP and should be accessible from yardstick host.
+ .. note:: VM should be build with static IP and be accessible from the
+ Yardstick host.
SR-IOV Config pod.yaml describing Topology
+----------+ +-------------------------+
| | | ^ ^ |
| | | | | |
- | | (0)<----->(0) | ------ | |
- | TG1 | | SUT | |
- | | | | |
- | | (n)<----->(n) |------------------ |
+ | | (0)<----->(0) | ------ SUT | |
+ | TG1 | | | |
+ | | (n)<----->(n) | ----------------- |
+ | | | |
+----------+ +-------------------------+
trafficgen_1 host
++++++++++++++++++++++++++++++++++++++++
.. code-block:: console
- +--------------------+
- | |
- | |
- | DUT |
- | (VNF) |
- | |
- +--------------------+
- | VF NIC | | VF NIC |
- +--------+ +--------+
- ^ ^
- | |
- | |
- +----------+ +-------------------------+ +--------------+
- | | | ^ ^ | | |
- | | | | | | | |
- | | (0)<----->(0) | ------ | | | TG2 |
- | TG1 | | SUT | | | (UDP Replay) |
- | | | | | | |
- | | (n)<----->(n) | ------ | (n)<-->(n) | |
- +----------+ +-------------------------+ +--------------+
- trafficgen_1 host trafficgen_2
-
-Before executing Yardstick test cases, make sure that pod.yaml reflects the
+ +--------------------+
+ | |
+ | |
+ | DUT |
+ | (VNF) |
+ | |
+ +--------------------+
+ | VF NIC | | VF NIC |
+ +--------+ +--------+
+ ^ ^
+ | |
+ | |
+ +----------+ +---------------------+ +--------------+
+ | | | ^ ^ | | |
+ | | | | | | | |
+ | | (0)<----->(0) |----- | | | TG2 |
+ | TG1 | | SUT | | | (UDP Replay) |
+ | | | | | | |
+ | | (n)<----->(n) | -----| (n)<-->(n) | |
+ +----------+ +---------------------+ +--------------+
+ trafficgen_1 host trafficgen_2
+
+Before executing Yardstick test cases, make sure that ``pod.yaml`` reflects the
topology and update all the required fields.
.. code-block:: console
SR-IOV testcase update:
``<yardstick>/samples/vnf_samples/nsut/vfw/tc_sriov_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``
-Update "contexts" section
-'''''''''''''''''''''''''
+Update contexts section
+'''''''''''''''''''''''
.. code-block:: YAML
gateway_ip: '152.16.100.20'
-
OVS-DPDK
-~~~~~~~~
+^^^^^^^^
OVS-DPDK Pre-requisites
-~~~~~~~~~~~~~~~~~~~~~~~
++++++++++++++++++++++++
On Host, where VM is created:
- a) Create and configure a bridge named ``br-int`` for VM to connect to external network.
- Currently this can be done using VXLAN tunnel.
+ a) Create and configure a bridge named ``br-int`` for VM to connect to
+ external network. Currently this can be done using VXLAN tunnel.
Execute the following on host, where VM is created:
Yardstick has a tool for building this custom image with SampleVNF.
It is necessary to have ``sudo`` rights to use this tool.
- Also you may need to install several additional packages to use this tool, by
- following the commands below::
+ You may need to install several additional packages to use this tool, by
+ following the commands below::
- sudo apt-get update && sudo apt-get install -y qemu-utils kpartx
+ sudo apt-get update && sudo apt-get install -y qemu-utils kpartx
- This image can be built using the following command in the directory where Yardstick is installed::
+ This image can be built using the following command in the directory where
+ Yardstick is installed::
- export YARD_IMG_ARCH='amd64'
- sudo echo "Defaults env_keep += \'YARD_IMG_ARCH\'" >> /etc/sudoers
- sudo tools/yardstick-img-dpdk-modify tools/ubuntu-server-cloudimg-samplevnf-modify.sh
+ export YARD_IMG_ARCH='amd64'
+ sudo echo "Defaults env_keep += \'YARD_IMG_ARCH\'" >> /etc/sudoers
+ sudo tools/yardstick-img-dpdk-modify tools/ubuntu-server-cloudimg-samplevnf-modify.sh
- for more details refer to chapter :doc:`04-installation`
+ for more details refer to chapter :doc:`04-installation`
- .. note:: VM should be build with static IP and should be accessible from yardstick host.
+ .. note:: VM should be build with static IP and should be accessible from
+ yardstick host.
- c) OVS & DPDK version.
- - OVS 2.7 and DPDK 16.11.1 above version is supported
+3. OVS & DPDK version.
+ * OVS 2.7 and DPDK 16.11.1 above version is supported
- d) Setup OVS/DPDK on host.
- Please refer to below link on how to setup `OVS-DPDK <http://docs.openvswitch.org/en/latest/intro/install/dpdk/>`_
+4. Setup `OVS-DPDK`_ on host.
OVS-DPDK Config pod.yaml describing Topology
trafficgen_1 host trafficgen_2
-Before executing Yardstick test cases, make sure that pod.yaml reflects the
-topology and update all the required fields.
-
-.. code-block:: console
+Before executing Yardstick test cases, make sure that the ``pod.yaml`` reflects
+the topology and update all the required fields::
cp <yardstick>/etc/yardstick/nodes/standalone/trex_bm.yaml.sample /etc/yardstick/nodes/standalone/pod_trex.yaml
cp <yardstick>/etc/yardstick/nodes/standalone/host_ovs.yaml /etc/yardstick/nodes/standalone/host_ovs.yaml
ovs_dpdk testcase update:
``<yardstick>/samples/vnf_samples/nsut/vfw/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``
-Update "contexts" section
-'''''''''''''''''''''''''
+Update contexts section
+'''''''''''''''''''''''
.. code-block:: YAML
gateway_ip: '152.16.100.20'
-Network Service Benchmarking - OpenStack with SR-IOV support
-------------------------------------------------------------
+OpenStack with SR-IOV support
+-----------------------------
This section describes how to run a Sample VNF test case, using Heat context,
with SR-IOV. It also covers how to install OpenStack in Ubuntu 16.04, using
DevStack, with SR-IOV support.
-Single node OpenStack setup with external TG
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Single node OpenStack with external TG
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: console
Host pre-configuration
++++++++++++++++++++++
-.. warning:: The following configuration requires sudo access to the system. Make
- sure that your user have the access.
+.. warning:: The following configuration requires sudo access to the system.
+ Make sure that your user have the access.
-Enable the Intel VT-d or AMD-Vi extension in the BIOS. Some system manufacturers
-disable this extension by default.
+Enable the Intel VT-d or AMD-Vi extension in the BIOS. Some system
+manufacturers disable this extension by default.
Activate the Intel VT-d or AMD-Vi extension in the kernel by modifying the GRUB
config file ``/etc/default/grub``.
-For the Intel platform:
-
-.. code:: bash
+For the Intel platform::
...
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on"
...
-For the AMD platform:
-
-.. code:: bash
+For the AMD platform::
...
GRUB_CMDLINE_LINUX_DEFAULT="amd_iommu=on"
sudo update-grub
sudo reboot
-Make sure the extension has been enabled:
-
-.. code:: bash
+Make sure the extension has been enabled::
sudo journalctl -b 0 | grep -e IOMMU -e DMAR
Feb 06 14:50:14 hostname kernel: DMAR: dmar1: reg_base_addr e0ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de
Feb 06 14:50:14 hostname kernel: DMAR: DRHD base: 0x000000ee7fc000 flags: 0x0
+.. TODO: Refer to the yardstick installation guide for proxy set up
+
Setup system proxy (if needed). Add the following configuration into the
``/etc/environment`` file:
sudo -EH apt-get upgrade
sudo -EH apt-get dist-upgrade
-Install dependencies needed for the DevStack
+Install dependencies needed for DevStack
.. code:: bash
- sudo -EH apt-get install python
- sudo -EH apt-get install python-dev
- sudo -EH apt-get install python-pip
+ sudo -EH apt-get install python python-dev python-pip
Setup SR-IOV ports on the host:
DevStack installation
+++++++++++++++++++++
-Use official `Devstack <https://docs.openstack.org/devstack/pike/>`_
-documentation to install OpenStack on a host. Please note, that stable
-``pike`` branch of devstack repo should be used during the installation.
-The required `local.conf`` configuration file are described below.
+If you want to try out NSB, but don't have OpenStack set-up, you can use
+`Devstack`_ to install OpenStack on a host. Please note, that the
+``stable/pike`` branch of devstack repo should be used during the installation.
+The required ``local.conf`` configuration file are described below.
DevStack configuration file:
Start the devstack installation on a host.
-
TG host configuration
+++++++++++++++++++++
-Yardstick automatically install and configure Trex traffic generator on TG
+Yardstick automatically installs and configures Trex traffic generator on TG
host based on provided POD file (see below). Anyway, it's recommended to check
-the compatibility of the installed NIC on the TG server with software Trex using
-the manual at https://trex-tgn.cisco.com/trex/doc/trex_manual.html.
-
+the compatibility of the installed NIC on the TG server with software Trex
+using the `manual <https://trex-tgn.cisco.com/trex/doc/trex_manual.html>`_.
Run the Sample VNF test case
++++++++++++++++++++++++++++
OpenStack environment with SR-IOV support: ``samples/vnf_samples/nsut/vfw/
tc_heat_sriov_external_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``.
-Install yardstick using `Install Yardstick (NSB Testing)`_ steps for OpenStack
+Install Yardstick using `Install Yardstick (NSB Testing)`_ steps for OpenStack
context.
Create pod file for TG in the yardstick repo folder located in the yardstick
++++++++++++++++++++++++++++++++++++
Pre-configuration of the controller and compute hosts are the same as
-described in `Host pre-configuration`_ section. Follow the steps in the section.
-
+described in `Host pre-configuration`_ section.
DevStack configuration
++++++++++++++++++++++
-Use official `Devstack <https://docs.openstack.org/devstack/pike/>`_
-documentation to install OpenStack on a host. Please note, that stable
-``pike`` branch of devstack repo should be used during the installation.
-The required `local.conf`` configuration file are described below.
+A reference ``local.conf`` for deploying OpenStack in a multi-host environment
+using `Devstack`_ is shown in this section. The ``stable/pike`` branch of
+devstack repo should be used during the installation.
.. note:: Update the devstack configuration files by replacing angluar brackets
with a short description inside.
Start the devstack installation on the controller and compute hosts.
-
Run the sample vFW TC
+++++++++++++++++++++
-Install yardstick using `Install Yardstick (NSB Testing)`_ steps for OpenStack
+Install Yardstick using `Install Yardstick (NSB Testing)`_ steps for OpenStack
context.
-Run sample vFW RFC2544 SR-IOV TC (``samples/vnf_samples/nsut/vfw/
-tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``) in the heat
-context using steps described in `NS testing - using yardstick CLI`_ section
-and the following yardtick command line arguments:
+Run the sample vFW RFC2544 SR-IOV test case
+(``samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``)
+in the heat context using steps described in
+`NS testing - using yardstick CLI`_ section and the following Yardstick command
+line arguments:
.. code:: bash
samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
-Enabling other Traffic generator
---------------------------------
+Enabling other Traffic generators
+---------------------------------
IxLoad
~~~~~~
.. code-block:: console
- cp <repo>/etc/yardstick/nodes/pod.yaml.nsb.sample.ixia etc/yardstick/nodes/pod_ixia.yaml
+ cp <repo>/etc/yardstick/nodes/pod.yaml.nsb.sample.ixia \
+ etc/yardstick/nodes/pod_ixia.yaml
Config ``pod_ixia.yaml``
.. literalinclude:: code/pod_ixia.yaml
:language: console
- for sriov/ovs_dpdk pod files, please refer to above Standalone Virtualization for ovs-dpdk/sriov configuration
+ for sriov/ovs_dpdk pod files, please refer to `Standalone Virtualization`_
+ for ovs-dpdk/sriov configuration
3. Start IxOS TCL Server (Install 'Ixia IxExplorer IxOS <version>')
You will also need to configure the IxLoad machine to start the IXIA
* Go to:
``Start->Programs->Ixia->IxOS->IxOS 8.01-GA-Patch1->Ixia Tcl Server IxOS 8.01-GA-Patch1``
or
- ``"C:\Program Files (x86)\Ixia\IxOS\8.01-GA-Patch1\ixTclServer.exe"``
+ ``C:\Program Files (x86)\Ixia\IxOS\8.01-GA-Patch1\ixTclServer.exe``
4. Create a folder ``Results`` in c:\ and share the folder on the network.
``<repo>/samples/vnf_samples/nsut/vfw/tc_baremetal_http_ixload_1b_Requests-65000_Concurrency.yaml``
IxNetwork
-~~~~~~~~~
+^^^^^^^^^
IxNetwork testcases use IxNetwork API Python Bindings module, which is
installed as part of the requirements of the project.
.. code-block:: console
- cp <repo>/etc/yardstick/nodes/pod.yaml.nsb.sample.ixia etc/yardstick/nodes/pod_ixia.yaml
+ cp <repo>/etc/yardstick/nodes/pod.yaml.nsb.sample.ixia \
+ etc/yardstick/nodes/pod_ixia.yaml
- Config pod_ixia.yaml
+ Configure ``pod_ixia.yaml``
.. literalinclude:: code/pod_ixia.yaml
:language: console
- for sriov/ovs_dpdk pod files, please refer to above Standalone Virtualization for ovs-dpdk/sriov configuration
+ for sriov/ovs_dpdk pod files, please refer to above
+ `Standalone Virtualization`_ for ovs-dpdk/sriov configuration
2. Start IxNetwork TCL Server
You will also need to configure the IxNetwork machine to start the IXIA
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, 2016-2018 Intel Corporation.
+..
+ Convention for heading levels in Yardstick documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
Yardstick - NSB Testing - Operation
===================================
^^^^^^^^^^^^^^^^^
The configuration of the availability zone is requred in cases where location
-of exact compute host/group of compute hosts needs to be specified for SampleVNF
-or traffic generator in the heat test case. If this is the case, please follow
-the instructions below.
+of exact compute host/group of compute hosts needs to be specified for
+:term:`SampleVNF` or traffic generator in the heat test case. If this is the
+case, please follow the instructions below.
.. _`Create a host aggregate`:
.. code-block:: bash
# create host aggregate
- openstack aggregate create --zone <AZ_NAME> --property availability_zone=<AZ_NAME> <AGG_NAME>
+ openstack aggregate create --zone <AZ_NAME> \
+ --property availability_zone=<AZ_NAME> <AGG_NAME>
# show available hosts
openstack compute service list --service nova-compute
# add selected host into the host aggregate
networks:
...
-There are two example of SampleVNF scale out test case which use the availability zone
-feature to specify the exact location of scaled VNFs and traffic generators.
+There are two example of SampleVNF scale out test case which use the
+``availability zone`` feature to specify the exact location of scaled VNFs and
+traffic generators.
Those are:
| 5 | agg1 | AZ_NAME_1 |
+----+------+-------------------+
-2. If no host aggregates are configured, please use `steps above`__ to
- configure them.
+2. If no host aggregates are configured, please follow the instructions to
+ `Create a host aggregate`_
-__ `Create a host aggregate`_
-
-3. Run the SampleVNF PROX scale-out test case, specifying the availability
- zone of each VNF and traffic generator as a task arguments.
+3. Run the SampleVNF PROX scale-out test case, specifying the
+ ``availability zone`` of each VNF and traffic generator as task arguments.
.. note:: The ``az_0`` and ``az_1`` should be changed according to the host
- aggregates created in the OpenStack.
+ aggregates created in the OpenStack.
.. code-block:: console
- yardstick -d task start\
+ yardstick -d task start \
<repo>/samples/vnf_samples/nsut/prox/tc_prox_heat_context_l2fwd_multiflow-2-scale-out.yaml\
--task-args='{
"num_vnfs": 4, "availability_zone": {
-------------
NSB can collect KPIs from collected. We have support for various plugins
-enabled by the Barometer project.
+enabled by the :term:`Barometer` project.
The default yardstick-samplevnf has collectd installed. This allows for
collecting KPIs from the VNF.
To collectd KPIs from the NFVi compute nodes:
-
* install_collectd on the compute nodes
* create pod.yaml for the compute nodes
* enable specific plugins depending on the vswitch and DPDK
- example pod.yaml section for Compute node running collectd.
+ example ``pod.yaml`` section for Compute node running collectd.
.. code-block:: yaml
VNFs performance data with scale-out helps
- * in capacity planning to meet the given network node requirements
- * in comparison between different VNF vendor offerings
+ * capacity planning to meet the given network node requirements
+ * comparison between different VNF vendor offerings
* better the scale-out index, provides the flexibility in meeting future
capacity requirements
Sample test case file
^^^^^^^^^^^^^^^^^^^^^
- 1. Prepare SampleVNF image and copy it to ``flavor/images``.
- 2. Prepare context files for TREX and SampleVNF under ``contexts/file``.
- 3. Add bridge named ``br-int`` to the baremetal where SampleVNF image is deployed.
- 4. Modify ``networks/phy_port`` accordingly to the baremetal setup.
- 5. Run test from:
+1. Prepare SampleVNF image and copy it to ``flavor/images``.
+2. Prepare context files for TREX and SampleVNF under ``contexts/file``.
+3. Add bridge named ``br-int`` to the baremetal where SampleVNF image is deployed.
+4. Modify ``networks/phy_port`` accordingly to the baremetal setup.
+5. Run test from:
.. literalinclude:: /../samples/vnf_samples/nsut/acl/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
:language: yaml
--- /dev/null
+# Copyright (c) 2018 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{% set vports = vports or 2 %}
+---
+schema: yardstick:task:0.1
+scenarios:
+- type: NSPerf
+ traffic_profile: ../../traffic_profiles/ixia_ipv4_latency_L3.yaml
+ topology: agnostic_vnf_topology_ixia_{{ vports }}ports.yaml
+ ixia_config: IxiaL3
+ extra_args:
+ vports: {{ vports }}
+ nodes:
+ tg__0: tg_0.yardstick
+ vnf__0: vnf_0.yardstick
+ options:
+ framesize:
+ uplink: {64B: 100}
+ downlink: {64B: 100}
+ flow:
+ src_ip:
+{% for vport in range(0,vports,2|int) %}
+ - '152.{{ vport }}.0.1-152.{{ vport }}.0.50'
+{% endfor %}
+ dst_ip:
+{% for vport in range(1,vports,2|int) %}
+ - '152.{{ vport }}.1.1-152.{{ vport }}.1.150'
+{% endfor %}
+ count: 1
+ traffic_type: 4
+ rfc2544:
+ allowed_drop_rate: 0.0001 - 0.0001
+ vnf__0:
+ []
+ runner:
+ type: Iteration
+ iterations: 10
+ interval: 35
+context:
+ type: Node
+ name: yardstick
+ nfvi_type: baremetal
+ file: /etc/yardstick/nodes/pod_ixia.yaml
network_ports:
mgmt:
cidr: '1.1.1.62/24'
-{% for port_num in range(1,vports,2|int) %}
- xe{{ loop.index0 * 2 }}:
- - uplink_{{ loop.index0 }}
- xe{{ (loop.index0 * 2) + 1 }}:
- - downlink_{{ loop.index0 }}
+{% for port_num in range((vports/2)|int) %}
+ xe{{ port_num * 2 }}:
+ - uplink_{{ port_num }}
+ xe{{ (port_num * 2) + 1 }}:
+ - downlink_{{ port_num }}
{% endfor %}
networks:
-{% for port_num in range(1,vports,2|int) %}
- uplink_{{ loop.index0 }}:
- port_num: {{ networks.uplink[loop.index0].port_num }}
- phy_port: "{{ networks.uplink[loop.index0].phy_port }}"
- vpci: "{{ networks.uplink[loop.index0].vpci }}"
- cidr: "{{ networks.uplink[loop.index0].cidr }}"
- gateway_ip: "{{ networks.uplink[loop.index0].gateway_ip }}"
- downlink_{{ loop.index0 }}:
- port_num: {{ networks.downlink[loop.index0].port_num }}
- phy_port: "{{ networks.downlink[loop.index0].phy_port }}"
- vpci: "{{ networks.downlink[loop.index0].vpci }}"
- cidr: "{{ networks.downlink[loop.index0].cidr }}"
- gateway_ip: "{{ networks.downlink[loop.index0].gateway_ip }}"
+{% for port_num in range((vports/2)|int) %}
+ uplink_{{ port_num }}:
+ port_num: {{ networks.uplink[port_num].port_num }}
+ phy_port: "{{ networks.uplink[port_num].phy_port }}"
+ vpci: "{{ networks.uplink[port_num].vpci }}"
+ cidr: "{{ networks.uplink[port_num].cidr }}"
+ gateway_ip: "{{ networks.uplink[port_num].gateway_ip }}"
+ downlink_{{ port_num }}:
+ port_num: {{ networks.downlink[port_num].port_num }}
+ phy_port: "{{ networks.downlink[port_num].phy_port }}"
+ vpci: "{{ networks.downlink[port_num].vpci }}"
+ cidr: "{{ networks.downlink[port_num].cidr }}"
+ gateway_ip: "{{ networks.downlink[port_num].gateway_ip }}"
{% endfor %}
--- /dev/null
+# Copyright (c) 2018 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{% set vports = vports or 2 %}
+{% set duration = duration or 400 %}
+---
+schema: "yardstick:task:0.1"
+
+scenarios:
+-
+ type: NSPerf
+ traffic_profile: ../../traffic_profiles/prox_mpls_tag_untag.yaml
+ extra_args:
+ vports: {{ vports }}
+ topology: prox-tg-topology-scale-up.yaml
+
+ nodes:
+ tg__0: tg__0.yardstick
+ vnf__0: vnf__0.yardstick
+
+ options:
+ vnf__0:
+ prox_path: /opt/nsb_bin/prox
+ prox_config: "configs/handle_mpls_tag_untag-{{ vports }}.cfg"
+ prox_args:
+ "-t": ""
+
+ tg__0:
+ prox_path: /opt/nsb_bin/prox
+ prox_config: "configs/gen_mpls_tag_untag-{{ vports }}.cfg"
+ prox_args:
+ "-e": ""
+ "-t": ""
+
+ runner:
+ type: Duration
+ # we kill after duration, independent of test duration, so set this high
+ duration: {{ duration }}
+
+contexts:
+ - name: yardstick
+ type: Node
+ file: prox_tg_bm.yaml
+ - name: yardstick
+ type: StandaloneOvsDpdk
+ file: /etc/yardstick/nodes/standalone/host_ovs.yaml
+ vm_deploy: True
+ ovs_properties:
+ version:
+ ovs: 2.8.0
+ dpdk: 17.05.2
+ pmd_threads: 2
+ ram:
+ socket_0: 2048
+ socket_1: 2048
+ queues: 4
+ vpath: "/usr/local"
+ flavor:
+ images: "/var/lib/libvirt/images/yardstick-nsb-image.img"
+ ram: 16384
+ extra_specs:
+ hw:cpu_sockets: 1
+ hw:cpu_cores: 10
+ hw:cpu_threads: 2
+ servers:
+ vnf__0:
+ network_ports:
+ mgmt:
+ cidr: '1.1.1.62/24'
+{% for port_num in range((vports/2)|int) %}
+ xe{{ port_num * 2 }}:
+ - uplink_{{ port_num }}
+ xe{{ (port_num * 2) + 1 }}:
+ - downlink_{{ port_num }}
+{% endfor %}
+ networks:
+{% for port_num in range((vports/2)|int) %}
+ uplink_{{ port_num }}:
+ port_num: {{ networks.uplink[port_num].port_num }}
+ phy_port: "{{ networks.uplink[port_num].phy_port }}"
+ vpci: "{{ networks.uplink[port_num].vpci }}"
+ cidr: "{{ networks.uplink[port_num].cidr }}"
+ gateway_ip: "{{ networks.uplink[port_num].gateway_ip }}"
+ downlink_{{ port_num }}:
+ port_num: {{ networks.downlink[port_num].port_num }}
+ phy_port: "{{ networks.downlink[port_num].phy_port }}"
+ vpci: "{{ networks.downlink[port_num].vpci }}"
+ cidr: "{{ networks.downlink[port_num].cidr }}"
+ gateway_ip: "{{ networks.downlink[port_num].gateway_ip }}"
+{% endfor %}
--- /dev/null
+# Copyright (c) 2018 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{% set vports = vports or 2 %}
+{% set duration = duration or 400 %}
+---
+schema: "yardstick:task:0.1"
+
+scenarios:
+-
+ type: NSPerf
+ traffic_profile: ../../traffic_profiles/prox_mpls_tag_untag.yaml
+ extra_args:
+ vports: {{ vports }}
+ topology: prox-tg-topology-scale-up.yaml
+
+ nodes:
+ tg__0: tg__0.yardstick
+ vnf__0: vnf__0.yardstick
+
+ options:
+ vnf__0:
+ prox_path: /opt/nsb_bin/prox
+ prox_config: "configs/handle_mpls_tag_untag-{{ vports }}.cfg"
+ prox_args:
+ "-t": ""
+
+ tg__0:
+ prox_path: /opt/nsb_bin/prox
+ prox_config: "configs/gen_mpls_tag_untag-{{ vports }}.cfg"
+ prox_args:
+ "-e": ""
+ "-t": ""
+
+ runner:
+ type: Duration
+ # we kill after duration, independent of test duration, so set this high
+ duration: {{ duration }}
+
+contexts:
+ - name: yardstick
+ type: Node
+ file: prox_tg_bm.yaml
+ - name: yardstick
+ type: StandaloneSriov
+ file: /etc/yardstick/nodes/standalone/host_sriov.yaml
+ vm_deploy: True
+ flavor:
+ images: "/var/lib/libvirt/images/yardstick-nsb-image.img"
+ ram: 16384
+ extra_specs:
+ hw:cpu_sockets: 1
+ hw:cpu_cores: 10
+ hw:cpu_threads: 2
+ user: ""
+ password: ""
+ servers:
+ vnf__0:
+ network_ports:
+ mgmt:
+ cidr: '1.1.1.62/24'
+{% for port_num in range((vports/2)|int) %}
+ xe{{ port_num * 2 }}:
+ - uplink_{{ port_num }}
+ xe{{ (port_num * 2) + 1 }}:
+ - downlink_{{ port_num }}
+{% endfor %}
+ networks:
+{% for port_num in range((vports/2)|int) %}
+ uplink_{{ port_num }}:
+ phy_port: "{{ networks.uplink[port_num].phy_port }}"
+ vpci: "{{ networks.uplink[port_num].vpci }}"
+ cidr: "{{ networks.uplink[port_num].cidr }}"
+ gateway_ip: "{{ networks.uplink[port_num].gateway_ip }}"
+ downlink_{{ port_num }}:
+ phy_port: "{{ networks.downlink[port_num].phy_port }}"
+ vpci: "{{ networks.downlink[port_num].vpci }}"
+ cidr: "{{ networks.downlink[port_num].cidr }}"
+ gateway_ip: "{{ networks.downlink[port_num].gateway_ip }}"
+{% endfor %}
--- /dev/null
+# Copyright (c) 2018 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+{% set vports = get(extra_args, 'vports', '2') %}
+---
+schema: "nsb:traffic_profile:0.1"
+
+# This file is a template, it will be filled with values from tc.yaml before passing to the traffic generator
+
+name: rfc2544
+description: Traffic profile to run RFC2544 latency with L3 support
+traffic_profile:
+ traffic_type : IXIARFC2544Profile # defines traffic behavior - constant or look for highest possible throughput
+ frame_rate : 100% # pc of linerate
+ duration: {{ duration }}
+ enable_latency: True
+
+{% for vport in range(vports|int) %}
+uplink_{{ vport }}:
+ ipv4:
+ id: {{ (vport * 2) + 1 }}
+ outer_l2:
+ framesize:
+ 64B: "{{get(imix, 'imix.uplink.64B', '0') }}"
+ 128B: "{{get(imix, 'imix.uplink.128B', '0') }}"
+ 256B: "{{get(imix, 'imix.uplink.256B', '0') }}"
+ 373b: "{{get(imix, 'imix.uplink.373B', '0') }}"
+ 512B: "{{get(imix, 'imix.uplink.512B', '0') }}"
+ 570B: "{{get(imix, 'imix.uplink.570B', '0') }}"
+ 1024B: "{{get(imix, 'imix.uplink.1024B', '0') }}"
+ 1280B: "{{get(imix, 'imix.uplink.1280B', '0') }}"
+ 1400B: "{{get(imix, 'imix.uplink.1400B', '0') }}"
+ 1500B: "{{get(imix, 'imix.uplink.1500B', '0') }}"
+ 1518B: "{{get(imix, 'imix.uplink.1518B', '0') }}"
+
+downlink_{{vport}}:
+ ipv4:
+ id: {{ (vport * 2) + 2 }}
+ outer_l2:
+ framesize:
+ 64B: "{{get(imix, 'imix.downlink.64B', '0') }}"
+ 128B: "{{get(imix, 'imix.downlink.128B', '0') }}"
+ 256B: "{{get(imix, 'imix.downlink.256B', '0') }}"
+ 373b: "{{get(imix, 'imix.downlink.373B', '0') }}"
+ 512B: "{{get(imix, 'imix.downlink.512B', '0') }}"
+ 570B: "{{get(imix, 'imix.downlink.570B', '0') }}"
+ 1024B: "{{get(imix, 'imix.downlink.1024B', '0') }}"
+ 1280B: "{{get(imix, 'imix.downlink.1280B', '0') }}"
+ 1400B: "{{get(imix, 'imix.downlink.1400B', '0') }}"
+ 1500B: "{{get(imix, 'imix.downlink.1500B', '0') }}"
+ 1518B: "{{get(imix, 'imix.downlink.1518B', '0') }}"
+{% endfor %}
pkey = pkg_resources.resource_string(
'yardstick.resources',
h_join('files/yardstick_key', self.name)).decode('utf-8')
-
+ key_filename = pkg_resources.resource_filename('yardstick.resources',
+ h_join('files/yardstick_key', self.name))
result = {
"user": server.context.user,
"pkey": pkey,
+ "key_filename": key_filename,
"private_ip": server.private_ip,
"interfaces": server.interfaces,
"routing_table": self.generate_routing_table(server),
message = 'Field "%(field_name)s" not present in stack item %(stack_item)s'
+class IncorrectFlowOption(YardstickException):
+ message = 'Flow option {option} for {link} is incorrect'
+
+
class SLAValidationError(YardstickException):
message = '%(case_name)s SLA validation failed. Error: %(error_msg)s'
return ipaddress.IPv4Address(six.text_type(ip_addr))
+def get_ip_range_count(iprange):
+ start_range, end_range = iprange.split("-")
+ start = int(make_ipv4_address(start_range))
+ end = int(make_ipv4_address(end_range))
+ return end - start
+
+
+def get_ip_range_start(iprange):
+ return str(make_ipv4_address(iprange.split("-")[0]))
+
+
def safe_ip_address(ip_addr):
""" get ip address version v6 or v4 """
try:
vports = self.ixnet.getList(self.ixnet.getRoot(), 'vport')
return vports
+ def get_static_interface(self, vport):
+ return self.ixnet.getList(vport, 'interface')
+
def _get_config_element_by_flow_group_name(self, flow_group_name):
"""Get a config element using the flow group name
-# Copyright (c) 2016-2017 Intel Corporation
+# Copyright (c) 2016-2018 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
import six
from yardstick.common import utils
+from yardstick.common import exceptions
from yardstick.network_services.libs.ixia_libs.ixnet import ixnet_api
from yardstick.network_services.vnf_generic.vnf.sample_vnf import SampleVNFTrafficGen
from yardstick.network_services.vnf_generic.vnf.sample_vnf import ClientResourceHelper
class IxiaBasicScenario(object):
+ """Ixia Basic scenario for flow from port to port"""
+
def __init__(self, client, context_cfg, ixia_cfg):
self.client = client
def apply_config(self):
pass
+ def run_protocols(self):
+ pass
+
+ def stop_protocols(self):
+ pass
+
def create_traffic_model(self, traffic_profile=None):
# pylint: disable=unused-argument
vports = self.client.get_vports()
self.client.create_traffic_model(self._uplink_vports,
self._downlink_vports)
- def run_protocols(self):
- pass
- def stop_protocols(self):
- pass
+class IxiaL3Scenario(IxiaBasicScenario):
+ """Ixia scenario for L3 flow between static ip's"""
+
+ def _add_static_ips(self):
+ vports = self.client.get_vports()
+ uplink_intf_vport = [(self.client.get_static_interface(vport), vport)
+ for vport in vports[::2]]
+ downlink_intf_vport = [(self.client.get_static_interface(vport), vport)
+ for vport in vports[1::2]]
+
+ for index in range(len(uplink_intf_vport)):
+ intf, vport = uplink_intf_vport[index]
+ try:
+ iprange = self.ixia_cfg['flow'].get('src_ip')[index]
+ start_ip = utils.get_ip_range_start(iprange)
+ count = utils.get_ip_range_count(iprange)
+ self.client.add_static_ipv4(intf, vport, start_ip, count, '32')
+ except IndexError:
+ raise exceptions.IncorrectFlowOption(
+ option="src_ip", link="uplink_{}".format(index))
+
+ intf, vport = downlink_intf_vport[index]
+ try:
+ iprange = self.ixia_cfg['flow'].get('dst_ip')[index]
+ start_ip = utils.get_ip_range_start(iprange)
+ count = utils.get_ip_range_count(iprange)
+ self.client.add_static_ipv4(intf, vport, start_ip, count, '32')
+ except IndexError:
+ raise exceptions.IncorrectFlowOption(
+ option="dst_ip", link="downlink_{}".format(index))
+
+ def _add_interfaces(self):
+ vports = self.client.get_vports()
+ uplink_vports = (vport for vport in vports[::2])
+ downlink_vports = (vport for vport in vports[1::2])
+
+ ix_node = next(node for _, node in self.context_cfg['nodes'].items()
+ if node['role'] == 'IxNet')
+
+ for intf in ix_node['interfaces'].values():
+ ip = intf.get('local_ip')
+ mac = intf.get('local_mac')
+ gateway = None
+ try:
+ gateway = next(route.get('gateway')
+ for route in ix_node.get('routing_table')
+ if route.get('if') == intf.get('ifname'))
+ except StopIteration:
+ LOG.debug("Gateway not provided")
+
+ if 'uplink' in intf.get('vld_id'):
+ self.client.add_interface(next(uplink_vports),
+ ip, mac, gateway)
+ else:
+ self.client.add_interface(next(downlink_vports),
+ ip, mac, gateway)
+
+ def apply_config(self):
+ self._add_interfaces()
+ self._add_static_ips()
+
+ def create_traffic_model(self, traffic_profile=None):
+ # pylint: disable=unused-argument
+ vports = self.client.get_vports()
+ self._uplink_vports = vports[::2]
+ self._downlink_vports = vports[1::2]
+
+ uplink_endpoints = [port + '/protocols/static'
+ for port in self._uplink_vports]
+ downlink_endpoints = [port + '/protocols/static'
+ for port in self._downlink_vports]
+
+ self.client.create_ipv4_traffic_model(uplink_endpoints,
+ downlink_endpoints)
class IxiaPppoeClientScenario(object):
self._ixia_scenarios = {
"IxiaBasic": IxiaBasicScenario,
+ "IxiaL3": IxiaL3Scenario,
"IxiaPppoeClient": IxiaPppoeClientScenario,
}
for addr in addr_list:
self.assertRaises(Exception, utils.make_ipv4_address, addr)
+ def test_get_ip_range_count(self):
+ iprange = "192.168.0.1-192.168.0.25"
+ count = utils.get_ip_range_count(iprange)
+ self.assertEqual(count, 24)
+
+ def test_get_ip_range_start(self):
+ iprange = "192.168.0.1-192.168.0.25"
+ start = utils.get_ip_range_start(iprange)
+ self.assertEqual(start, "192.168.0.1")
def test_safe_ip_address(self):
addr_list = self.GOOD_IP_V4_ADDRESS_STR_LIST
self.ixnet.getRoot.return_value = 'my_root'
self.ixnet_gen = ixnet_api.IxNextgen()
self.ixnet_gen._ixnet = self.ixnet
+ self._mock_log = mock.patch.object(ixnet_api.log, 'info')
+ self.mock_log = self._mock_log.start()
+ self.addCleanup(self._stop_mocks)
+
+ def _stop_mocks(self):
+ self.mock_log.stop()
def test_get_config(self):
tg_cfg = {
import unittest
import mock
+import paramiko
+
from yardstick.common import exceptions as y_exceptions
from yardstick.common import utils
from yardstick.network_services.nfvi.resource import ResourceProfile
from yardstick.network_services.vnf_generic.vnf.base import VnfdHelper
from yardstick.network_services.vnf_generic.vnf import sample_vnf
-from yardstick.network_services.vnf_generic.vnf.vnf_ssh_helper import VnfSshHelper
+from yardstick.network_services.vnf_generic.vnf import vnf_ssh_helper
from yardstick.network_services.vnf_generic.vnf.sample_vnf import SampleVNFDeployHelper
from yardstick.network_services.vnf_generic.vnf.sample_vnf import ScenarioHelper
from yardstick.network_services.vnf_generic.vnf.sample_vnf import ResourceHelper
from yardstick.network_services.vnf_generic.vnf.sample_vnf import DpdkVnfSetupEnvHelper
from yardstick.tests.unit.network_services.vnf_generic.vnf import test_base
from yardstick.benchmark.contexts import base as ctx_base
+from yardstick import ssh
class MockError(Exception):
'virtual-interface': {
'dst_mac': '00:00:00:00:00:03',
'vpci': '0000:05:00.0',
+ 'dpdk_port_num': 0,
+ 'driver': 'i40e',
'local_ip': '152.16.100.19',
'type': 'PCI-PASSTHROUGH',
'netmask': '255.255.255.0',
- 'dpdk_port_num': 0,
'bandwidth': '10 Gbps',
'dst_ip': '152.16.100.20',
'local_mac': '00:00:00:00:00:01',
'virtual-interface': {
'dst_mac': '00:00:00:00:00:04',
'vpci': '0000:05:00.1',
+ 'dpdk_port_num': 1,
+ 'driver': 'ixgbe',
'local_ip': '152.16.40.19',
'type': 'PCI-PASSTHROUGH',
'netmask': '255.255.255.0',
- 'dpdk_port_num': 1,
'bandwidth': '10 Gbps',
'dst_ip': '152.16.40.20',
'local_mac': '00:00:00:00:00:02',
}
}
+ def setUp(self):
+ self.ssh_helper = vnf_ssh_helper.VnfSshHelper(
+ self.VNFD_0['mgmt-interface'], 'my/bin/path')
+ self.ssh_helper._run = mock.Mock()
+
def assertAll(self, iterable, message=None):
self.assertTrue(all(iterable), message)
def test_get_class(self):
- self.assertIs(VnfSshHelper.get_class(), VnfSshHelper)
+ self.assertIs(vnf_ssh_helper.VnfSshHelper.get_class(),
+ vnf_ssh_helper.VnfSshHelper)
- @mock.patch('yardstick.ssh.paramiko')
+ @mock.patch.object(ssh, 'paramiko')
def test_copy(self, _):
- ssh_helper = VnfSshHelper(self.VNFD_0['mgmt-interface'], 'my/bin/path')
- ssh_helper._run = mock.Mock()
-
- ssh_helper.execute('ls')
- self.assertTrue(ssh_helper.is_connected)
- result = ssh_helper.copy()
- self.assertIsInstance(result, VnfSshHelper)
+ self.ssh_helper.execute('ls')
+ self.assertTrue(self.ssh_helper.is_connected)
+ result = self.ssh_helper.copy()
+ self.assertIsInstance(result, vnf_ssh_helper.VnfSshHelper)
self.assertFalse(result.is_connected)
- self.assertEqual(result.bin_path, ssh_helper.bin_path)
- self.assertEqual(result.host, ssh_helper.host)
- self.assertEqual(result.port, ssh_helper.port)
- self.assertEqual(result.user, ssh_helper.user)
- self.assertEqual(result.password, ssh_helper.password)
- self.assertEqual(result.key_filename, ssh_helper.key_filename)
-
- @mock.patch('yardstick.ssh.paramiko')
+ self.assertEqual(result.bin_path, self.ssh_helper.bin_path)
+ self.assertEqual(result.host, self.ssh_helper.host)
+ self.assertEqual(result.port, self.ssh_helper.port)
+ self.assertEqual(result.user, self.ssh_helper.user)
+ self.assertEqual(result.password, self.ssh_helper.password)
+ self.assertEqual(result.key_filename, self.ssh_helper.key_filename)
+
+ @mock.patch.object(paramiko, 'SSHClient')
def test_upload_config_file(self, mock_paramiko):
- ssh_helper = VnfSshHelper(self.VNFD_0['mgmt-interface'], 'my/bin/path')
- ssh_helper._run = mock.MagicMock()
+ self.assertFalse(self.ssh_helper.is_connected)
+ cfg_file = self.ssh_helper.upload_config_file('/my/prefix', 'my content')
+ self.assertTrue(self.ssh_helper.is_connected)
+ mock_paramiko.assert_called_once()
+ self.assertEqual(cfg_file, '/my/prefix')
- self.assertFalse(ssh_helper.is_connected)
- cfg_file = ssh_helper.upload_config_file('my/prefix', 'my content')
- self.assertTrue(ssh_helper.is_connected)
- mock_paramiko.SSHClient.assert_called_once()
+ @mock.patch.object(paramiko, 'SSHClient')
+ def test_upload_config_file_path_does_not_exist(self, mock_paramiko):
+ self.assertFalse(self.ssh_helper.is_connected)
+ cfg_file = self.ssh_helper.upload_config_file('my/prefix', 'my content')
+ self.assertTrue(self.ssh_helper.is_connected)
+ mock_paramiko.assert_called_once()
self.assertTrue(cfg_file.startswith('/tmp'))
- cfg_file = ssh_helper.upload_config_file('/my/prefix', 'my content')
- self.assertTrue(ssh_helper.is_connected)
- mock_paramiko.SSHClient.assert_called_once()
- self.assertEqual(cfg_file, '/my/prefix')
-
def test_join_bin_path(self):
- ssh_helper = VnfSshHelper(self.VNFD_0['mgmt-interface'], 'my/bin/path')
-
expected_start = 'my'
expected_middle_list = ['bin']
expected_end = 'path'
- result = ssh_helper.join_bin_path()
+ result = self.ssh_helper.join_bin_path()
self.assertTrue(result.startswith(expected_start))
self.assertAll(middle in result for middle in expected_middle_list)
self.assertTrue(result.endswith(expected_end))
expected_middle_list.append(expected_end)
expected_end = 'some_file.sh'
- result = ssh_helper.join_bin_path('some_file.sh')
+ result = self.ssh_helper.join_bin_path('some_file.sh')
self.assertTrue(result.startswith(expected_start))
self.assertAll(middle in result for middle in expected_middle_list)
self.assertTrue(result.endswith(expected_end))
expected_middle_list.append('some_dir')
expected_end = 'some_file.sh'
- result = ssh_helper.join_bin_path('some_dir', 'some_file.sh')
+ result = self.ssh_helper.join_bin_path('some_dir', 'some_file.sh')
self.assertTrue(result.startswith(expected_start))
self.assertAll(middle in result for middle in expected_middle_list)
self.assertTrue(result.endswith(expected_end))
- @mock.patch('yardstick.ssh.paramiko')
- @mock.patch('yardstick.ssh.provision_tool')
+ @mock.patch.object(paramiko, 'SSHClient')
+ @mock.patch.object(ssh, 'provision_tool')
def test_provision_tool(self, mock_provision_tool, mock_paramiko):
- ssh_helper = VnfSshHelper(self.VNFD_0['mgmt-interface'], 'my/bin/path')
- ssh_helper._run = mock.MagicMock()
-
- self.assertFalse(ssh_helper.is_connected)
- ssh_helper.provision_tool()
- self.assertTrue(ssh_helper.is_connected)
- mock_paramiko.SSHClient.assert_called_once()
+ self.assertFalse(self.ssh_helper.is_connected)
+ self.ssh_helper.provision_tool()
+ self.assertTrue(self.ssh_helper.is_connected)
+ mock_paramiko.assert_called_once()
mock_provision_tool.assert_called_once()
- ssh_helper.provision_tool(tool_file='my_tool.sh')
- self.assertTrue(ssh_helper.is_connected)
- mock_paramiko.SSHClient.assert_called_once()
+ self.ssh_helper.provision_tool(tool_file='my_tool.sh')
+ self.assertTrue(self.ssh_helper.is_connected)
+ mock_paramiko.assert_called_once()
self.assertEqual(mock_provision_tool.call_count, 2)
- ssh_helper.provision_tool('tool_path', 'my_tool.sh')
- self.assertTrue(ssh_helper.is_connected)
- mock_paramiko.SSHClient.assert_called_once()
+ self.ssh_helper.provision_tool('tool_path', 'my_tool.sh')
+ self.assertTrue(self.ssh_helper.is_connected)
+ mock_paramiko.assert_called_once()
self.assertEqual(mock_provision_tool.call_count, 3)
class TestIxiaBasicScenario(unittest.TestCase):
-
def setUp(self):
self._mock_IxNextgen = mock.patch.object(ixnet_api, 'IxNextgen')
self.mock_IxNextgen = self._mock_IxNextgen.start()
self.assertIsInstance(self.scenario, tg_rfc2544_ixia.IxiaBasicScenario)
self.assertEqual(self.scenario.client, self.mock_IxNextgen)
- def test_apply_config(self):
- self.assertIsNone(self.scenario.apply_config())
-
def test_create_traffic_model(self):
self.mock_IxNextgen.get_vports.return_value = [1, 2, 3, 4]
self.scenario.create_traffic_model()
self.scenario.client.get_vports.assert_called_once()
self.scenario.client.create_traffic_model.assert_called_once_with([1, 3], [2, 4])
+ def test_apply_config(self):
+ self.assertIsNone(self.scenario.apply_config())
+
def test_run_protocols(self):
self.assertIsNone(self.scenario.run_protocols())
self.assertIsNone(self.scenario.stop_protocols())
+class TestIxiaL3Scenario(TestIxiaBasicScenario):
+ IXIA_CFG = {
+ 'flow': {
+ 'src_ip': ['192.168.0.1-192.168.0.50'],
+ 'dst_ip': ['192.168.1.1-192.168.1.150']
+ }
+ }
+
+ CONTEXT_CFG = {
+ 'nodes': {
+ 'tg__0': {
+ 'role': 'IxNet',
+ 'interfaces': {
+ 'xe0': {
+ 'vld_id': 'uplink_0',
+ 'local_ip': '10.1.1.1',
+ 'local_mac': 'aa:bb:cc:dd:ee:ff',
+ 'ifname': 'xe0'
+ },
+ 'xe1': {
+ 'vld_id': 'downlink_0',
+ 'local_ip': '20.2.2.2',
+ 'local_mac': 'bb:bb:cc:dd:ee:ee',
+ 'ifname': 'xe1'
+ }
+ },
+ 'routing_table': [{
+ 'network': "152.16.100.20",
+ 'netmask': '255.255.0.0',
+ 'gateway': '152.16.100.21',
+ 'if': 'xe0'
+ }]
+ }
+ }
+ }
+
+ def setUp(self):
+ super(TestIxiaL3Scenario, self).setUp()
+ self.ixia_cfg = self.IXIA_CFG
+ self.context_cfg = self.CONTEXT_CFG
+ self.scenario = tg_rfc2544_ixia.IxiaL3Scenario(self.mock_IxNextgen,
+ self.context_cfg,
+ self.ixia_cfg)
+
+ def test___init___(self):
+ self.assertIsInstance(self.scenario, tg_rfc2544_ixia.IxiaL3Scenario)
+ self.assertEqual(self.scenario.client, self.mock_IxNextgen)
+
+ def test_create_traffic_model(self):
+ self.mock_IxNextgen.get_vports.return_value = ['1', '2']
+ self.scenario.create_traffic_model()
+ self.scenario.client.get_vports.assert_called_once()
+ self.scenario.client.create_ipv4_traffic_model.\
+ assert_called_once_with(['1/protocols/static'],
+ ['2/protocols/static'])
+
+ def test_apply_config(self):
+ self.scenario._add_interfaces = mock.Mock()
+ self.scenario._add_static_ips = mock.Mock()
+ self.assertIsNone(self.scenario.apply_config())
+
+ def test__add_static(self):
+ self.mock_IxNextgen.get_vports.return_value = ['1', '2']
+ self.mock_IxNextgen.get_static_interface.side_effect = ['intf1',
+ 'intf2']
+
+ self.scenario._add_static_ips()
+
+ self.mock_IxNextgen.get_static_interface.assert_any_call('1')
+ self.mock_IxNextgen.get_static_interface.assert_any_call('2')
+
+ self.scenario.client.add_static_ipv4.assert_any_call(
+ 'intf1', '1', '192.168.0.1', 49, '32')
+ self.scenario.client.add_static_ipv4.assert_any_call(
+ 'intf2', '2', '192.168.1.1', 149, '32')
+
+ def test__add_interfaces(self):
+ self.mock_IxNextgen.get_vports.return_value = ['1', '2']
+
+ self.scenario._add_interfaces()
+
+ self.mock_IxNextgen.add_interface.assert_any_call('1',
+ '10.1.1.1',
+ 'aa:bb:cc:dd:ee:ff',
+ '152.16.100.21')
+ self.mock_IxNextgen.add_interface.assert_any_call('2',
+ '20.2.2.2',
+ 'bb:bb:cc:dd:ee:ee',
+ None)
+
+
class TestIxiaPppoeClientScenario(unittest.TestCase):
IXIA_CFG = {
Code Review / yardstick.git / commitdiff