.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, 2016-2017 Intel Corporation.
+.. (c) OPNFV, 2016-2019 Intel Corporation.
.. Convention for heading levels in Yardstick documentation:
NSB extension includes:
* Generic data models of Network Services, based on ETSI spec
- `ETSI GS NFV-TST 001`_
-* Standalone :term:`context` for VNF testing with SRIOV, OVS, OVS-DPDK, etc
+ `ETSI GS NFV-TST001`_
+* Standalone :term:`context` for VNF testing SRIOV, OVS-DPDK, etc
* Generic VNF configuration models and metrics implemented with Python
classes
* Traffic generator features and traffic profiles
* L4-L7 state-full traffic profiles
* Tunneling protocol/network overlay support
+* Scenarios that handle NSB test cases execution
+
+ * NSPerf - scenario that handles generic NSB test case execution
+ (setup and init tg/vnf, trigger traffic on tg, collect kpi)
+ * NSPerf-RFC2544 - scenario that allows repeatable triggering of traffic on
+ traffic generators until test case acceptance criteria is met
+ (for example RFC2544 binary search)
+
* Test case samples
* Ping
Components of Network Service
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. TODO: provide a list of components in this section and describe them in
+ later sub-sections
+
+.. Components are the methodology, TGs, framework extensions, KPI collection,
+ Testcases, SampleVNFs
+.. Framework extentions include: VNF models, NSPerf Scenario, contexts
+
* *Models for Network Service benchmarking*: The Network Service benchmarking
requires the proper modelling approach. The NSB provides models using Python
files and defining of NSDs and VNFDs.
* Correlated UDP traffic using TREX traffic generator and replay VNF.
- * using different IMIX configuration like pure voice, pure video traffic etc
- * using different number IP flows e.g. 1, 1K, 16K, 64K, 256K, 1M flows
+ * Using different IMIX configuration like pure voice, pure video traffic etc
+ * Using different number IP flows e.g. 1, 1K, 16K, 64K, 256K, 1M flows
* Using different number of rules configured e.g. 1, 1K, 10K rules
For UDP correlated traffic following Key Performance Indicators are collected
* RFC2544 throughput for various loss rate defined (1% is a default)
+KPI Collection
+^^^^^^^^^^^^^^
+
+KPI collection is the process of sampling KPIs at multiple intervals to allow
+for investigation into anomalies during runtime. Some KPI intervals are
+adjustable. KPIs are collected from traffic generators and NFVI for the SUT.
+There is already some reporting in NSB available, but NSB collects all KPIs for
+analytics to process.
+
+Below is an example list of basic KPIs:
+
+* Throughput
+* Latency
+* Packet delay variation
+* Maximum establishment rate
+* Maximum tear-down rate
+* Maximum simultaneous number of sessions
+
+Of course, there can be many other KPIs that will be relevant for a specific
+NFVI, but in most cases these KPIs are enough to give you a basic picture of
+the SUT. NSB also uses :term:`collectd` in order to collect the KPIs. Currently
+the following collectd plug-ins are enabled for NSB testcases:
+
+* Libvirt
+* Interface stats
+* OvS events
+* vSwitch stats
+* Huge Pages
+* RAM
+* CPU usage
+* Intel® PMU
+* Intel® RDT
+
Graphical Overview
------------------
Code Review / yardstick.git / blobdiff