-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Ericsson AB and others.
-
-====================
-Yardstick Test Cases
-====================
-
-Abstract
-========
-
-This chapter lists available Yardstick test cases.
-Yardstick test cases are divided in two main categories:
-
-* *Generic NFVI Test Cases* - Test Cases developed to realize the methodology
- described in :doc:`02-methodology`
-
-* *OPNFV Feature Test Cases* - Test Cases developed to verify one or more
- aspect of a feature delivered by an OPNFV Project.
-
-Generic NFVI Test Case Descriptions
-===================================
-
-.. toctree::
- :maxdepth: 1
-
- opnfv_yardstick_tc001.rst
- opnfv_yardstick_tc002.rst
- opnfv_yardstick_tc004.rst
- opnfv_yardstick_tc005.rst
- opnfv_yardstick_tc006.rst
- opnfv_yardstick_tc008.rst
- opnfv_yardstick_tc009.rst
- opnfv_yardstick_tc010.rst
- opnfv_yardstick_tc011.rst
- opnfv_yardstick_tc012.rst
- opnfv_yardstick_tc014.rst
- opnfv_yardstick_tc024.rst
- opnfv_yardstick_tc037.rst
- opnfv_yardstick_tc038.rst
- opnfv_yardstick_tc042.rst
- opnfv_yardstick_tc043.rst
- opnfv_yardstick_tc044.rst
- opnfv_yardstick_tc055.rst
- opnfv_yardstick_tc061.rst
- opnfv_yardstick_tc063.rst
- opnfv_yardstick_tc069.rst
- opnfv_yardstick_tc070.rst
- opnfv_yardstick_tc071.rst
- opnfv_yardstick_tc072.rst
- opnfv_yardstick_tc073.rst
- opnfv_yardstick_tc074.rst
- opnfv_yardstick_tc075.rst
- opnfv_yardstick_tc076.rst
- opnfv_yardstick_tc078.rst
- opnfv_yardstick_tc079.rst
- opnfv_yardstick_tc080.rst
- opnfv_yardstick_tc081.rst
- opnfv_yardstick_tc083.rst
- opnfv_yardstick_tc084.rst
-
-OPNFV Feature Test Cases
-========================
-
-H A
----
-
-.. toctree::
- :maxdepth: 1
-
- opnfv_yardstick_tc019.rst
- opnfv_yardstick_tc025.rst
- opnfv_yardstick_tc045.rst
- opnfv_yardstick_tc046.rst
- opnfv_yardstick_tc047.rst
- opnfv_yardstick_tc048.rst
- opnfv_yardstick_tc049.rst
- opnfv_yardstick_tc050.rst
- opnfv_yardstick_tc051.rst
- opnfv_yardstick_tc052.rst
- opnfv_yardstick_tc053.rst
- opnfv_yardstick_tc054.rst
- opnfv_yardstick_tc056.rst
- opnfv_yardstick_tc057.rst
- opnfv_yardstick_tc058.rst
- opnfv_yardstick_tc087.rst
- opnfv_yardstick_tc088.rst
- opnfv_yardstick_tc089.rst
- opnfv_yardstick_tc090.rst
- opnfv_yardstick_tc091.rst
- opnfv_yardstick_tc092.rst
- opnfv_yardstick_tc093.rst
-
-IPv6
-----
-
-.. toctree::
- :maxdepth: 1
-
- opnfv_yardstick_tc027.rst
-
-KVM
----
-
-.. toctree::
- :maxdepth: 1
-
- opnfv_yardstick_tc028.rst
-
-Parser
-------
-
-.. toctree::
- :maxdepth: 1
-
- opnfv_yardstick_tc040.rst
-
-StorPerf
---------
-
-.. toctree::
- :maxdepth: 1
-
- opnfv_yardstick_tc074.rst
-
-Templates
-=========
-
-.. toctree::
- :maxdepth: 1
-
- testcase_description_v2_template
- Yardstick_task_templates
-
+.. This work is licensed under a Creative Commons Attribution 4.0 International\r
+.. License.\r
+.. http://creativecommons.org/licenses/by/4.0\r
+.. (c) OPNFV, Ericsson AB and others.\r
+\r
+====================\r
+Yardstick Test Cases\r
+====================\r
+\r
+Abstract\r
+========\r
+\r
+This chapter lists available Yardstick test cases.\r
+Yardstick test cases are divided in two main categories:\r
+\r
+* *Generic NFVI Test Cases* - Test Cases developed to realize the methodology\r
+ described in :doc:`02-methodology`\r
+\r
+* *OPNFV Feature Test Cases* - Test Cases developed to verify one or more\r
+ aspect of a feature delivered by an OPNFV Project.\r
+\r
+Generic NFVI Test Case Descriptions\r
+===================================\r
+\r
+.. toctree::\r
+ :maxdepth: 1\r
+\r
+ opnfv_yardstick_tc001.rst\r
+ opnfv_yardstick_tc002.rst\r
+ opnfv_yardstick_tc004.rst\r
+ opnfv_yardstick_tc005.rst\r
+ opnfv_yardstick_tc006.rst\r
+ opnfv_yardstick_tc008.rst\r
+ opnfv_yardstick_tc009.rst\r
+ opnfv_yardstick_tc010.rst\r
+ opnfv_yardstick_tc011.rst\r
+ opnfv_yardstick_tc012.rst\r
+ opnfv_yardstick_tc014.rst\r
+ opnfv_yardstick_tc015.rst\r
+ opnfv_yardstick_tc024.rst\r
+ opnfv_yardstick_tc037.rst\r
+ opnfv_yardstick_tc038.rst\r
+ opnfv_yardstick_tc042.rst\r
+ opnfv_yardstick_tc043.rst\r
+ opnfv_yardstick_tc044.rst\r
+ opnfv_yardstick_tc055.rst\r
+ opnfv_yardstick_tc061.rst\r
+ opnfv_yardstick_tc063.rst\r
+ opnfv_yardstick_tc069.rst\r
+ opnfv_yardstick_tc070.rst\r
+ opnfv_yardstick_tc071.rst\r
+ opnfv_yardstick_tc072.rst\r
+ opnfv_yardstick_tc073.rst\r
+ opnfv_yardstick_tc074.rst\r
+ opnfv_yardstick_tc075.rst\r
+ opnfv_yardstick_tc076.rst\r
+ opnfv_yardstick_tc078.rst\r
+ opnfv_yardstick_tc079.rst\r
+ opnfv_yardstick_tc080.rst\r
+ opnfv_yardstick_tc081.rst\r
+ opnfv_yardstick_tc083.rst\r
+ opnfv_yardstick_tc084.rst\r
+\r
+OPNFV Feature Test Cases\r
+========================\r
+\r
+H A\r
+---\r
+\r
+.. toctree::\r
+ :maxdepth: 1\r
+\r
+ opnfv_yardstick_tc019.rst\r
+ opnfv_yardstick_tc025.rst\r
+ opnfv_yardstick_tc045.rst\r
+ opnfv_yardstick_tc046.rst\r
+ opnfv_yardstick_tc047.rst\r
+ opnfv_yardstick_tc048.rst\r
+ opnfv_yardstick_tc049.rst\r
+ opnfv_yardstick_tc050.rst\r
+ opnfv_yardstick_tc051.rst\r
+ opnfv_yardstick_tc052.rst\r
+ opnfv_yardstick_tc053.rst\r
+ opnfv_yardstick_tc054.rst\r
+ opnfv_yardstick_tc056.rst\r
+ opnfv_yardstick_tc057.rst\r
+ opnfv_yardstick_tc058.rst\r
+ opnfv_yardstick_tc087.rst\r
+ opnfv_yardstick_tc088.rst\r
+ opnfv_yardstick_tc089.rst\r
+ opnfv_yardstick_tc090.rst\r
+ opnfv_yardstick_tc091.rst\r
+ opnfv_yardstick_tc092.rst\r
+ opnfv_yardstick_tc093.rst\r
+\r
+IPv6\r
+----\r
+\r
+.. toctree::\r
+ :maxdepth: 1\r
+\r
+ opnfv_yardstick_tc027.rst\r
+\r
+KVM\r
+---\r
+\r
+.. toctree::\r
+ :maxdepth: 1\r
+\r
+ opnfv_yardstick_tc028.rst\r
+\r
+Parser\r
+------\r
+\r
+.. toctree::\r
+ :maxdepth: 1\r
+\r
+ opnfv_yardstick_tc040.rst\r
+\r
+StorPerf\r
+--------\r
+\r
+.. toctree::\r
+ :maxdepth: 1\r
+\r
+ opnfv_yardstick_tc074.rst\r
+\r
+Templates\r
+=========\r
+\r
+.. toctree::\r
+ :maxdepth: 1\r
+\r
+ testcase_description_v2_template\r
+ Yardstick_task_templates\r
+\r
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International\r
+.. License.\r
+.. http://creativecommons.org/licenses/by/4.0\r
+.. (c) OPNFV, Orange and others.\r
+\r
+*************************************\r
+Yardstick Test Case Description TC015\r
+*************************************\r
+\r
+.. _unixbench: https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench\r
+\r
++-----------------------------------------------------------------------------+\r
+| Processing speed with impact on energy consumption and CPU load |\r
+| |\r
++--------------+--------------------------------------------------------------+\r
+|test case id | OPNFV_YARDSTICK_TC015_PROCESSING SPEED |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|metric | score of single cpu running, |\r
+| | score of parallel running, |\r
+| | energy consumption |\r
+| | cpu load |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|test purpose | The purpose of TC015 is to evaluate the IaaS compute |\r
+| | performance with regards to CPU processing speed with |\r
+| | its impact on the energy consumption |\r
+| | It measures score of single cpu running and parallel |\r
+| | running. Energy consumption and cpu load are monitored while |\r
+| | the cpu test is running. |\r
+| | |\r
+| | The purpose is also to be able to spot the trends. |\r
+| | Test results, graphs and similar shall be stored for |\r
+| | comparison reasons and product evolution understanding |\r
+| | between different OPNFV versions and/or configurations, |\r
+| | different server types. |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|test tool | UnixBench |\r
+| | |\r
+| | Unixbench is the most used CPU benchmarking software tool. |\r
+| | It can measure the performance of bash scripts, CPUs in |\r
+| | multithreading and single threading. It can also measure the |\r
+| | performance for parallel tasks. Also, specific disk IO for |\r
+| | small and large files are performed. You can use it to |\r
+| | measure either linux dedicated servers and linux vps |\r
+| | servers, running CentOS, Debian, Ubuntu, Fedora and other |\r
+| | distros. |\r
+| | |\r
+| | (UnixBench is not always part of a Linux distribution, hence |\r
+| | it needs to be installed. As an example see the |\r
+| | /yardstick/tools/ directory for how to generate a Linux |\r
+| | image with UnixBench included.) |\r
+| | |\r
+| | Redfish API |\r
+| | This HTTPS interface is provided by BMC of every telco grade |\r
+| | server. Is is a standard interface. |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|test | The UnixBench runs system benchmarks on a compute, getting |\r
+|description | information on the CPUs in the system. If the system has |\r
+| | more than one CPU, the tests will be run twice -- once with |\r
+| | a single copy of each test running at once, and once with N |\r
+| | N copies, where N is the number of CPUs. |\r
+| | |\r
+| | UnixBench will process a set of results from a single test |\r
+| | by averaging the individual pass results into a single final |\r
+| | value. |\r
+| | |\r
+| | While the cpu test is running Energy scenario run in |\r
+| | background to monitor the number of watt consumed by the |\r
+| | compute server on the fly. The same is done using Cpuload |\r
+| | scenario to monitor the overall percentage of CPU used on |\r
+| | the fly. This enables to balance the CPU score with its |\r
+| | impact on energy consumption. Synchronized measurements |\r
+| | enables to look at any relation between CPU load and energy |\r
+| | consumption. |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|configuration | file: opnfv_yardstick_tc015.yaml |\r
+| | |\r
+| | run_mode: |\r
+| | Run Energy and Cpuload in background |\r
+| | Run unixbench in quiet mode or verbose mode |\r
+| | test_type: dhry2reg, whetstone and so on |\r
+| | |\r
+| | Duration and Interval are set globally for Energy and |\r
+| | Cpuload, aligned with duration of UnixBench test. |\r
+| | SLA can be set for each scenario type. Default is NA. |\r
+| | For SLA with single_score and parallel_score, both can be |\r
+| | set by user, default is NA. |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|applicability | Test shall be applied to node context only |\r
+| | It can be configured with different: |\r
+| | |\r
+| | * test types: dhry2reg, whetstone |\r
+| | |\r
+| | Default values exist. |\r
+| | |\r
+| | SLA (optional) : min_score: The minimun UnixBench score that |\r
+| | is accepted. |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|usability | This test case is one of Yardstick's generic test. Thus it |\r
+| | is runnable on most of the scenarios. |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|references | unixbench_ |\r
+| | |\r
+| | ETSI-NFV-TST001 |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|pre-test | The target shall have unixbench installed on it. |\r
+|conditions | |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|test sequence | description and expected result |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|step 1 | Yardstick is connected with the target node using ssh. |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|step 2 | Energy and Cpuload are launched silently in background one |\r
+| | after the other. |\r
+| | Then UnixBench is invoked. All the tests are executed using |\r
+| | the "Run" script in the top-level of UnixBench directory. |\r
+| | The "Run" script will run a standard "index" test, and save |\r
+| | the report in the "results" directory. Then the report is |\r
+| | processed by "unixbench_benchmark" and checked against the |\r
+| | SLA. |\r
+| | While unibench runs energy and cpu load are catched |\r
+| | periodically according to interval value. |\r
+| | |\r
+| | Result: Logs are stored. |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
+|test verdict | Fails only if SLA is not passed, or if there is a test case |\r
+| | execution problem. |\r
+| | |\r
++--------------+--------------------------------------------------------------+\r
--- /dev/null
+##############################################################################\r
+# Copyright (c) 2019 Orange and others.\r
+#\r
+# All rights reserved. This program and the accompanying materials\r
+# are made available under the terms of the Apache License, Version 2.0\r
+# which accompanies this distribution, and is available at\r
+# http://www.apache.org/licenses/LICENSE-2.0\r
+##############################################################################\r
+---\r
+# Sample benchmark task config file\r
+# Monitor energy consumption during a cpu test\r
+# Should be used inside test cases where UnixBench is replaced by other\r
+# scenario type. This enables to easily consider energy consumption as a\r
+# mandatory metric for any performance test\r
+\r
+schema: "yardstick:task:0.1"\r
+\r
+{% set target = target or "node5"%}\r
+{% set file = file or '/etc/yardstick/pod.yaml' %}\r
+{% set duree = 60 %}\r
+{% set intervalle = 5 %}\r
+scenarios:\r
+\r
+-\r
+ type: Energy\r
+ run_in_background: true\r
+ target: {{target}}.LF\r
+\r
+ runner:\r
+ type: Duration\r
+ duration: {{duree}}\r
+ interval: {{intervalle}}\r
+\r
+ sla:\r
+ action: monitor\r
+\r
+-\r
+ type: UnixBench\r
+ options:\r
+ run_mode: 'verbose'\r
+ test_type: 'dhry2reg'\r
+ host: {{target}}.LF\r
+\r
+ runner:\r
+ type: Iteration\r
+ iterations: 1\r
+ interval: 1\r
+\r
+\r
+context:\r
+ type: Node\r
+ name: LF\r
+ file: {{file}}\r
--- /dev/null
+##############################################################################\r
+# Copyright (c) 2019 Orange and others.\r
+#\r
+# All rights reserved. This program and the accompanying materials\r
+# are made available under the terms of the Apache License, Version 2.0\r
+# which accompanies this distribution, and is available at\r
+# http://www.apache.org/licenses/LICENSE-2.0\r
+##############################################################################\r
+---\r
+\r
+schema: "yardstick:task:0.1"\r
+description: >\r
+ Yardstick TC015 config file;\r
+ Measure Processing speed using unixbench with its impact on energy consumption and CPU load.\r
+\r
+{% set target = target or "node5"%}\r
+{% set file = file or '/etc/yardstick/pod.yaml' %}\r
+{% set duree = 60 %}\r
+{% set intervalle = 5 %}\r
+scenarios:\r
+\r
+-\r
+ type: Energy\r
+ run_in_background: true\r
+ target: {{target}}.LF\r
+\r
+ runner:\r
+ type: Duration\r
+ duration: {{duree}}\r
+ interval: {{intervalle}}\r
+\r
+-\r
+ type: CPUload\r
+ run_in_background: true\r
+ options:\r
+ interval: 1\r
+ count: 1\r
+ host: {{target}}.LF\r
+ runner:\r
+ type: Duration\r
+ duration: {{duree}}\r
+ interval: {{intervalle}}\r
+\r
+-\r
+ type: UnixBench\r
+ options:\r
+ run_mode: 'verbose'\r
+ test_type: 'dhry2reg'\r
+ host: {{target}}.LF\r
+\r
+ runner:\r
+ type: Iteration\r
+ iterations: 1\r
+ interval: 1\r
+\r
+\r
+context:\r
+ type: Node\r
+ name: LF\r
+ file: {{file}}\r