1 .. This work is licensed under a Creative Commons Attribution 4.0 International
\r
3 .. http://creativecommons.org/licenses/by/4.0
\r
4 .. (c) OPNFV, Orange and others.
\r
6 *************************************
\r
7 Yardstick Test Case Description TC015
\r
8 *************************************
\r
10 .. _unixbench: https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench
\r
12 +-----------------------------------------------------------------------------+
\r
13 | Processing speed with impact on energy consumption and CPU load |
\r
15 +--------------+--------------------------------------------------------------+
\r
16 |test case id | OPNFV_YARDSTICK_TC015_PROCESSING SPEED |
\r
18 +--------------+--------------------------------------------------------------+
\r
19 |metric | score of single cpu running, |
\r
20 | | score of parallel running, |
\r
21 | | energy consumption |
\r
24 +--------------+--------------------------------------------------------------+
\r
25 |test purpose | The purpose of TC015 is to evaluate the IaaS compute |
\r
26 | | performance with regards to CPU processing speed with |
\r
27 | | its impact on the energy consumption |
\r
28 | | It measures score of single cpu running and parallel |
\r
29 | | running. Energy consumption and cpu load are monitored while |
\r
30 | | the cpu test is running. |
\r
32 | | The purpose is also to be able to spot the trends. |
\r
33 | | Test results, graphs and similar shall be stored for |
\r
34 | | comparison reasons and product evolution understanding |
\r
35 | | between different OPNFV versions and/or configurations, |
\r
36 | | different server types. |
\r
38 +--------------+--------------------------------------------------------------+
\r
39 |test tool | UnixBench |
\r
41 | | Unixbench is the most used CPU benchmarking software tool. |
\r
42 | | It can measure the performance of bash scripts, CPUs in |
\r
43 | | multithreading and single threading. It can also measure the |
\r
44 | | performance for parallel tasks. Also, specific disk IO for |
\r
45 | | small and large files are performed. You can use it to |
\r
46 | | measure either linux dedicated servers and linux vps |
\r
47 | | servers, running CentOS, Debian, Ubuntu, Fedora and other |
\r
50 | | (UnixBench is not always part of a Linux distribution, hence |
\r
51 | | it needs to be installed. As an example see the |
\r
52 | | /yardstick/tools/ directory for how to generate a Linux |
\r
53 | | image with UnixBench included.) |
\r
56 | | This HTTPS interface is provided by BMC of every telco grade |
\r
57 | | server. Is is a standard interface. |
\r
59 +--------------+--------------------------------------------------------------+
\r
60 |test | The UnixBench runs system benchmarks on a compute, getting |
\r
61 |description | information on the CPUs in the system. If the system has |
\r
62 | | more than one CPU, the tests will be run twice -- once with |
\r
63 | | a single copy of each test running at once, and once with N |
\r
64 | | N copies, where N is the number of CPUs. |
\r
66 | | UnixBench will process a set of results from a single test |
\r
67 | | by averaging the individual pass results into a single final |
\r
70 | | While the cpu test is running Energy scenario run in |
\r
71 | | background to monitor the number of watt consumed by the |
\r
72 | | compute server on the fly. The same is done using Cpuload |
\r
73 | | scenario to monitor the overall percentage of CPU used on |
\r
74 | | the fly. This enables to balance the CPU score with its |
\r
75 | | impact on energy consumption. Synchronized measurements |
\r
76 | | enables to look at any relation between CPU load and energy |
\r
79 +--------------+--------------------------------------------------------------+
\r
80 |configuration | file: opnfv_yardstick_tc015.yaml |
\r
83 | | Run Energy and Cpuload in background |
\r
84 | | Run unixbench in quiet mode or verbose mode |
\r
85 | | test_type: dhry2reg, whetstone and so on |
\r
87 | | Duration and Interval are set globally for Energy and |
\r
88 | | Cpuload, aligned with duration of UnixBench test. |
\r
89 | | SLA can be set for each scenario type. Default is NA. |
\r
90 | | For SLA with single_score and parallel_score, both can be |
\r
91 | | set by user, default is NA. |
\r
93 +--------------+--------------------------------------------------------------+
\r
94 |applicability | Test shall be applied to node context only |
\r
95 | | It can be configured with different: |
\r
97 | | * test types: dhry2reg, whetstone |
\r
99 | | Default values exist. |
\r
101 | | SLA (optional) : min_score: The minimun UnixBench score that |
\r
104 +--------------+--------------------------------------------------------------+
\r
105 |usability | This test case is one of Yardstick's generic test. Thus it |
\r
106 | | is runnable on most of the scenarios. |
\r
108 +--------------+--------------------------------------------------------------+
\r
109 |references | unixbench_ |
\r
111 | | ETSI-NFV-TST001 |
\r
113 +--------------+--------------------------------------------------------------+
\r
114 |pre-test | The target shall have unixbench installed on it. |
\r
117 +--------------+--------------------------------------------------------------+
\r
118 |test sequence | description and expected result |
\r
120 +--------------+--------------------------------------------------------------+
\r
121 |step 1 | Yardstick is connected with the target node using ssh. |
\r
123 +--------------+--------------------------------------------------------------+
\r
124 |step 2 | Energy and Cpuload are launched silently in background one |
\r
125 | | after the other. |
\r
126 | | Then UnixBench is invoked. All the tests are executed using |
\r
127 | | the "Run" script in the top-level of UnixBench directory. |
\r
128 | | The "Run" script will run a standard "index" test, and save |
\r
129 | | the report in the "results" directory. Then the report is |
\r
130 | | processed by "unixbench_benchmark" and checked against the |
\r
132 | | While unibench runs energy and cpu load are catched |
\r
133 | | periodically according to interval value. |
\r
135 | | Result: Logs are stored. |
\r
137 +--------------+--------------------------------------------------------------+
\r
138 |test verdict | Fails only if SLA is not passed, or if there is a test case |
\r
139 | | execution problem. |
\r
141 +--------------+--------------------------------------------------------------+
\r
Code Review / yardstick.git / blob