1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
6 *************************************
7 Yardstick Test Case Description TC078
8 *************************************
10 .. _spec_cpu2006: https://www.spec.org/cpu2006/
12 +-----------------------------------------------------------------------------+
13 |Compute Performance |
15 +--------------+--------------------------------------------------------------+
16 |test case id | OPNFV_YARDSTICK_TC078_SPEC CPU 2006 |
18 +--------------+--------------------------------------------------------------+
19 |metric | compute-intensive performance |
21 +--------------+--------------------------------------------------------------+
22 |test purpose | The purpose of TC078 is to evaluate the IaaS compute |
23 | | performance by using SPEC CPU 2006 benchmark. The SPEC CPU |
24 | | 2006 benchmark has several different ways to measure |
25 | | computer performance. One way is to measure how fast the |
26 | | computer completes a single task; this is called a speed |
27 | | measurement. Another way is to measure how many tasks |
28 | | computer can accomplish in a certain amount of time; this is |
29 | | called a throughput, capacity or rate measurement. |
31 +--------------+--------------------------------------------------------------+
32 |test tool | SPEC CPU 2006 |
34 | | The SPEC CPU 2006 benchmark is SPEC's industry-standardized, |
35 | | CPU-intensive benchmark suite, stressing a system's |
36 | | processor, memory subsystem and compiler. This benchmark |
37 | | suite includes the SPECint benchmarks and the SPECfp |
38 | | benchmarks. The SPECint 2006 benchmark contains 12 different |
39 | | enchmark tests and the SPECfp 2006 benchmark contains 19 |
40 | | different benchmark tests. |
42 | | SPEC CPU 2006 is not always part of a Linux distribution. |
43 | | SPEC requires that users purchase a license and agree with |
44 | | their terms and conditions. For this test case, users must |
45 | | manually download cpu2006-1.2.iso from the SPEC website and |
46 | | save it under the yardstick/resources folder (e.g. /home/ |
47 | | opnfv/repos/yardstick/yardstick/resources/cpu2006-1.2.iso) |
48 | | SPEC CPU® 2006 benchmark is available for purchase via the |
49 | | SPEC order form (https://www.spec.org/order.html). |
51 +--------------+--------------------------------------------------------------+
52 |test | This test case uses SPEC CPU 2006 benchmark to measure |
53 |description | compute-intensive performance of hosts. |
55 +--------------+--------------------------------------------------------------+
56 |configuration | file: spec_cpu.yaml (in the 'samples' directory) |
58 | | benchmark_subset is set to int. |
60 | | SLA is not available in this test case. |
62 +--------------+--------------------------------------------------------------+
63 |applicability | Test can be configured with different: |
65 | | * benchmark_subset - a subset of SPEC CPU2006 benchmarks to |
67 | | * SPECint_benchmark - a SPECint benchmark to run; |
68 | | * SPECint_benchmark - a SPECfp benchmark to run; |
69 | | * output_format - desired report format; |
70 | | * runspec_config - SPEC CPU2006 config file provided to the |
72 | | * runspec_iterations - the number of benchmark iterations |
73 | | to execute. For a reportable run, must be 3; |
74 | | * runspec_tune - tuning to use (base, peak, or all). For a |
75 | | reportable run, must be either base or all. Reportable |
76 | | runs do base first, then (optionally) peak; |
77 | | * runspec_size - size of input data to run (test, train, or |
78 | | ref). Reportable runs ensure that your binaries can |
79 | | produce correct results with the test and train workloads |
81 +--------------+--------------------------------------------------------------+
82 |usability | This test case is used for executing SPEC CPU 2006 benchmark |
83 | | physical servers. The SPECint 2006 benchmark takes |
84 | | approximately 5 hours. |
86 +--------------+--------------------------------------------------------------+
87 |references | spec_cpu2006_ |
91 +--------------+--------------------------------------------------------------+
92 |pre-test | To run and install SPEC CPU2006, the following are required: |
93 |conditions | * For SPECint2006: Both C99 and C++98 compilers; |
94 | | * For SPECfp2006: All three of C99, C++98 and Fortran-95 |
96 | | * At least 8GB of disk space availabile on the system. |
98 +--------------+--------------------------------------------------------------+
99 |test sequence | description and expected result |
101 +--------------+--------------------------------------------------------------+
102 |step 1 | cpu2006-1.2.iso has been saved under the yardstick/resources |
103 | | folder (e.g. /home/opnfv/repos/yardstick/yardstick/resources |
104 | | /cpu2006-1.2.iso). Additional, to use your custom runspec |
105 | | config file you can save it under the yardstick/resources/ |
106 | | files folder and specify the config file name in the |
107 | | runspec_config parameter. |
109 +--------------+--------------------------------------------------------------+
110 |step 2 | Upload SPEC CPU2006 ISO to the target server and install |
111 | | SPEC CPU2006 via ansible. |
113 +--------------+--------------------------------------------------------------+
114 |step 3 | Yardstick is connected with the target server by using ssh. |
115 | | If custom runspec config file is used, this file is copyied |
116 | | from yardstick to the target server via the ssh tunnel. |
118 +--------------+--------------------------------------------------------------+
119 |step 4 | SPEC CPU2006 benchmark is invoked and SPEC CPU 2006 metrics |
122 +--------------+--------------------------------------------------------------+
123 |step 5 | Text, HTML, CSV, PDF, and Configuration file outputs for the |
124 | | SPEC CPU 2006 metrics are fetch from the server and stored |
125 | | under /tmp/result folder. |
127 +--------------+--------------------------------------------------------------+
128 |step 6 | uninstall SPEC CPU2006 and remove cpu2006-1.2.iso from the |
129 | | target server . |
131 +--------------+--------------------------------------------------------------+
132 |test verdict | None. SPEC CPU2006 results are collected and stored. |
134 +--------------+--------------------------------------------------------------+