1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, Ericsson AB and others.
6 *************************************
7 Yardstick Test Case Description TC012
8 *************************************
10 .. _bw_mem: http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html
12 +-----------------------------------------------------------------------------+
15 +--------------+--------------------------------------------------------------+
16 |test case id | OPNFV_YARDSTICK_TC012_MEMORY BANDWIDTH |
18 +--------------+--------------------------------------------------------------+
19 |metric | Memory read/write bandwidth (MBps) |
21 +--------------+--------------------------------------------------------------+
22 |test purpose | The purpose of TC012 is to evaluate the IaaS compute |
23 | | performance with regards to memory throughput. |
24 | | It measures the rate at which data can be read from and |
25 | | written to the memory (this includes all levels of memory). |
27 | | The purpose is also to be able to spot the trends. |
28 | | Test results, graphs and similar shall be stored for |
29 | | comparison reasons and product evolution understanding |
30 | | between different OPNFV versions and/or configurations. |
32 +--------------+--------------------------------------------------------------+
33 |test tool | LMbench |
35 | | LMbench is a suite of operating system microbenchmarks. |
36 | | This test uses bw_mem tool from that suite including: |
38 | | * Cached file read |
39 | | * Memory copy (bcopy) |
45 | | (LMbench is not always part of a Linux distribution, hence |
46 | | it needs to be installed. As an example see the |
47 | | /yardstick/tools/ directory for how to generate a Linux |
48 | | image with LMbench included.) |
50 +--------------+--------------------------------------------------------------+
51 |test | LMbench bw_mem benchmark allocates twice the specified |
52 |description | amount of memory, zeros it, and then times the copying of |
53 | | the first half to the second half. The benchmark is invoked |
54 | | in a host VM on a compute blade. Results are reported in |
55 | | megabytes moved per second. |
57 +--------------+--------------------------------------------------------------+
58 |configuration | File: opnfv_yardstick_tc012.yaml |
60 | | * SLA (optional): 15000 (MBps) min_bw: The minimum amount of |
61 | | memory bandwidth that is accepted. |
62 | | * Size: 10 240 kB - test allocates twice that size |
63 | | (20 480kB) zeros it and then measures the time it takes to |
64 | | copy from one side to another. |
65 | | * Benchmark: rdwr - measures the time to read data into |
66 | | memory and then write data to the same location. |
67 | | * Warmup: 0 - the number of iterations to perform before |
68 | | taking actual measurements. |
69 | | * Iterations: 10 - test is run 10 times iteratively. |
70 | | * Interval: 1 - there is 1 second delay between each |
73 | | SLA is optional. The SLA in this test case serves as an |
74 | | example. Considerably higher bandwidth is expected. |
75 | | However, to cover most configurations, both baremetal and |
76 | | fully virtualized ones, this value should be possible to |
77 | | achieve and acceptable for black box testing. |
78 | | Many heavy IO applications start to suffer badly if the |
79 | | read/write bandwidths are lower than this. |
81 +--------------+--------------------------------------------------------------+
82 |applicability | Test can be configured with different: |
85 | | * memory operations (such as rd, wr, rdwr, cp, frd, fwr, |
86 | | fcp, bzero, bcopy); |
87 | | * number of warmup iterations; |
88 | | * iterations and intervals. |
90 | | Default values exist. |
92 | | SLA (optional) : min_bandwidth: The minimun memory bandwidth |
93 | | that is accepted. |
95 +--------------+--------------------------------------------------------------+
96 |usability | This test case is one of Yardstick's generic test. Thus it |
97 | | is runnable on most of the scenarios. |
99 +--------------+--------------------------------------------------------------+
100 |references | LMbench bw_mem_ |
102 | | ETSI-NFV-TST001 |
104 +--------------+--------------------------------------------------------------+
105 |pre-test | The test case image needs to be installed into Glance |
106 |conditions | with Lmbench included in the image. |
108 | | No POD specific requirements have been identified. |
110 +--------------+--------------------------------------------------------------+
111 |test sequence | description and expected result |
113 +--------------+--------------------------------------------------------------+
114 |step 1 | A host VM with LMbench installed is booted. |
116 +--------------+--------------------------------------------------------------+
117 |step 2 | Yardstick is connected with the host VM by using ssh. |
118 | | "lmbench_bandwidth_benchmark" bash script is copied from |
119 | | Jump Host to the host VM via ssh tunnel. |
121 +--------------+--------------------------------------------------------------+
122 |step 3 | 'lmbench_bandwidth_benchmark' script is invoked. LMbench's |
123 | | bw_mem benchmark starts to measures memory read/write |
124 | | bandwidth. Memory read/write bandwidth results are recorded |
125 | | and checked against the SLA. Logs are produced and stored. |
127 | | Result: Logs are stored. |
129 +--------------+--------------------------------------------------------------+
130 |step 4 | The host VM is deleted. |
132 +--------------+--------------------------------------------------------------+
133 |test verdict | Test fails if the measured memory bandwidth is below the SLA |
134 | | value or if there is a test case execution problem. |
136 +--------------+--------------------------------------------------------------+