1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
4 .. (c) OPNFV, Ericsson AB and others.
6 *************************************
7 Yardstick Test Case Description TC010
8 *************************************
10 .. _lat_mem_rd: http://manpages.ubuntu.com/manpages/trusty/lat_mem_rd.8.html
12 +-----------------------------------------------------------------------------+
15 +--------------+--------------------------------------------------------------+
16 |test case id | OPNFV_YARDSTICK_TC010_MEMORY LATENCY |
18 +--------------+--------------------------------------------------------------+
19 |metric | Memory read latency (nanoseconds) |
21 +--------------+--------------------------------------------------------------+
22 |test purpose | The purpose of TC010 is to evaluate the IaaS compute |
23 | | performance with regards to memory read latency. |
24 | | It measures the memory read latency for varying memory sizes |
25 | | and strides. Whole memory hierarchy is measured. |
27 | | The purpose is also to be able to spot the trends. |
28 | | Test results, graphs and similar shall be stored for |
29 | | comparison reasons and product evolution understanding |
30 | | between different OPNFV versions and/or configurations. |
32 +--------------+--------------------------------------------------------------+
33 |test tool | Lmbench |
35 | | Lmbench is a suite of operating system microbenchmarks. This |
36 | | test uses lat_mem_rd tool from that suite including: |
37 | | * Context switching |
38 | | * Networking: connection establishment, pipe, TCP, UDP, and |
40 | | * File system creates and deletes |
41 | | * Process creation |
42 | | * Signal handling |
43 | | * System call overhead |
44 | | * Memory read latency |
46 | | (LMbench is not always part of a Linux distribution, hence |
47 | | it needs to be installed. As an example see the |
48 | | /yardstick/tools/ directory for how to generate a Linux |
49 | | image with LMbench included.) |
51 +--------------+--------------------------------------------------------------+
52 |test | LMbench lat_mem_rd benchmark measures memory read latency |
53 |description | for varying memory sizes and strides. |
55 | | The benchmark runs as two nested loops. The outer loop is |
56 | | the stride size. The inner loop is the array size. For each |
57 | | array size, the benchmark creates a ring of pointers that |
58 | | point backward one stride.Traversing the array is done by: |
60 | | p = (char **)*p; |
62 | | in a for loop (the over head of the for loop is not |
63 | | significant; the loop is an unrolled loop 100 loads long). |
64 | | The size of the array varies from 512 bytes to (typically) |
65 | | eight megabytes. For the small sizes, the cache will have an |
66 | | effect, and the loads will be much faster. This becomes much |
67 | | more apparent when the data is plotted. |
69 | | Only data accesses are measured; the instruction cache is |
72 | | The results are reported in nanoseconds per load and have |
73 | | been verified accurate to within a few nanoseconds on an SGI |
76 +--------------+--------------------------------------------------------------+
77 |configuration | File: opnfv_yardstick_tc010.yaml |
79 | | * SLA (max_latency): 30 nanoseconds |
80 | | * Stride - 128 bytes |
81 | | * Stop size - 64 megabytes |
82 | | * Iterations: 10 - test is run 10 times iteratively. |
83 | | * Interval: 1 - there is 1 second delay between each |
86 | | SLA is optional. The SLA in this test case serves as an |
87 | | example. Considerably lower read latency is expected. |
88 | | However, to cover most configurations, both baremetal and |
89 | | fully virtualized ones, this value should be possible to |
90 | | achieve and acceptable for black box testing. |
91 | | Many heavy IO applications start to suffer badly if the |
92 | | read latency is higher than this. |
94 +--------------+--------------------------------------------------------------+
95 |applicability | Test can be configured with different: |
99 | | * iterations and intervals. |
101 | | Default values exist. |
103 | | SLA (optional) : max_latency: The maximum memory latency |
104 | | that is accepted. |
106 +--------------+--------------------------------------------------------------+
107 |usability | This test case is one of Yardstick's generic test. Thus it |
108 | | is runnable on most of the scenarios. |
110 +--------------+--------------------------------------------------------------+
111 |references | LMbench lat_mem_rd_ |
113 | | ETSI-NFV-TST001 |
115 +--------------+--------------------------------------------------------------+
116 |pre-test | The test case image needs to be installed into Glance |
117 |conditions | with Lmbench included in the image. |
119 | | No POD specific requirements have been identified. |
121 +--------------+--------------------------------------------------------------+
122 |test sequence | description and expected result |
124 +--------------+--------------------------------------------------------------+
125 |step 1 | The host is installed as client. LMbench's lat_mem_rd tool |
126 | | is invoked and logs are produced and stored. |
128 | | Result: logs are stored. |
130 +--------------+--------------------------------------------------------------+
131 |step 1 | A host VM with LMbench installed is booted. |
133 +--------------+--------------------------------------------------------------+
134 |step 2 | Yardstick is connected with the host VM by using ssh. |
135 | | 'lmbench_latency_benchmark' bash script is copyied from Jump |
136 | | Host to the host VM via the ssh tunnel. |
138 +--------------+--------------------------------------------------------------+
139 |step 3 | 'lmbench_latency_benchmark' script is invoked. LMbench's |
140 | | lat_mem_rd benchmark starts to measures memory read latency |
141 | | for varying memory sizes and strides. Memory read latency |
142 | | are recorded and checked against the SLA. Logs are produced |
145 | | Result: Logs are stored. |
147 +--------------+--------------------------------------------------------------+
148 |step 4 | The host VM is deleted. |
150 +--------------+--------------------------------------------------------------+
151 |test verdict | Test fails if the measured memory latency is above the SLA |
152 | | value or if there is a test case execution problem. |
154 +--------------+--------------------------------------------------------------+