1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
6 Convention for heading levels in Yardstick:
7 ======= Heading 0 (reserved for the title in a document)
12 Avoid deeper levels because they do not render well.
14 Test results analysis for Euphrates and Fraser releases
15 =======================================================
20 The round-trip-time (RTT) between 2 VMs on different blades is measured using
23 Most test run measurements result on average between 0.39 and 4.00 ms.
24 Compared with Euphrates release, the average RTT result of the same pod experiences
25 a slight decline in Fraser release. For example, the average RTT of arm-pod5 is
26 1.518 in Ehphrates and 1.714 in Fraser. The average RTT of intel-pod18 is 1.6575
27 ms in Ehphrates and 1.856 ms in Fraser.
31 "huawei-pod2:stable/euphrates": [0.3925],
33 "lf-pod2:stable/euphrates": [0.5315],
35 "lf-pod1:stable/euphrates": [0.62],
37 "huawei-pod2:stable/fraser": [0.677],
39 "lf-pod1:stable/fraser": [0.725],
41 "flex-pod2:stable/euphrates": [0.795],
43 "huawei-pod12:stable/euphrates": [0.87],
45 "ericsson-pod1:stable/fraser": [0.9165],
47 "huawei-pod12:stable/fraser": [1.0465],
49 "lf-pod2:stable/fraser": [1.2325],
51 "intel-pod5:stable/euphrates": [1.25],
53 "ericsson-virtual3:stable/euphrates": [1.2655],
55 "ericsson-pod1:stable/euphrates": [1.372],
57 "zte-pod2:stable/fraser": [1.395],
59 "arm-pod5:stable/euphrates": [1.518],
61 "huawei-virtual4:stable/euphrates": [1.5355],
63 "ericsson-virtual4:stable/fraser": [1.582],
65 "huawei-virtual3:stable/euphrates": [1.606],
67 "intel-pod18:stable/euphrates": [1.6575],
69 "huawei-virtual4:stable/fraser": [1.697],
71 "huawei-virtual8:stable/euphrates": [1.709],
73 "arm-pod5:stable/fraser": [1.714],
75 "huawei-virtual3:stable/fraser": [1.716],
77 "intel-pod18:stable/fraser": [1.856],
79 "huawei-virtual2:stable/euphrates": [1.872],
81 "arm-pod6:stable/euphrates": [1.895],
83 "huawei-virtual2:stable/fraser": [1.964],
85 "huawei-virtual1:stable/fraser": [1.9765],
87 "huawei-virtual9:stable/euphrates": [2.0745],
89 "arm-pod6:stable/fraser": [2.209],
91 "huawei-virtual1:stable/euphrates": [2.495],
93 "ericsson-virtual2:stable/euphrates": [2.7895],
95 "ericsson-virtual4:stable/euphrates": [3.768],
97 "ericsson-virtual1:stable/euphrates": [3.8035],
99 "ericsson-virtual3:stable/fraser": [3.9175],
101 "ericsson-virtual2:stable/fraser": [4.004]
108 The tool we use to measure memory read latency is lmbench, which is a series of
109 micro benchmarks intended to measure basic operating system and hardware system
110 metrics. Compared with Euphrates release, the memory read latency of the same pod
111 also experience a slight decline. Virtual pods seem to have a higher memory read
112 latency than physical pods. Compared with X86 pods, the memory read latency of
113 arm pods is significant higher.
117 "ericsson-pod1:stable/euphrates": [5.7785],
119 "flex-pod2:stable/euphrates": [5.908],
121 "ericsson-virtual1:stable/euphrates": [6.412],
123 "intel-pod18:stable/euphrates": [6.5905],
125 "intel-pod5:stable/euphrates": [6.6975],
127 "ericsson-pod1:stable/fraser": [7.0645],
129 "ericsson-virtual4:stable/euphrates": [7.183],
131 "intel-pod18:stable/fraser": [7.4465],
133 "zte-pod2:stable/fraser": [8.1865],
135 "ericsson-virtual2:stable/euphrates": [8.4985],
137 "huawei-pod2:stable/euphrates": [8.877],
139 "huawei-pod12:stable/euphrates": [9.091],
141 "huawei-pod2:stable/fraser": [9.236],
143 "huawei-pod12:stable/fraser": [9.615],
145 "ericsson-virtual3:stable/euphrates": [9.719],
147 "ericsson-virtual2:stable/fraser": [9.8925],
149 "huawei-virtual4:stable/euphrates": [10.1195],
151 "huawei-virtual3:stable/euphrates": [10.19],
153 "huawei-virtual2:stable/fraser": [10.22],
155 "huawei-virtual1:stable/euphrates": [10.3045],
157 "huawei-virtual9:stable/euphrates": [10.318],
159 "ericsson-virtual4:stable/fraser": [10.5465],
161 "ericsson-virtual3:stable/fraser": [10.9355],
163 "huawei-virtual3:stable/fraser": [10.95],
165 "huawei-virtual2:stable/euphrates": [11.274],
167 "huawei-virtual4:stable/fraser": [11.557],
169 "lf-pod1:stable/euphrates": [15.7025],
171 "lf-pod2:stable/euphrates": [15.8495],
173 "lf-pod2:stable/fraser": [16.5595],
175 "lf-pod1:stable/fraser": [16.8395],
177 "arm-pod5:stable/euphrates": [18.092],
179 "arm-pod5:stable/fraser": [18.744],
181 "huawei-virtual1:stable/fraser": [19.8235],
183 "huawei-virtual8:stable/euphrates": [33.999],
185 "arm-pod6:stable/euphrates": [41.5605],
187 "arm-pod6:stable/fraser": [55.804]
194 Iperf3 is a tool for evaluating the packet delay variation between 2 VMs on
195 different blades. In general, the packet delay variation of the two releases
200 "arm-pod6:stable/fraser": [1],
202 "ericsson-pod1:stable/fraser": [1],
204 "ericsson-virtual2:stable/fraser": [1],
206 "ericsson-virtual3:stable/fraser": [1],
208 "lf-pod2:stable/fraser": [1],
210 "huawei-virtual1:stable/fraser": [2997],
212 "huawei-virtual2:stable/euphrates": [2997],
214 "flex-pod2:stable/euphrates": [2997.5],
216 "huawei-virtual3:stable/euphrates": [2998],
218 "huawei-virtual3:stable/fraser": [2999],
220 "huawei-virtual9:stable/euphrates": [3000],
222 "huawei-virtual8:stable/euphrates": [3001],
224 "huawei-virtual4:stable/euphrates": [3002],
226 "huawei-virtual4:stable/fraser": [3002],
228 "ericsson-virtual3:stable/euphrates": [3006],
230 "huawei-virtual1:stable/euphrates": [3007],
232 "ericsson-virtual2:stable/euphrates": [3009],
234 "intel-pod18:stable/euphrates": [3010],
236 "ericsson-virtual4:stable/euphrates": [3017],
238 "lf-pod2:stable/euphrates": [3021],
240 "arm-pod5:stable/euphrates": [3022],
242 "arm-pod6:stable/euphrates": [3022],
244 "ericsson-pod1:stable/euphrates": [3022],
246 "huawei-pod12:stable/euphrates": [3022],
248 "huawei-pod12:stable/fraser": [3022],
250 "huawei-pod2:stable/euphrates": [3022],
252 "huawei-pod2:stable/fraser": [3022],
254 "intel-pod18:stable/fraser": [3022],
256 "intel-pod5:stable/euphrates": [3022],
258 "lf-pod1:stable/euphrates": [3022],
260 "lf-pod1:stable/fraser": [3022],
262 "zte-pod2:stable/fraser": [3022],
264 "huawei-virtual2:stable/fraser": [3025]
271 Lmbench is also used to measure the memory read and write bandwidth.
272 Like TC010, compared with Euphrates release, the memory read and write bandwidth
273 of the same pod also experience a slight decline. And compared with X86 pods, the memory
274 read and write bandwidth of arm pods is significant lower.
278 "lf-pod1:stable/euphrates": [22912.39],
280 "lf-pod2:stable/euphrates": [22637.67],
282 "lf-pod1:stable/fraser": [20552.9],
284 "flex-pod2:stable/euphrates": [20229.99],
286 "lf-pod2:stable/fraser": [20058.925],
288 "ericsson-pod1:stable/fraser": [18930.78],
290 "intel-pod18:stable/fraser": [18757.545],
292 "ericsson-virtual1:stable/euphrates": [17474.965],
294 "ericsson-pod1:stable/euphrates": [17127.38],
296 "ericsson-virtual4:stable/euphrates": [16219.97],
298 "ericsson-virtual2:stable/euphrates": [15652.28],
300 "ericsson-virtual3:stable/euphrates": [15551.26],
302 "ericsson-virtual4:stable/fraser": [15389.465],
304 "ericsson-virtual2:stable/fraser": [15343.79],
306 "huawei-pod2:stable/euphrates": [15017.2],
308 "huawei-pod2:stable/fraser": [14870.78],
310 "huawei-virtual4:stable/euphrates": [14266.34],
312 "huawei-virtual1:stable/euphrates": [14233.035],
314 "huawei-virtual3:stable/euphrates": [14227.63],
316 "zte-pod2:stable/fraser": [14157.99],
318 "huawei-pod12:stable/euphrates": [14147.245],
320 "huawei-pod12:stable/fraser": [14126.99],
322 "intel-pod18:stable/euphrates": [14058.33],
324 "huawei-virtual3:stable/fraser": [13929.67],
326 "huawei-virtual2:stable/euphrates": [13862.85],
328 "huawei-virtual4:stable/fraser": [13847.155],
330 "huawei-virtual2:stable/fraser": [13702.92],
332 "huawei-virtual1:stable/fraser": [13496.45],
334 "intel-pod5:stable/euphrates": [13280.32],
336 "ericsson-virtual3:stable/fraser": [12733.19],
338 "huawei-virtual9:stable/euphrates": [12559.445],
340 "huawei-virtual8:stable/euphrates": [8998.02],
342 "arm-pod5:stable/euphrates": [4388.875],
344 "arm-pod5:stable/fraser": [4326.11],
346 "arm-pod6:stable/euphrates": [4260.2],
348 "arm-pod6:stable/fraser": [3809.885]
355 The Unixbench is used to evaluate the IaaS processing speed with regards to
356 score of single CPU running and parallel running. Below are the single CPU running
357 scores. It can be seen that the processing test results vary from scores 715 to 3737.
358 In general, the single CPU score of the two releases look similar.
362 "lf-pod2:stable/fraser": [3737.6],
364 "lf-pod2:stable/euphrates": [3723.95],
366 "lf-pod1:stable/fraser": [3702.7],
368 "lf-pod1:stable/euphrates": [3669],
370 "intel-pod5:stable/euphrates": [3388.6],
372 "intel-pod18:stable/euphrates": [3298.4],
374 "flex-pod2:stable/euphrates": [3208.6],
376 "ericsson-pod1:stable/fraser": [3131.6],
378 "intel-pod18:stable/fraser": [3098.1],
380 "ericsson-virtual1:stable/euphrates": [2988.9],
382 "zte-pod2:stable/fraser": [2831.4],
384 "ericsson-pod1:stable/euphrates": [2669.1],
386 "ericsson-virtual4:stable/euphrates": [2598.5],
388 "ericsson-virtual2:stable/fraser": [2559.7],
390 "ericsson-virtual3:stable/euphrates": [2553.15],
392 "huawei-pod2:stable/euphrates": [2531.2],
394 "huawei-pod2:stable/fraser": [2528.9],
396 "ericsson-virtual4:stable/fraser": [2527.8],
398 "ericsson-virtual2:stable/euphrates": [2526.9],
400 "huawei-virtual4:stable/euphrates": [2407.4],
402 "huawei-virtual3:stable/fraser": [2379.1],
404 "huawei-virtual3:stable/euphrates": [2374.6],
406 "huawei-virtual4:stable/fraser": [2362.1],
408 "huawei-virtual2:stable/euphrates": [2326.4],
410 "huawei-virtual9:stable/euphrates": [2324.95],
412 "huawei-virtual1:stable/euphrates": [2302.6],
414 "huawei-virtual2:stable/fraser": [2299.3],
416 "huawei-pod12:stable/euphrates": [2232.2],
418 "huawei-pod12:stable/fraser": [2229],
420 "huawei-virtual1:stable/fraser": [2171.3],
422 "ericsson-virtual3:stable/fraser": [2104.8],
424 "huawei-virtual8:stable/euphrates": [2085.3],
426 "arm-pod5:stable/fraser": [1764.2],
428 "arm-pod5:stable/euphrates": [1754.4],
430 "arm-pod6:stable/euphrates": [716.15],
432 "arm-pod6:stable/fraser": [715.4]
439 With the block size changing from 1 kb to 512 kb, the memory write bandwidth
440 tends to become larger first and then smaller within every run test. Below are
441 the scores for 32mb block array.
445 "intel-pod18:stable/euphrates": [18871.79],
447 "intel-pod18:stable/fraser": [16939.24],
449 "intel-pod5:stable/euphrates": [16055.79],
451 "arm-pod6:stable/euphrates": [13327.02],
453 "arm-pod6:stable/fraser": [11895.71],
455 "flex-pod2:stable/euphrates": [9384.585],
457 "zte-pod2:stable/fraser": [9375.33],
459 "ericsson-pod1:stable/euphrates": [9331.535],
461 "huawei-pod12:stable/euphrates": [9164.88],
463 "ericsson-pod1:stable/fraser": [9140.42],
465 "huawei-pod2:stable/euphrates": [9026.52],
467 "huawei-pod12:stable/fraser": [8993.37],
469 "huawei-virtual9:stable/euphrates": [8825.805],
471 "huawei-pod2:stable/fraser": [8794.01],
473 "huawei-virtual2:stable/fraser": [7670.21],
475 "ericsson-virtual1:stable/euphrates": [7615.97],
477 "ericsson-virtual4:stable/euphrates": [7539.23],
479 "arm-pod5:stable/fraser": [7479.32],
481 "arm-pod5:stable/euphrates": [7403.38],
483 "huawei-virtual3:stable/euphrates": [7247.89],
485 "ericsson-virtual2:stable/fraser": [7219.21],
487 "huawei-virtual2:stable/euphrates": [7205.35],
489 "huawei-virtual1:stable/euphrates": [7196.405],
491 "ericsson-virtual3:stable/euphrates": [7173.72],
493 "huawei-virtual4:stable/euphrates": [7131.47],
495 "ericsson-virtual2:stable/euphrates": [7129.08],
497 "huawei-virtual4:stable/fraser": [7059.045],
499 "huawei-virtual3:stable/fraser": [7023.57],
501 "lf-pod1:stable/euphrates": [6928.18],
503 "lf-pod2:stable/euphrates": [6875.88],
505 "lf-pod2:stable/fraser": [6834.7],
507 "lf-pod1:stable/fraser": [6775.27],
509 "ericsson-virtual4:stable/fraser": [6522.86],
511 "ericsson-virtual3:stable/fraser": [5835.59],
513 "huawei-virtual8:stable/euphrates": [5729.705],
515 "huawei-virtual1:stable/fraser": [5617.12]
522 For this test case, we use perf to measure context-switches under load.
523 High context switch rates are not themselves an issue, but they may point the
524 way to a more significant problem.
528 "zte-pod2:stable/fraser": [306.5],
530 "huawei-pod12:stable/euphrates": [316],
532 "lf-pod2:stable/fraser": [337.5],
534 "intel-pod18:stable/euphrates": [340],
536 "intel-pod18:stable/fraser": [343.5],
538 "intel-pod5:stable/euphrates": [357.5],
540 "ericsson-pod1:stable/euphrates": [384],
542 "lf-pod2:stable/euphrates": [394.5],
544 "huawei-pod12:stable/fraser": [399],
546 "lf-pod1:stable/euphrates": [435],
548 "lf-pod1:stable/fraser": [454],
550 "flex-pod2:stable/euphrates": [476],
552 "huawei-pod2:stable/euphrates": [518],
554 "huawei-pod2:stable/fraser": [544.5],
556 "arm-pod5:stable/euphrates": [869.5],
558 "huawei-virtual9:stable/euphrates": [1002],
560 "huawei-virtual4:stable/fraser": [1138],
562 "huawei-virtual4:stable/euphrates": [1174],
564 "huawei-virtual3:stable/euphrates": [1239],
566 "ericsson-pod1:stable/fraser": [1305],
568 "huawei-virtual2:stable/euphrates": [1430],
570 "huawei-virtual3:stable/fraser": [1433],
572 "huawei-virtual1:stable/fraser": [1470],
574 "huawei-virtual1:stable/euphrates": [1489],
576 "arm-pod6:stable/fraser": [1738.5],
578 "arm-pod6:stable/euphrates": [1883.5]
585 TC083 measures network latency and throughput between VMs using netperf.
586 The test results shown below are for UDP throughout.
590 "lf-pod1:stable/euphrates": [2204.42],
592 "lf-pod2:stable/fraser": [1893.39],
594 "intel-pod18:stable/euphrates": [1835.55],
596 "lf-pod2:stable/euphrates": [1676.705],
598 "intel-pod5:stable/euphrates": [1612.555],
600 "zte-pod2:stable/fraser": [1543.995],
602 "lf-pod1:stable/fraser": [1480.86],
604 "intel-pod18:stable/fraser": [1417.015],
606 "flex-pod2:stable/euphrates": [1370.23],
608 "huawei-pod12:stable/euphrates": [1300.12]