1 .. This work is licensed under a Creative Commons Attribution 4.0 International
3 .. http://creativecommons.org/licenses/by/4.0
5 =======================================================
6 Test results analysis for Euphrates and Fraser releases
7 =======================================================
12 The round-trip-time (RTT) between 2 VMs on different blades is measured using
15 Most test run measurements result on average between 0.39 and 4.00 ms.
16 Compared with Euphrates release, the average RTT result of the same pod experiences
17 a slight decline in Fraser release. For example, the average RTT of arm-pod5 is
18 1.518 in Ehphrates and 1.714 in Fraser. The average RTT of intel-pod18 is 1.6575
19 ms in Ehphrates and 1.856 ms in Fraser.
23 "huawei-pod2:stable/euphrates": [0.3925],
25 "lf-pod2:stable/euphrates": [0.5315],
27 "lf-pod1:stable/euphrates": [0.62],
29 "huawei-pod2:stable/fraser": [0.677],
31 "lf-pod1:stable/fraser": [0.725],
33 "flex-pod2:stable/euphrates": [0.795],
35 "huawei-pod12:stable/euphrates": [0.87],
37 "ericsson-pod1:stable/fraser": [0.9165],
39 "huawei-pod12:stable/fraser": [1.0465],
41 "lf-pod2:stable/fraser": [1.2325],
43 "intel-pod5:stable/euphrates": [1.25],
45 "ericsson-virtual3:stable/euphrates": [1.2655],
47 "ericsson-pod1:stable/euphrates": [1.372],
49 "zte-pod2:stable/fraser": [1.395],
51 "arm-pod5:stable/euphrates": [1.518],
53 "huawei-virtual4:stable/euphrates": [1.5355],
55 "ericsson-virtual4:stable/fraser": [1.582],
57 "huawei-virtual3:stable/euphrates": [1.606],
59 "intel-pod18:stable/euphrates": [1.6575],
61 "huawei-virtual4:stable/fraser": [1.697],
63 "huawei-virtual8:stable/euphrates": [1.709],
65 "arm-pod5:stable/fraser": [1.714],
67 "huawei-virtual3:stable/fraser": [1.716],
69 "intel-pod18:stable/fraser": [1.856],
71 "huawei-virtual2:stable/euphrates": [1.872],
73 "arm-pod6:stable/euphrates": [1.895],
75 "huawei-virtual2:stable/fraser": [1.964],
77 "huawei-virtual1:stable/fraser": [1.9765],
79 "huawei-virtual9:stable/euphrates": [2.0745],
81 "arm-pod6:stable/fraser": [2.209],
83 "huawei-virtual1:stable/euphrates": [2.495],
85 "ericsson-virtual2:stable/euphrates": [2.7895],
87 "ericsson-virtual4:stable/euphrates": [3.768],
89 "ericsson-virtual1:stable/euphrates": [3.8035],
91 "ericsson-virtual3:stable/fraser": [3.9175],
93 "ericsson-virtual2:stable/fraser": [4.004]
100 The tool we use to measure memory read latency is lmbench, which is a series of
101 micro benchmarks intended to measure basic operating system and hardware system
102 metrics. Compared with Euphrates release, the memory read latency of the same pod
103 also experience a slight decline. Virtual pods seem to have a higher memory read
104 latency than physical pods. Compared with X86 pods, the memory read latency of
105 arm pods is significant higher.
109 "ericsson-pod1:stable/euphrates": [5.7785],
111 "flex-pod2:stable/euphrates": [5.908],
113 "ericsson-virtual1:stable/euphrates": [6.412],
115 "intel-pod18:stable/euphrates": [6.5905],
117 "intel-pod5:stable/euphrates": [6.6975],
119 "ericsson-pod1:stable/fraser": [7.0645],
121 "ericsson-virtual4:stable/euphrates": [7.183],
123 "intel-pod18:stable/fraser": [7.4465],
125 "zte-pod2:stable/fraser": [8.1865],
127 "ericsson-virtual2:stable/euphrates": [8.4985],
129 "huawei-pod2:stable/euphrates": [8.877],
131 "huawei-pod12:stable/euphrates": [9.091],
133 "huawei-pod2:stable/fraser": [9.236],
135 "huawei-pod12:stable/fraser": [9.615],
137 "ericsson-virtual3:stable/euphrates": [9.719],
139 "ericsson-virtual2:stable/fraser": [9.8925],
141 "huawei-virtual4:stable/euphrates": [10.1195],
143 "huawei-virtual3:stable/euphrates": [10.19],
145 "huawei-virtual2:stable/fraser": [10.22],
147 "huawei-virtual1:stable/euphrates": [10.3045],
149 "huawei-virtual9:stable/euphrates": [10.318],
151 "ericsson-virtual4:stable/fraser": [10.5465],
153 "ericsson-virtual3:stable/fraser": [10.9355],
155 "huawei-virtual3:stable/fraser": [10.95],
157 "huawei-virtual2:stable/euphrates": [11.274],
159 "huawei-virtual4:stable/fraser": [11.557],
161 "lf-pod1:stable/euphrates": [15.7025],
163 "lf-pod2:stable/euphrates": [15.8495],
165 "lf-pod2:stable/fraser": [16.5595],
167 "lf-pod1:stable/fraser": [16.8395],
169 "arm-pod5:stable/euphrates": [18.092],
171 "arm-pod5:stable/fraser": [18.744],
173 "huawei-virtual1:stable/fraser": [19.8235],
175 "huawei-virtual8:stable/euphrates": [33.999],
177 "arm-pod6:stable/euphrates": [41.5605],
179 "arm-pod6:stable/fraser": [55.804]
186 Iperf3 is a tool for evaluating the packet delay variation between 2 VMs on
187 different blades. In general, the packet delay variation of the two releases
192 "arm-pod6:stable/fraser": [1],
194 "ericsson-pod1:stable/fraser": [1],
196 "ericsson-virtual2:stable/fraser": [1],
198 "ericsson-virtual3:stable/fraser": [1],
200 "lf-pod2:stable/fraser": [1],
202 "huawei-virtual1:stable/fraser": [2997],
204 "huawei-virtual2:stable/euphrates": [2997],
206 "flex-pod2:stable/euphrates": [2997.5],
208 "huawei-virtual3:stable/euphrates": [2998],
210 "huawei-virtual3:stable/fraser": [2999],
212 "huawei-virtual9:stable/euphrates": [3000],
214 "huawei-virtual8:stable/euphrates": [3001],
216 "huawei-virtual4:stable/euphrates": [3002],
218 "huawei-virtual4:stable/fraser": [3002],
220 "ericsson-virtual3:stable/euphrates": [3006],
222 "huawei-virtual1:stable/euphrates": [3007],
224 "ericsson-virtual2:stable/euphrates": [3009],
226 "intel-pod18:stable/euphrates": [3010],
228 "ericsson-virtual4:stable/euphrates": [3017],
230 "lf-pod2:stable/euphrates": [3021],
232 "arm-pod5:stable/euphrates": [3022],
234 "arm-pod6:stable/euphrates": [3022],
236 "ericsson-pod1:stable/euphrates": [3022],
238 "huawei-pod12:stable/euphrates": [3022],
240 "huawei-pod12:stable/fraser": [3022],
242 "huawei-pod2:stable/euphrates": [3022],
244 "huawei-pod2:stable/fraser": [3022],
246 "intel-pod18:stable/fraser": [3022],
248 "intel-pod5:stable/euphrates": [3022],
250 "lf-pod1:stable/euphrates": [3022],
252 "lf-pod1:stable/fraser": [3022],
254 "zte-pod2:stable/fraser": [3022],
256 "huawei-virtual2:stable/fraser": [3025]
263 Lmbench is also used to measure the memory read and write bandwidth.
264 Like TC010, compared with Euphrates release, the memory read and write bandwidth
265 of the same pod also experience a slight decline. And compared with X86 pods, the memory
266 read and write bandwidth of arm pods is significant lower.
270 "lf-pod1:stable/euphrates": [22912.39],
272 "lf-pod2:stable/euphrates": [22637.67],
274 "lf-pod1:stable/fraser": [20552.9],
276 "flex-pod2:stable/euphrates": [20229.99],
278 "lf-pod2:stable/fraser": [20058.925],
280 "ericsson-pod1:stable/fraser": [18930.78],
282 "intel-pod18:stable/fraser": [18757.545],
284 "ericsson-virtual1:stable/euphrates": [17474.965],
286 "ericsson-pod1:stable/euphrates": [17127.38],
288 "ericsson-virtual4:stable/euphrates": [16219.97],
290 "ericsson-virtual2:stable/euphrates": [15652.28],
292 "ericsson-virtual3:stable/euphrates": [15551.26],
294 "ericsson-virtual4:stable/fraser": [15389.465],
296 "ericsson-virtual2:stable/fraser": [15343.79],
298 "huawei-pod2:stable/euphrates": [15017.2],
300 "huawei-pod2:stable/fraser": [14870.78],
302 "huawei-virtual4:stable/euphrates": [14266.34],
304 "huawei-virtual1:stable/euphrates": [14233.035],
306 "huawei-virtual3:stable/euphrates": [14227.63],
308 "zte-pod2:stable/fraser": [14157.99],
310 "huawei-pod12:stable/euphrates": [14147.245],
312 "huawei-pod12:stable/fraser": [14126.99],
314 "intel-pod18:stable/euphrates": [14058.33],
316 "huawei-virtual3:stable/fraser": [13929.67],
318 "huawei-virtual2:stable/euphrates": [13862.85],
320 "huawei-virtual4:stable/fraser": [13847.155],
322 "huawei-virtual2:stable/fraser": [13702.92],
324 "huawei-virtual1:stable/fraser": [13496.45],
326 "intel-pod5:stable/euphrates": [13280.32],
328 "ericsson-virtual3:stable/fraser": [12733.19],
330 "huawei-virtual9:stable/euphrates": [12559.445],
332 "huawei-virtual8:stable/euphrates": [8998.02],
334 "arm-pod5:stable/euphrates": [4388.875],
336 "arm-pod5:stable/fraser": [4326.11],
338 "arm-pod6:stable/euphrates": [4260.2],
340 "arm-pod6:stable/fraser": [3809.885]
347 The Unixbench is used to evaluate the IaaS processing speed with regards to
348 score of single CPU running and parallel running. Below are the single CPU running
349 scores. It can be seen that the processing test results vary from scores 715 to 3737.
350 In general, the single CPU score of the two releases look similar.
354 "lf-pod2:stable/fraser": [3737.6],
356 "lf-pod2:stable/euphrates": [3723.95],
358 "lf-pod1:stable/fraser": [3702.7],
360 "lf-pod1:stable/euphrates": [3669],
362 "intel-pod5:stable/euphrates": [3388.6],
364 "intel-pod18:stable/euphrates": [3298.4],
366 "flex-pod2:stable/euphrates": [3208.6],
368 "ericsson-pod1:stable/fraser": [3131.6],
370 "intel-pod18:stable/fraser": [3098.1],
372 "ericsson-virtual1:stable/euphrates": [2988.9],
374 "zte-pod2:stable/fraser": [2831.4],
376 "ericsson-pod1:stable/euphrates": [2669.1],
378 "ericsson-virtual4:stable/euphrates": [2598.5],
380 "ericsson-virtual2:stable/fraser": [2559.7],
382 "ericsson-virtual3:stable/euphrates": [2553.15],
384 "huawei-pod2:stable/euphrates": [2531.2],
386 "huawei-pod2:stable/fraser": [2528.9],
388 "ericsson-virtual4:stable/fraser": [2527.8],
390 "ericsson-virtual2:stable/euphrates": [2526.9],
392 "huawei-virtual4:stable/euphrates": [2407.4],
394 "huawei-virtual3:stable/fraser": [2379.1],
396 "huawei-virtual3:stable/euphrates": [2374.6],
398 "huawei-virtual4:stable/fraser": [2362.1],
400 "huawei-virtual2:stable/euphrates": [2326.4],
402 "huawei-virtual9:stable/euphrates": [2324.95],
404 "huawei-virtual1:stable/euphrates": [2302.6],
406 "huawei-virtual2:stable/fraser": [2299.3],
408 "huawei-pod12:stable/euphrates": [2232.2],
410 "huawei-pod12:stable/fraser": [2229],
412 "huawei-virtual1:stable/fraser": [2171.3],
414 "ericsson-virtual3:stable/fraser": [2104.8],
416 "huawei-virtual8:stable/euphrates": [2085.3],
418 "arm-pod5:stable/fraser": [1764.2],
420 "arm-pod5:stable/euphrates": [1754.4],
422 "arm-pod6:stable/euphrates": [716.15],
424 "arm-pod6:stable/fraser": [715.4]
431 With the block size changing from 1 kb to 512 kb, the memory write bandwidth
432 tends to become larger first and then smaller within every run test. Below are
433 the scores for 32mb block array.
437 "intel-pod18:stable/euphrates": [18871.79],
439 "intel-pod18:stable/fraser": [16939.24],
441 "intel-pod5:stable/euphrates": [16055.79],
443 "arm-pod6:stable/euphrates": [13327.02],
445 "arm-pod6:stable/fraser": [11895.71],
447 "flex-pod2:stable/euphrates": [9384.585],
449 "zte-pod2:stable/fraser": [9375.33],
451 "ericsson-pod1:stable/euphrates": [9331.535],
453 "huawei-pod12:stable/euphrates": [9164.88],
455 "ericsson-pod1:stable/fraser": [9140.42],
457 "huawei-pod2:stable/euphrates": [9026.52],
459 "huawei-pod12:stable/fraser": [8993.37],
461 "huawei-virtual9:stable/euphrates": [8825.805],
463 "huawei-pod2:stable/fraser": [8794.01],
465 "huawei-virtual2:stable/fraser": [7670.21],
467 "ericsson-virtual1:stable/euphrates": [7615.97],
469 "ericsson-virtual4:stable/euphrates": [7539.23],
471 "arm-pod5:stable/fraser": [7479.32],
473 "arm-pod5:stable/euphrates": [7403.38],
475 "huawei-virtual3:stable/euphrates": [7247.89],
477 "ericsson-virtual2:stable/fraser": [7219.21],
479 "huawei-virtual2:stable/euphrates": [7205.35],
481 "huawei-virtual1:stable/euphrates": [7196.405],
483 "ericsson-virtual3:stable/euphrates": [7173.72],
485 "huawei-virtual4:stable/euphrates": [7131.47],
487 "ericsson-virtual2:stable/euphrates": [7129.08],
489 "huawei-virtual4:stable/fraser": [7059.045],
491 "huawei-virtual3:stable/fraser": [7023.57],
493 "lf-pod1:stable/euphrates": [6928.18],
495 "lf-pod2:stable/euphrates": [6875.88],
497 "lf-pod2:stable/fraser": [6834.7],
499 "lf-pod1:stable/fraser": [6775.27],
501 "ericsson-virtual4:stable/fraser": [6522.86],
503 "ericsson-virtual3:stable/fraser": [5835.59],
505 "huawei-virtual8:stable/euphrates": [5729.705],
507 "huawei-virtual1:stable/fraser": [5617.12]
514 For this test case, we use perf to measure context-switches under load.
515 High context switch rates are not themselves an issue, but they may point the
516 way to a more significant problem.
520 "zte-pod2:stable/fraser": [306.5],
522 "huawei-pod12:stable/euphrates": [316],
524 "lf-pod2:stable/fraser": [337.5],
526 "intel-pod18:stable/euphrates": [340],
528 "intel-pod18:stable/fraser": [343.5],
530 "intel-pod5:stable/euphrates": [357.5],
532 "ericsson-pod1:stable/euphrates": [384],
534 "lf-pod2:stable/euphrates": [394.5],
536 "huawei-pod12:stable/fraser": [399],
538 "lf-pod1:stable/euphrates": [435],
540 "lf-pod1:stable/fraser": [454],
542 "flex-pod2:stable/euphrates": [476],
544 "huawei-pod2:stable/euphrates": [518],
546 "huawei-pod2:stable/fraser": [544.5],
548 "arm-pod5:stable/euphrates": [869.5],
550 "huawei-virtual9:stable/euphrates": [1002],
552 "huawei-virtual4:stable/fraser": [1138],
554 "huawei-virtual4:stable/euphrates": [1174],
556 "huawei-virtual3:stable/euphrates": [1239],
558 "ericsson-pod1:stable/fraser": [1305],
560 "huawei-virtual2:stable/euphrates": [1430],
562 "huawei-virtual3:stable/fraser": [1433],
564 "huawei-virtual1:stable/fraser": [1470],
566 "huawei-virtual1:stable/euphrates": [1489],
568 "arm-pod6:stable/fraser": [1738.5],
570 "arm-pod6:stable/euphrates": [1883.5]
577 TC083 measures network latency and throughput between VMs using netperf.
578 The test results shown below are for UDP throughout.
582 "lf-pod1:stable/euphrates": [2204.42],
584 "lf-pod2:stable/fraser": [1893.39],
586 "intel-pod18:stable/euphrates": [1835.55],
588 "lf-pod2:stable/euphrates": [1676.705],
590 "intel-pod5:stable/euphrates": [1612.555],
592 "zte-pod2:stable/fraser": [1543.995],
594 "lf-pod1:stable/fraser": [1480.86],
596 "intel-pod18:stable/fraser": [1417.015],
598 "flex-pod2:stable/euphrates": [1370.23],
600 "huawei-pod12:stable/euphrates": [1300.12]