docs/testing/user/userguide/opnfv_yardstick_tc004.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International
   2 .. License.
   3 .. http://creativecommons.org/licenses/by/4.0
   4 .. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
   5
   6 *************************************
   7 Yardstick Test Case Description TC004
   8 *************************************
   9
  10 .. _cachestat: https://github.com/brendangregg/perf-tools/tree/master/fs
  11
  12 +-----------------------------------------------------------------------------+
  13 |Cache Utilization                                                            |
  14 |                                                                             |
  15 +--------------+--------------------------------------------------------------+
  16 |test case id  | OPNFV_YARDSTICK_TC004_CACHE Utilization                      |
  17 |              |                                                              |
  18 +--------------+--------------------------------------------------------------+
  19 |metric        | cache hit, cache miss, hit/miss ratio, buffer size and page  |
  20 |              | cache size                                                   |
  21 |              |                                                              |
  22 +--------------+--------------------------------------------------------------+
  23 |test purpose  | The purpose of TC004 is to evaluate the IaaS compute         |
  24 |              | capability with regards to cache utilization.This test case  |
  25 |              | should be run in parallel with other Yardstick test cases    |
  26 |              | and not run as a stand-alone test case.                      |
  27 |              |                                                              |
  28 |              | This test case measures cache usage statistics, including    |
  29 |              | cache hit, cache miss, hit ratio, buffer cache size and page |
  30 |              | cache size, with some wokloads runing on the infrastructure. |
  31 |              | Both average and maximun values are collected.               |
  32 |              |                                                              |
  33 |              | The purpose is also to be able to spot the trends.           |
  34 |              | Test results, graphs and similar shall be stored for         |
  35 |              | comparison reasons and product evolution understanding       |
  36 |              | between different OPNFV versions and/or configurations.      |
  37 |              |                                                              |
  38 +--------------+--------------------------------------------------------------+
  39 |test tool     | cachestat                                                    |
  40 |              |                                                              |
  41 |              | cachestat is a tool using Linux ftrace capabilities for      |
  42 |              | showing Linux page cache hit/miss statistics.                |
  43 |              |                                                              |
  44 |              | (cachestat is not always part of a Linux distribution, hence |
  45 |              | it needs to be installed. As an example see the              |
  46 |              | /yardstick/tools/ directory for how to generate a Linux      |
  47 |              | image with cachestat included.)                              |
  48 |              |                                                              |
  49 +--------------+--------------------------------------------------------------+
  50 |test          | cachestat test is invoked in a host VM on a compute blade,   |
  51 |description   | cachestat test requires some other test cases running in the |
  52 |              | host to stimulate workload.                                  |
  53 |              |                                                              |
  54 +--------------+--------------------------------------------------------------+
  55 |configuration | File: cachestat.yaml (in the 'samples' directory)            |
  56 |              |                                                              |
  57 |              | Interval is set 1. Test repeat, pausing every 1 seconds      |
  58 |              | in-between.                                                  |
  59 |              | Test durarion is set to 60 seconds.                          |
  60 |              |                                                              |
  61 |              | SLA is not available in this test case.                      |
  62 |              |                                                              |
  63 +--------------+--------------------------------------------------------------+
  64 |applicability | Test can be configured with different:                       |
  65 |              |                                                              |
  66 |              |  * interval;                                                 |
  67 |              |  * runner Duration.                                          |
  68 |              |                                                              |
  69 |              | Default values exist.                                        |
  70 |              |                                                              |
  71 +--------------+--------------------------------------------------------------+
  72 |usability     | This test case is one of Yardstick's generic test. Thus it   |
  73 |              | is runnable on most of the scenarios.                        |
  74 |              |                                                              |
  75 +--------------+--------------------------------------------------------------+
  76 |references    | cachestat_                                                   |
  77 |              |                                                              |
  78 |              | ETSI-NFV-TST001                                              |
  79 |              |                                                              |
  80 +--------------+--------------------------------------------------------------+
  81 |pre-test      | The test case image needs to be installed into Glance        |
  82 |conditions    | with cachestat included in the image.                        |
  83 |              |                                                              |
  84 |              | No POD specific requirements have been identified.           |
  85 |              |                                                              |
  86 +--------------+--------------------------------------------------------------+
  87 |test sequence | description and expected result                              |
  88 |              |                                                              |
  89 +--------------+--------------------------------------------------------------+
  90 |step 1        | A host VM with cachestat installed is booted.                |
  91 |              |                                                              |
  92 +--------------+--------------------------------------------------------------+
  93 |step 2        | Yardstick is connected with the host VM by using ssh.        |
  94 |              | 'cache_stat' bash script is copyied from Jump Host to        |
  95 |              | the server VM via the ssh tunnel.                            |
  96 |              |                                                              |
  97 +--------------+--------------------------------------------------------------+
  98 |step 3        | 'cache_stat' script is invoked. Raw cache usage statistics   |
  99 |              | are collected and filtrated. Average and maximum values are  |
 100 |              | calculated and recorded. Logs are produced and stored.       |
 101 |              |                                                              |
 102 |              | Result: Logs are stored.                                     |
 103 |              |                                                              |
 104 +--------------+--------------------------------------------------------------+
 105 |step 4        | The host VM is deleted.                                      |
 106 |              |                                                              |
 107 +--------------+--------------------------------------------------------------+
 108 |test verdict  | None. Cache utilization results are collected and stored.    |
 109 |              |                                                              |
 110 +--------------+--------------------------------------------------------------+