docs/testing/user/userguide/opnfv_yardstick_tc014.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International
   2 .. License.
   3 .. http://creativecommons.org/licenses/by/4.0
   4 .. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
   5
   6 *************************************
   7 Yardstick Test Case Description TC014
   8 *************************************
   9
  10 .. _unixbench: https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench
  11
  12 +-----------------------------------------------------------------------------+
  13 |Processing speed                                                             |
  14 |                                                                             |
  15 +--------------+--------------------------------------------------------------+
  16 |test case id  | OPNFV_YARDSTICK_TC014_PROCESSING SPEED                       |
  17 |              |                                                              |
  18 +--------------+--------------------------------------------------------------+
  19 |metric        | score of single cpu running,                                 |
  20 |              | score of parallel running                                    |
  21 |              |                                                              |
  22 +--------------+--------------------------------------------------------------+
  23 |test purpose  | The purpose of TC014 is to evaluate the IaaS compute         |
  24 |              | performance with regards to CPU processing speed.            |
  25 |              | It measures score of single cpu running and parallel         |
  26 |              | running.                                                     |
  27 |              |                                                              |
  28 |              | The purpose is also to be able to spot the trends.           |
  29 |              | Test results, graphs and similar shall be stored for         |
  30 |              | comparison reasons and product evolution understanding       |
  31 |              | between different OPNFV versions and/or configurations.      |
  32 |              |                                                              |
  33 +--------------+--------------------------------------------------------------+
  34 |test tool     | UnixBench                                                    |
  35 |              |                                                              |
  36 |              | Unixbench is the most used CPU benchmarking software tool.   |
  37 |              | It can measure the performance of bash scripts, CPUs in      |
  38 |              | multithreading and single threading. It can also measure the |
  39 |              | performance for parallel taks. Also, specific disk IO for    |
  40 |              | small and large files are performed. You can use it to       |
  41 |              | measure either linux dedicated servers and linux vps         |
  42 |              | servers, running CentOS, Debian, Ubuntu, Fedora and other    |
  43 |              | distros.                                                     |
  44 |              |                                                              |
  45 |              | (UnixBench is not always part of a Linux distribution, hence |
  46 |              | it needs to be installed. As an example see the              |
  47 |              | /yardstick/tools/ directory for how to generate a Linux      |
  48 |              | image with UnixBench included.)                              |
  49 |              |                                                              |
  50 +--------------+--------------------------------------------------------------+
  51 |test          | The UnixBench runs system benchmarks in a host VM on a       |
  52 |description   | compute blade, getting information on the CPUs in the        |
  53 |              | system. If the system has more than one CPU, the tests will  |
  54 |              | be run twice -- once with a single copy of each test running |
  55 |              | at once, and once with N copies, where N is the number of    |
  56 |              | CPUs.                                                        |
  57 |              |                                                              |
  58 |              | UnixBench will processs a set of results from a single test  |
  59 |              | by averaging the individal pass results into a single final  |
  60 |              | value.                                                       |
  61 |              |                                                              |
  62 +--------------+--------------------------------------------------------------+
  63 |configuration | file: opnfv_yardstick_tc014.yaml                             |
  64 |              |                                                              |
  65 |              | run_mode: Run unixbench in quiet mode or verbose mode        |
  66 |              | test_type: dhry2reg, whetstone and so on                     |
  67 |              |                                                              |
  68 |              | For SLA with single_score and parallel_score, both can be    |
  69 |              | set by user, default is NA.                                  |
  70 |              |                                                              |
  71 +--------------+--------------------------------------------------------------+
  72 |applicability | Test can be configured with different:                       |
  73 |              |                                                              |
  74 |              |  * test types;                                               |
  75 |              |  * dhry2reg;                                                 |
  76 |              |  * whetstone.                                                |
  77 |              |                                                              |
  78 |              | Default values exist.                                        |
  79 |              |                                                              |
  80 |              | SLA (optional) : min_score: The minimun UnixBench score that |
  81 |              | is accepted.                                                 |
  82 |              |                                                              |
  83 +--------------+--------------------------------------------------------------+
  84 |usability     | This test case is one of Yardstick's generic test. Thus it   |
  85 |              | is runnable on most of the scenarios.                        |
  86 |              |                                                              |
  87 +--------------+--------------------------------------------------------------+
  88 |references    | unixbench_                                                   |
  89 |              |                                                              |
  90 |              | ETSI-NFV-TST001                                              |
  91 |              |                                                              |
  92 +--------------+--------------------------------------------------------------+
  93 |pre-test      | The test case image needs to be installed into Glance        |
  94 |conditions    | with unixbench included in it.                               |
  95 |              |                                                              |
  96 |              | No POD specific requirements have been identified.           |
  97 |              |                                                              |
  98 +--------------+--------------------------------------------------------------+
  99 |test sequence | description and expected result                              |
 100 |              |                                                              |
 101 +--------------+--------------------------------------------------------------+
 102 |step 1        | A host VM with UnixBench installed is booted.                |
 103 |              |                                                              |
 104 +--------------+--------------------------------------------------------------+
 105 |step 2        | Yardstick is connected with the host VM by using ssh.        |
 106 |              | "unixbench_benchmark" bash script is copied from Jump Host   |
 107 |              | to the host VM via ssh tunnel.                               |
 108 |              |                                                              |
 109 +--------------+--------------------------------------------------------------+
 110 |step 3        | UnixBench is invoked. All the tests are executed using the   |
 111 |              | "Run" script in the top-level of UnixBench directory.        |
 112 |              | The "Run" script will run a standard "index" test, and save  |
 113 |              | the report in the "results" directory. Then the report is    |
 114 |              | processed by "unixbench_benchmark" and checked againsted the |
 115 |              | SLA.                                                         |
 116 |              |                                                              |
 117 |              | Result: Logs are stored.                                     |
 118 |              |                                                              |
 119 +--------------+--------------------------------------------------------------+
 120 |step 4        | The host VM is deleted.                                      |
 121 |              |                                                              |
 122 +--------------+--------------------------------------------------------------+
 123 |test verdict  | Fails only if SLA is not passed, or if there is a test case  |
 124 |              | execution problem.                                           |
 125 |              |                                                              |
 126 +--------------+--------------------------------------------------------------+