docs/testing/user/userguide/test_results.rst

   1 .. SPDX-License-Identifier: CC-BY-4.0
   2
   3 Test results
   4 ============
   5
   6 Manual testing
   7 --------------
   8
   9 In manual mode test results are displayed in the console and result files
  10 are put in /home/opnfv/functest/results.
  11
  12 If you want additional logs, you may configure the logging.ini under
  13 /usr/lib/python2.7/site-packages/functest/ci.
  14
  15 Automated testing
  16 -----------------
  17
  18 In automated mode, test results are displayed in jenkins logs, a summary is
  19 provided at the end of the job and can be described as follow::
  20
  21  +-------------------------+----------------------------------------------------------+
  22  |         ENV VAR         |                          VALUE                           |
  23  +-------------------------+----------------------------------------------------------+
  24  |      INSTALLER_TYPE     |                          daisy                           |
  25  |     DEPLOY_SCENARIO     |                  os-nosdn-nofeature-ha                   |
  26  |        BUILD_TAG        |     jenkins-functest-daisy-baremetal-daily-master-67     |
  27  |         CI_LOOP         |                          daily                           |
  28  +-------------------------+----------------------------------------------------------+
  29
  30 ::
  31
  32  +------------------------------+------------------+---------------------+------------------+----------------+
  33  |          TEST CASE           |     PROJECT      |         TIER        |     DURATION     |     RESULT     |
  34  +------------------------------+------------------+---------------------+------------------+----------------+
  35  |       connection_check       |     functest     |     healthcheck     |      00:08       |      PASS      |
  36  |          api_check           |     functest     |     healthcheck     |      04:22       |      PASS      |
  37  |      snaps_health_check      |     functest     |     healthcheck     |      00:35       |      PASS      |
  38  |          vping_ssh           |     functest     |        smoke        |      00:54       |      PASS      |
  39  |        vping_userdata        |     functest     |        smoke        |      00:27       |      PASS      |
  40  |     tempest_smoke_serial     |     functest     |        smoke        |      19:39       |      FAIL      |
  41  |         rally_sanity         |     functest     |        smoke        |      15:16       |      PASS      |
  42  |       refstack_defcore       |     functest     |        smoke        |      15:55       |      PASS      |
  43  |         snaps_smoke          |     functest     |        smoke        |      26:45       |      FAIL      |
  44  |           cloudify_ims       |     functest     |         vnf         |      23:56       |      PASS      |
  45  |        orchestra_openims     |     orchestra    |         vnf         |      15:07       |      PASS      |
  46  |     orchestra_clearwaterims  |     orchestra    |         vnf         |      19:10       |      PASS      |
  47  |           vyos_vrouter       |     functest     |         vnf         |      00:00       |      SKIP      |
  48  +------------------------------+------------------+---------------------+------------------+----------------+
  49
  50 Results are automatically pushed to the test results database, some additional
  51 result files are pushed to OPNFV artifact web sites.
  52
  53 Based on the results stored in the result database, a `Functest reporting`_
  54 portal is also automatically updated. This portal provides information on the
  55 overall status per scenario and per installer
  56
  57 .. _`Functest reporting`: http://testresults.opnfv.org/reporting/master/functest/status-apex.html