Release results overview

This change includes the list of scenarios tested for the release.

Change-Id: Id1e6a07378560e7b88cabde16caca0fcc26544dc
Signed-off-by: Ana C <ana.cunha@ericsson.com>

diff --git a/docs/results/overview.rst b/docs/results/overview.rst
index 30834d7..2515703 100644 (file)
--- a/docs/results/overview.rst
+++ b/docs/results/overview.rst
@@ -21,6 +21,10 @@ This document is part of deliverables of the OPNFV release brahmaputra.1.0
 Scope
 -----
 
+.. _Dashboard: http://130.211.154.108/grafana/dashboard/db/yardstick-main
+.. _Jenkins: https://build.opnfv.org/ci/view/yardstick/
+.. _Scenarios: http://130.211.154.108/grafana/dashboard/db/yardstick-scenarios
+
 This document provides an overview of the results of test cases developed by
 the OPNFV Yardstick Project, executed on OPNFV community labs.
 
@@ -30,8 +34,8 @@ automated via Jenkins Jobs.
 
 Test results are visible in the following dashboard:
 
-* *Yardstick Dashboard*: uses influx DB to store test results and Grafana for
-   visualization
+* *Yardstick* Dashboard_:  uses influx DB to store test results and Grafana for
+   visualization (user: opnfv/ password: opnfv)
 
 
 References
@@ -42,9 +46,35 @@ References
 * OPNFV Brahamputra release note for Yardstick.
 
 
+
 General
 =======
 
+Yardstick Test Cases have been executed for scenarios and features defined in
+this OPNFV release.
+
+The test environments were installed by one of the following: Apex, Compass,
+Fuel or Joid; one single installer per POD.
+
+The results of executed tests are available in Dashboard_ and all logs stored
+in Jenkins_.
+
+After one week of measurments, in general, SDN ONOS showed lower latency than
+SDN ODL, which showed lower latency than an environment installed with pure
+OpenStack. Additional time and PODs make this a non-conclusive statement, see
+Scenarios_ for a snapshot and Dashboard_ for complete results.
+
+It was not possible to execute the entire Yardstick test cases suite on the
+PODs assigned for release verification over a longer period of time, due to
+continuous work on the software components and blocking faults either on
+environment, feature or test framework.
+
+Four consecutive successful runs was defined as criteria for release.
+It is a recommendation to run Yardstick test cases over a longer period
+of time in order to better understand the behavior of the system.
+
+
+
 Document change procedures and history
 --------------------------------------
 
@@ -58,7 +88,7 @@ Document change procedures and history
 | **Release designation**              | Brahmaputra                          |
 |                                      |                                      |
 +--------------------------------------+--------------------------------------+
-| **Release date**                     | 2016-02-25                           |
+| **Release date**                     | Feb 25th, 2016                       |
 |                                      |                                      |
 +--------------------------------------+--------------------------------------+
 | **Purpose of the delivery**          | OPNFV Brahmaputra release test       |

diff --git a/docs/results/results.rst b/docs/results/results.rst
index fa6825e..08acab5 100644 (file)
--- a/docs/results/results.rst
+++ b/docs/results/results.rst
@@ -1,8 +1,7 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International
 .. License.
 .. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd, Intel Corporation and
-.. others.
+
 
 ======================
 Yardstick Test Results
@@ -15,44 +14,85 @@ Yardstick Test Results
 Scenario Results
 ================
 
+.. _Dashboard: http://130.211.154.108/grafana/dashboard/db/yardstick-main
+.. _Jenkins: https://build.opnfv.org/ci/view/yardstick/
+
 The following documents contain results of Yardstick test cases executed on
-OPNFV labs, triggered by OPNFV CI pipeline, documented per verified scenario.
+OPNFV labs, triggered by OPNFV CI pipeline, documented per scenario.
+
+
+Ready scenarios
+---------------
+
+The following scenarios run at least four consecutive times Yardstick test
+cases suite:
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
-   apex-os-odl_l2-sfc-ha.rst
    apex-os-odl_l2-nofeature-ha.rst
-   apex-os-odl_l3-nofeature-ha.rst
-   apex-os-onos-nofeature-ha.rst
    compass-os-nosdn-nofeature-ha.rst
    compass-os-odl_l2-nofeature-ha.rst
    compass-os-onos-nofeature-ha.rst
-   fuel-os-nosdn-kvm-ha.rst
    fuel-os-nosdn-nofeature-ha.rst
-   fuel-os-nosdn-ovs-ha.rst
    fuel-os-odl_l2-nofeature-ha.rst
-   fuel-os-odl_l3-nofeature-ha.rst
    fuel-os-onos-nofeature-ha.rst
-   joid-os-nosdn-nofeature-ha.rst
    joid-os-odl_l2-nofeature-ha.rst
-   joid-os-onos-nofeature-ha.rst
+
+
+Limitations
+-----------
+
+The following scenarios run at least one time Yardstick test cases suite,
+partially or complete:
+
+   * fuel-os-odl_l2-sfc-ha
+
+   * fuel-os-odl_l2-bgpvpn-ha
+
+   * fuel-os-odl_l3-nofeature-ha
+
+   * joid-os-nosdn-nofeature-ha
+
+
+Test results of executed tests are avilable in Dashboard_ and logs in Jenkins_.
+
+
+Not executed
+------------
+
+For the following scenarios, Yardstick test cases were not run:
+
+
+   * apex-os-odl_l2-sfc-noha
+
+   * apex-os-odl_l3-nofeature-ha
+
+   * apex-os-onos-nofeature-ha
+
+   * fuel-os-nosdn-kvm-ha
+
+   * fuel-os-nosdn-ovs-ha
+
+   * fuel-os-odl_l3-nofeature-ha
+
+   * joid-os-onos-nofeature-ha
+
+Logs are available in Jenkins_.
 
 
 Feature Test Results
 ====================
 
-The following documents contain results of Yardstick test cases executed on
-OPNFV labs which were not dedicated for OPNFV CI release verification
-activities.
+The following features were verified by Yardstick test cases:
 
-Results of tests for OPNFV features verified as part a scenario are
-documented in the previous section.
+   * IPv6
 
-.. toctree::
-   :maxdepth: 2
+   * HA (see :doc:`yardstick-opnfv-ha`)
+
+   * KVM
+
+   * Parser
 
-   yardstick-opnfv-ha.rst
-   yardstick-opnfv-kvm.rst
-   yardstick-opnfv-parser.rst
-   yardstick-opnfv-vtc.rst
+.. note:: The test cases for IPv6 and Parser Projects are included in the
+  compass scenario.