Logic to trigger Security Scan project

[functest.git] / docs / release-notes / functest-release.rst

diff --git a/docs/release-notes/functest-release.rst b/docs/release-notes/functest-release.rst
index 2aa77bb..07389bb 100644 (file)
--- a/docs/release-notes/functest-release.rst
+++ b/docs/release-notes/functest-release.rst
@@ -97,9 +97,9 @@ Deliverables
 Software
 --------
 
- - The Functest Docker image: * TODO link *
+ - The Functest Docker image: https://hub.docker.com/r/opnfv/functest (tag: colorado.1.0)
 
- - The testapi Docker image: * TODO link *
+ - The testapi Docker image: https://hub.docker.com/r/opnfv/testapi (tag:colorado.1.0)
 
 
 Documents
@@ -167,13 +167,19 @@ precised, the scenario is a HA scenario):
 +---------------------+---------+---------+---------+---------+
 |   nosdn-noha        |         |         |    X    |    X    |
 +---------------------+---------+---------+---------+---------+
-|   odl_l2            |    X    |    X    |    X    |    X    |
+|   kvm               |         |         |    X    |         |
 +---------------------+---------+---------+---------+---------+
-|   odl_l2-noha       |         |         |    X    |         |
+|   kvm-noha          |         |         |    X    |         |
 +---------------------+---------+---------+---------+---------+
-|   odl_l3            |    X    |    X    |    X    |         |
+|   lxd               |         |         |         |    X    |
 +---------------------+---------+---------+---------+---------+
-|   odl_l3-noha       |         |         |    X    |         |
+|   lxd-noha          |         |         |         |    X    |
++---------------------+---------+---------+---------+---------+
+|   ovs-noha (dpdk)   |         |         |    X    |         |
++---------------------+---------+---------+---------+---------+
+|   odl_l2            |    X    |    X    |    X    |    X    |
++---------------------+---------+---------+---------+---------+
+|   odl_l2-noha       |         |         |    X    |         |
 +---------------------+---------+---------+---------+---------+
 |   odl_l2-bgpvpn     |    X    |         |    X    |         |
 +---------------------+---------+---------+---------+---------+
@@ -183,10 +189,12 @@ precised, the scenario is a HA scenario):
 +---------------------+---------+---------+---------+---------+
 |   odl_l2-moon       |         |    X    |         |         |
 +---------------------+---------+---------+---------+---------+
-|   odl_l2-sfc        |         |         |    X    |         |
-+---------------------+---------+---------+---------+---------+
 |   odl_l2-sfc-noha   |    X    |         |    X    |         |
 +---------------------+---------+---------+---------+---------+
+|   odl_l3            |    X    |    X    |    X    |         |
++---------------------+---------+---------+---------+---------+
+|   odl_l3-noha       |         |         |    X    |         |
++---------------------+---------+---------+---------+---------+
 |   onos              |         |    X    |    X    |    X    |
 +---------------------+---------+---------+---------+---------+
 |   onos-noha         |         |         |    X    |         |
@@ -195,19 +203,20 @@ precised, the scenario is a HA scenario):
 +---------------------+---------+---------+---------+---------+
 |   onos-sfc-noha     |         |         |    X    |         |
 +---------------------+---------+---------+---------+---------+
-|   ovs-noha (dpdk)   |         |         |    X    |         |
-+---------------------+---------+---------+---------+---------+
-|   kvm               |         |         |    X    |         |
-+---------------------+---------+---------+---------+---------+
-|   kvm-noha          |         |         |    X    |         |
-+---------------------+---------+---------+---------+---------+
 |   multisite         |         |         |    X    |         |
 +---------------------+---------+---------+---------+---------+
-|   lxd               |         |         |         |    X    |
+
+For Colorado 2.0, the following scenarios have been re-released:
+
 +---------------------+---------+---------+---------+---------+
-|   lxd-noha          |         |         |         |    X    |
+|    Scenario         |  Apex   | Compass |  Fuel   |   Joid  |
++=====================+=========+=========+=========+=========+
+|   onos              |    X    |    X    |    X    |    X    |
++---------------------+---------+---------+---------+---------+
+|   odl_l2-sfc-ha     |         |         |    X    |         |
 +---------------------+---------+---------+---------+---------+
 
+
 In Colorado, the functional tests have been sliced in 6 different
 categories:
 
@@ -280,8 +289,11 @@ Apex
 +------------------+---------+---------+-------------------+
 | odl_l2-fdio-noha |  12/15  |    80%  | `apex-res-6`_     |
 +------------------+---------+---------+-------------------+
-| odl_l2-sfc-noha  |  18/21  |    86%  | `apex-res-217`_   |
+| odl_l2-sfc-noha  |  18/24  |    75%  | `apex-res-217`_   |
 +------------------+---------+---------+-------------------+
+| onos-nofeature-ha|  20/21  |    95%  | `apex-res-423`_   |
++------------------+---------+---------+-------------------+
+
 
 Compass
 -------
@@ -298,9 +310,9 @@ Compass
 +------------------+---------+---------+-------------------+
 | odl_l2-moon      |  15/18  |    83%  | `compass-res-567`_|
 +------------------+---------+---------+-------------------+
-| onos-ha          |  15/15  |   100%  | `compass-res-77`_ |
+| onos-ha          |  15/15  |   100%  | `compass-res-285`_|
 +------------------+---------+---------+-------------------+
-| onos-sfc-ha      |  16/18  |   100%  | `compass-res-76`_ |
+| onos-sfc-ha      |  17/18  |    95%  | `compass-res-76`_ |
 +------------------+---------+---------+-------------------+
 
 Note: all the Compass tests for Colorado have been executed on virtual
@@ -322,21 +334,27 @@ Fuel
 +---------------------+---------+---------+-------------------+
 | nosdn-kvm-noha      |  15/15  |  100%   | `fuel-res-161`_   |
 +---------------------+---------+---------+-------------------+
+| nosdn-ovs           |  12/18  |   67%*  |  `fuel-res-213`_  |
++---------------------+---------+---------+-------------------+
 | nosdn-ovs-noha      |  15/15  |  100%   | `fuel-res-162`_   |
 +---------------------+---------+---------+-------------------+
 | odl_l2              |  21/21  |  100%   |  `fuel-res-123`_  |
 +---------------------+---------+---------+-------------------+
 | odl_l2-noha         |  17/18  |   94%   | `fuel-res-155`_   |
 +---------------------+---------+---------+-------------------+
-| odl_l2-bgpvpn       |  14/18  |   18%   | `fuel-res-119`_   |
+| odl_l2-bgpvpn       |  14/18  |   78%   | `fuel-res-119`_   |
 +---------------------+---------+---------+-------------------+
 | odl_l2-bgpvpn-noha  |  14/15  |   93%   | `fuel-res-160`_   |
 +---------------------+---------+---------+-------------------+
-| odl_l3              |  15/18  |   67%   | `fuel-res-115`_   |
+| odl_l2-sfc-noha     |   6/21  |   29%   | `fuel-res-219`_   |
++---------------------+---------+---------+-------------------+
+| odl_l2-sfc-ha       |  16/21  |   76%   | `fuel-res-376`_   |
++---------------------+---------+---------+-------------------+
+| odl_l3              |  15/18  |   83%   | `fuel-res-115`_   |
 +---------------------+---------+---------+-------------------+
 | odl_l3-noha         |  12/15  |   80%   | `fuel-res-164`_   |
 +---------------------+---------+---------+-------------------+
-| onos                |  20/21  |   95%   | `fuel-res-117`_   |
+| onos                |  20/21  |   95%   | `fuel-res-492`_   |
 +---------------------+---------+---------+-------------------+
 | onos-noha           |  18/18  |  100%   | `fuel-res-166`_   |
 +---------------------+---------+---------+-------------------+
@@ -347,20 +365,21 @@ Fuel
 | multisite           |  N.R    |  100%   | `fuel-res-8`_     |
 +---------------------+---------+---------+-------------------+
 
+*: all results passed, lacking iterations to reach the full score
 
-Results of Functest on AArch64 Colorado 1.0
+Results of Functest on AArch64 Colorado 3.0
 
 +---------------------+---------+---------+----------------------+
 |  Scenario           | Scoring | Success |  Results             |
 |                     |         | rate    |                      |
 +=====================+=========+=========+======================+
-| nosdn               |  18/18  |  100%   | `fuel-arm-res-29`_   |
+| nosdn               |  18/18  |  100%   | `fuel-arm-res-128`_  |
 +---------------------+---------+---------+----------------------+
-| odl_l2              |  21/21  |  100%   | `fuel-arm-res-24`_   |
+| odl_l2              |  21/21  |  100%   | `fuel-arm-res-122`_  |
 +---------------------+---------+---------+----------------------+
-| odl_l2-noha         |  17/18  |   94%   | `fuel-arm-res-30`_   |
+| odl_l2-noha         |  18/18  |  100%   | `fuel-arm-res-129`_  |
 +---------------------+---------+---------+----------------------+
-| odl_l3              |  14/18  |   78%   | `fuel-arm-res-28`_   |
+| odl_l3              |  17/18  |   95%   | `fuel-arm-res-135`_  |
 +---------------------+---------+---------+----------------------+
 
 
@@ -381,7 +400,7 @@ Joid
 +---------------------+---------+---------+-----------------+
 | odl_l2              |  21/21  |  100%   | `joid-res-103`_ |
 +---------------------+---------+---------+-----------------+
-| onos                |  21/21  |  100%   | `joid-res-99`_  |
+| onos                |  21/21  |  100%   | `joid-res-345`_ |
 +---------------------+---------+---------+-----------------+
 | onos-sfc            |  24/24  |  100%   | `joid-res-97`_  |
 +---------------------+---------+---------+-----------------+
@@ -538,6 +557,8 @@ Useful links
 
 .. _`apex-res-235`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-colorado-daily-colorado-235
 
+.. _`apex-res-423`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-colorado-daily-colorado-423
+
 .. _`compass-res-55`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-colorado-55
 
 .. _`compass-res-59`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-colorado-59
@@ -550,49 +571,55 @@ Useful links
 
 .. _`compass-res-567`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-master-567
 
-.. _`fuel-res-129`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-129
-
-.. _`fuel-res-154`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-154
+.. _`compass-res-285`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-master-285
 
-.. _`fuel-res-128`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-128
+.. _`fuel-res-8`: https://build.opnfv.org/ci/view/functest/job/functest-fuel-virtual-suite-colorado/lastSuccessfulBuild/console
 
-.. _`fuel-res-161`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-161
+.. _`fuel-res-115`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-115
 
-.. _`fuel-res-162`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-162
+.. _`fuel-res-117`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-117
 
 .. _`fuel-res-119`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-119
 
-.. _`fuel-res-160`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-160
+.. _`fuel-res-123`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-123
 
-.. _`fuel-res-115`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-115
+.. _`fuel-res-124`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-124
 
-.. _`fuel-res-117`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-117
+.. _`fuel-res-128`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-128
 
-.. _`fuel-res-124`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-124
+.. _`fuel-res-129`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-129
 
-.. _`fuel-res-8`: https://build.opnfv.org/ci/view/functest/job/functest-fuel-virtual-suite-colorado/lastSuccessfulBuild/console
+.. _`fuel-res-154`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-154
+
+.. _`fuel-res-155`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-155
+
+.. _`fuel-res-160`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-160
+
+.. _`fuel-res-161`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-161
+
+.. _`fuel-res-162`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-162
 
 .. _`fuel-res-164`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-164
 
 .. _`fuel-res-166`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-166
 
-.. _`fuel-res-155`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-155
+.. _`fuel-res-213`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-213
 
-.. _`fuel-res-123`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-colorado-123
+.. _`fuel-res-219`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-219
 
-.. _`fuel-arm-res-29`: https://build.opnfv.org/ci/view/armband/job/functest-fuel-armband-baremetal-daily-colorado/29/console
+.. _`fuel-res-376`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-376
 
-.. _`fuel-arm-res-24`: https://build.opnfv.org/ci/view/armband/job/functest-fuel-armband-baremetal-daily-colorado/24/console
+.. _`fuel-res-492`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-colorado-492
 
-.. _`fuel-arm-res-30`: https://build.opnfv.org/ci/view/armband/job/functest-fuel-armband-baremetal-daily-colorado/30/console
+.. _`fuel-arm-res-128`: https://build.opnfv.org/ci/view/armband/job/functest-fuel-armband-baremetal-daily-colorado/128/console
 
-.. _`fuel-arm-res-28`: https://build.opnfv.org/ci/view/armband/job/functest-fuel-armband-baremetal-daily-colorado/28/console
+.. _`fuel-arm-res-122`: https://build.opnfv.org/ci/view/armband/job/functest-fuel-armband-baremetal-daily-colorado/122/console
 
-.. _`joid-res-102`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-colorado-102
+.. _`fuel-arm-res-129`: https://build.opnfv.org/ci/view/armband/job/functest-fuel-armband-baremetal-daily-colorado/129/console
 
-.. _`joid-res-93`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-colorado-93
+.. _`fuel-arm-res-135`: https://build.opnfv.org/ci/view/armband/job/functest-fuel-armband-baremetal-daily-colorado/135/console
 
-.. _`joid-res-104`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-colorado-104
+.. _`joid-res-93`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-colorado-93
 
 .. _`joid-res-91`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-colorado-91
 
@@ -600,4 +627,11 @@ Useful links
 
 .. _`joid-res-97`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-colorado-97
 
+.. _`joid-res-102`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-colorado-102
+
 .. _`joid-res-103`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-colorado-103
+
+.. _`joid-res-104`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-colorado-104
+
+.. _`joid-res-345`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-colorado-345
+