docs/release/release-notes/functest-release.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
   2 .. SPDX-License-Identifier: CC-BY-4.0
   3
   4 =======
   5 License
   6 =======
   7
   8 OPNFV Danube release note for Functest Docs
   9 (c) by Jose Lausuch (Ericsson)
  10
  11 OPNFV Danube release note for Functest Docs
  12 are licensed under a Creative Commons Attribution 4.0 International License.
  13 You should have received a copy of the license along with this.
  14 If not, see <http://creativecommons.org/licenses/by/4.0/>.
  15
  16 ===========================================
  17 OPNFV Danube2.0 release note for Functest
  18 ===========================================
  19
  20 Abstract
  21 ========
  22
  23 This document describes the release note of Functest project.
  24
  25
  26 Version history
  27 ===============
  28
  29 +------------+----------+------------------+------------------------+
  30 | **Date**   | **Ver.** | **Author**       | **Comment**            |
  31 |            |          |                  |                        |
  32 +------------+----------+------------------+------------------------+
  33 | 2016-08-17 | 1.0.0    | Morgan Richomme  | Functest for           |
  34 |            |          | (Orange)         | Colorado.1.0 release   |
  35 +------------+----------+------------------+------------------------+
  36 | 2016-10-24 | 2.0.0    | Morgan Richomme  | Functest for           |
  37 |            |          | (Orange)         | Colorado.2.0 release   |
  38 +------------+----------+------------------+------------------------+
  39 | 2016-08-17 | 3.0.0    | Morgan Richomme  | Functest for           |
  40 |            |          | (Orange)         | Colorado.3.0 release   |
  41 +------------+----------+------------------+------------------------+
  42 | 2017-03-29 | 4.0.0    | Jose Lausuch     | Functest for           |
  43 |            |          | (Ericsson)       | Danube.1.0 release     |
  44 +------------+----------+------------------+------------------------+
  45 | 2017-05-04 | 5.0.0    | Jose Lausuch     | Functest for           |
  46 |            |          | (Ericsson)       | Danube.2.0 release     |
  47 +------------+----------+------------------+------------------------+
  48 | 2017-07-12 | 6.0.0    | Jose Lausuch     | Functest for           |
  49 |            |          | (Ericsson)       | Danube.3.0 release     |
  50 +------------+----------+------------------+------------------------+
  51
  52 OPNFV Danube Release
  53 ======================
  54
  55 Functest deals with functional testing of the OPNFV solution.
  56 It includes test cases developed within the project and test cases developed in
  57 other OPNFV projects and other upstream communities.
  58
  59 The internal test cases are:
  60
  61  * connection_check
  62  * api_check
  63  * snaps_health_check
  64  * vping_ssh
  65  * vping_userdata
  66  * tempest_smoke_serial
  67  * refstack_defcore
  68  * snaps_smoke
  69  * rally_sanity
  70  * odl
  71  * tempest_full_parallel
  72  * rally_full
  73  * cloudify_ims
  74
  75 The OPNFV projects integrated into Functest framework for automation are:
  76
  77  * barometer
  78  * bgpvpn
  79  * doctor
  80  * domino
  81  * fds
  82  * multisite
  83  * netready
  84  * onos
  85  * odl-sfc
  86  * odl-netvirt
  87  * parser
  88  * promise
  89  * security scan
  90  * orchestra_ims
  91  * vyos_vrouter
  92
  93 The validation of a scenario requires a subset of these tests depending
  94 on the installer and the scenario.
  95
  96 The test cases from vnf (cloudify_ims, orchestra_ims, vyos_vrouter) and
  97 component categories (tempest full parallel, Rally full) are not considered for
  98 scenario validation.
  99
 100 Release Data
 101 ============
 102
 103 +--------------------------------------+--------------------------------------+
 104 | **Project**                          | functest                             |
 105 |                                      |                                      |
 106 +--------------------------------------+--------------------------------------+
 107 | **Repo/tag**                         | danube.3.0                           |
 108 |                                      |                                      |
 109 +--------------------------------------+--------------------------------------+
 110 | **Release designation**              | Danube service release               |
 111 |                                      |                                      |
 112 +--------------------------------------+--------------------------------------+
 113 | **Release date**                     | July 14th 2017                       |
 114 |                                      |                                      |
 115 +--------------------------------------+--------------------------------------+
 116 | **Purpose of the delivery**          | Danube third release                 |
 117 |                                      |                                      |
 118 +--------------------------------------+--------------------------------------+
 119
 120 Deliverables
 121 ============
 122
 123 Software
 124 --------
 125
 126  - The Functest Docker image: https://hub.docker.com/r/opnfv/functest (tag: danube.3.0)
 127
 128  - The TestAPI Docker image: https://hub.docker.com/r/opnfv/testapi (tag:danube.3.0)
 129
 130
 131 Documents
 132 ---------
 133
 134  - Installation/configuration guide: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/configguide/index.html
 135
 136  - User Guide: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
 137
 138  - Developer Guide: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/developer/devguide/index.html
 139
 140
 141 Version change
 142 ==============
 143
 144
 145 - Bugfixes in refstack client
 146
 147 - Fixed vIMS onboarding and ims deployment
 148
 149 - Increased timeout of OpenBaton installation
 150
 151 - Switched to mitaka-eol in openstack repositories, as stable/mitake is not longer existing
 152
 153 - Minor fixes in logging and test cases
 154
 155 - 3 scenarios have been successfully executed on aarch64 architectures on Fuel deployments (nosdn, odl_l2 and odl_l3)
 156
 157
 158 Scenario Matrix
 159 ===============
 160
 161 For Danube 3.0, Functest was tested on the following HA scenarios (new
 162 dabube scenarios in bold):
 163
 164 +---------------------+---------+---------+---------+---------+
 165 |    Scenario         |  Apex   | Compass |  Fuel   |   Joid  |
 166 +=====================+=========+=========+=========+=========+
 167 |   nosdn             |    X    |    X    |    X    |    X    |
 168 +---------------------+---------+---------+---------+---------+
 169 | **fdio**            |    X    |         |         |         |
 170 +---------------------+---------+---------+---------+---------+
 171 |   kvm               |    X    |         |    X    |         |
 172 +---------------------+---------+---------+---------+---------+
 173 | **kvm_ovs_dpdk**    |         |         |    X    |         |
 174 +---------------------+---------+---------+---------+---------+
 175 | **kvm_ovs_dpdk-bar**|         |         |    X    |         |
 176 +---------------------+---------+---------+---------+---------+
 177 |   lxd               |         |         |         |    X    |
 178 +---------------------+---------+---------+---------+---------+
 179 | **ovs**             |         |         |    X    |         |
 180 +---------------------+---------+---------+---------+---------+
 181 | **openo**           |         |    X    |         |         |
 182 +---------------------+---------+---------+---------+---------+
 183 |   odl_l2            |         |    X    |   X     |    X    |
 184 +---------------------+---------+---------+---------+---------+
 185 |   odl-bgpvpn        |   X     |         |         |         |
 186 +---------------------+---------+---------+---------+---------+
 187 |   odl_l2-bgpvpn     |         |         |   X     |         |
 188 +---------------------+---------+---------+---------+---------+
 189 | **odl_l2-fdio**     |    X    |         |         |         |
 190 +---------------------+---------+---------+---------+---------+
 191 |   odl_l2-sfc        |         |         |    X    |         |
 192 +---------------------+---------+---------+---------+---------+
 193 |   odl_l3            |    X    |    X    |    X    |         |
 194 +---------------------+---------+---------+---------+---------+
 195 | **ocl**             |         |   X     |         |         |
 196 +---------------------+---------+---------+---------+---------+
 197 |   onos              |         |   X     |         |         |
 198 +---------------------+---------+---------+---------+---------+
 199 |   multisite         |         |         |    X    |         |
 200 +---------------------+---------+---------+---------+---------+
 201
 202 Non HA scenarios:
 203
 204 +---------------------+---------+---------+---------+---------+
 205 |    Scenario         |  Apex   | Compass |  Fuel   |   Joid  |
 206 +=====================+=========+=========+=========+=========+
 207 |   nosdn             |         |         |    X    |    X    |
 208 +---------------------+---------+---------+---------+---------+
 209 |   kvm               |         |         |    X    |         |
 210 +---------------------+---------+---------+---------+---------+
 211 | **kvm_ovs_dpdk**    |         |         |    X    |         |
 212 +---------------------+---------+---------+---------+---------+
 213 | **kvm_ovs_dpdk-bar**|         |         |    X    |         |
 214 +---------------------+---------+---------+---------+---------+
 215 |   lxd               |         |         |         |    X    |
 216 +---------------------+---------+---------+---------+---------+
 217 |   ovs               |         |         |    X    |         |
 218 +---------------------+---------+---------+---------+---------+
 219 |   odl_l2            |         |         |   X     |         |
 220 +---------------------+---------+---------+---------+---------+
 221 |   odl_l2-bgpvpn     |         |         |   X     |         |
 222 +---------------------+---------+---------+---------+---------+
 223 |   odl_l2-fdio       |    X    |         |         |         |
 224 +---------------------+---------+---------+---------+---------+
 225 | **odl_l3-fdio**     |    X    |         |         |         |
 226 +---------------------+---------+---------+---------+---------+
 227 |   odl_l2-sfc        |         |         |    X    |         |
 228 +---------------------+---------+---------+---------+---------+
 229 |   odl_l3            |         |         |    X    |         |
 230 +---------------------+---------+---------+---------+---------+
 231 | **odl_gluon**       |    X    |         |         |         |
 232 +---------------------+---------+---------+---------+---------+
 233
 234 Colorado deprecated scenarios:
 235
 236  * odl_l2-moon
 237  * onos-sfc
 238  * onos-noha
 239  * onos-sfc-noha
 240
 241 For the scenario validation, we consider only the healthcheck, smoke and
 242 features categories. These tests are run systematically in the CI daily loops.
 243
 244 Success criterias have been defined for these test cases, they can be
 245 PASS/FAIL or a success rate may be declared (100%, > 90%).
 246 All the criteria, as well as the test dependencies are declared in the
 247 testcases.yaml file located in the ci directory of the repository.
 248
 249 The scoring for the Danube release per installer can be described as
 250 follows.
 251
 252 The scoring is an indicator showing how many feature project test suites
 253 have been integrated in the scenario.
 254
 255 The scoring equals the number of tests * successful iteration of each
 256 test [0-3]. The scoring method is described in https://wiki.opnfv.org/pages/viewpage.action?pageId=6828617
 257
 258  e.g.
 259  apex/odl_l2-nofeature-ha
 260  tests = vping_ssh+vping_userdata+tempest+rally+odl+doctor+copper
 261  Scoring = 21/21 = 7 * 3
 262
 263 By default, if not specified, the scenarios are HA.
 264 HA means OpenStack High Availability (main services). Note that not
 265 all VIM (e.g. OpenStack) services are deployed in HA mode, as that
 266 depends upon support of the specific service for HA deployment.
 267 For example, in the Danube release, the Congress service
 268 is deployed in non-HA mode even for HA OPNFV scenarios, as explicit
 269 support for HA operation has not yet been verified.
 270 See the release notes of the installers for more details.
 271
 272
 273 Apex
 274 ----
 275
 276 +------------------+---------+---------+-----------------+
 277 |  Scenario        | Scoring | Success |    Results      |
 278 |                  |         | rate    |                 |
 279 +==================+=========+=========+=================+
 280 | nosdn            |  20/30  |    67%  | `apex-res-1`_   |
 281 +------------------+---------+---------+-----------------+
 282 | odl_l3           |  16/30  |    53%  | `apex-res-2`_   |
 283 +------------------+---------+---------+-----------------+
 284 | odl-bgpvpn       |  18/33  |    56%  | `apex-res-3`_   |
 285 +------------------+---------+---------+-----------------+
 286 | odl-gluon        |  20/33  |    61%  | `apex-res-4`_   |
 287 +------------------+---------+---------+-----------------+
 288 | kvm              |  20/30  |    67%  | `apex-res-5`_   |
 289 +------------------+---------+---------+-----------------+
 290 | odl_l2-fdio      |  25/33  |    76%  | `apex-res-6`_   |
 291 +------------------+---------+---------+-----------------+
 292 | odl_l2-fdio-noha |  24/33  |    73%  |   local push    |
 293 +------------------+---------+---------+-----------------+
 294 | odl_l3-fdio-noha |  18/30  |    60%  |   local push    |
 295 +------------------+---------+---------+-----------------+
 296 | fdio             |  25/27  |    93%  |   local push    |
 297 +------------------+---------+---------+-----------------+
 298
 299 Compass
 300 -------
 301
 302 +------------------+---------+---------+------------------+
 303 |  Scenario        | Scoring | Success |  Results         |
 304 |                  |         | rate    |                  |
 305 +==================+=========+=========+==================+
 306 | nosdn            |  29/30  |    97%  | `compass-res-1`_ |
 307 +------------------+---------+---------+------------------+
 308 | odl_l2           |  29/33  |    88%  | `compass-res-2`_ |
 309 +------------------+---------+---------+------------------+
 310 | odl_l3           |  23/30  |    77%  | `compass-res-3`_ |
 311 +------------------+---------+---------+------------------+
 312 | onos             |  29/30  |    97%  | `compass-res-4`_ |
 313 +------------------+---------+---------+------------------+
 314 | onos-sfc         |  20/36  |    56%  | `compass-res-5`_ |
 315 +------------------+---------+---------+------------------+
 316
 317 Note: all the Compass tests for Danube have been executed on virtual
 318 environment. Bare metal resources were used for Master branch.
 319
 320
 321 Fuel
 322 ----
 323
 324 +----------------------+---------+---------+----------------+
 325 |  Scenario            | Scoring | Success |  Results       |
 326 |                      |         | rate    |                |
 327 +======================+=========+=========+================+
 328 | nosdn                |  39/39  |  100%   | `fuel-res-1`_  |
 329 +----------------------+---------+---------+----------------+
 330 | nosdn-noha           |  36/36  |  100%   | `fuel-res-2`_  |
 331 +----------------------+---------+---------+----------------+
 332 | nosdn-kvm            |  39/39  |  100%   | `fuel-res-3`_  |
 333 +----------------------+---------+---------+----------------+
 334 | nosdn-kvm-noha       |  36/36  |  100%   | `fuel-res-4`_  |
 335 +----------------------+---------+---------+----------------+
 336 | nosdn-ovs            |  39/39  |  100%   | `fuel-res-5`_  |
 337 +----------------------+---------+---------+----------------+
 338 | nosdn-ovs-noha       |  36/36  |  100%   | `fuel-res-6`_  |
 339 +----------------------+---------+---------+----------------+
 340 | odl_l2               |  42/42  |  100%   | `fuel-res-7`_  |
 341 +----------------------+---------+---------+----------------+
 342 | odl_l2-noha          |  39/39  |  100%   | `fuel-res-8`_  |
 343 +----------------------+---------+---------+----------------+
 344 | odl_l2-sfc           |  45/45  |  100%   | `fuel-res-9`_  |
 345 +----------------------+---------+---------+----------------+
 346 | odl_l2-sfc-noha      |  35/42  |   83%   | `fuel-res-10`_ |
 347 +----------------------+---------+---------+----------------+
 348 | odl_l3               |  36/39  |   92%   | `fuel-res-11`_ |
 349 +----------------------+---------+---------+----------------+
 350 | odl_l3-noha          |  36/36  |  100%   | `fuel-res-12`_ |
 351 +----------------------+---------+---------+----------------+
 352 | kvm_ovs_dpdk         |  39/39  |  100%   | `fuel-res-13`_ |
 353 +----------------------+---------+---------+----------------+
 354 | kvm_ovs_dpdk_noha    |  35/36  |   97%   | `fuel-res-14`_ |
 355 +----------------------+---------+---------+----------------+
 356 | kvm_ovs_dpdk_bar     |  42/42  |  100%   | `fuel-res-15`_ |
 357 +----------------------+---------+---------+----------------+
 358 | kvm_ovs_dpdk_bar_noha|  36/39  |   92%   | `fuel-res-16`_ |
 359 +----------------------+---------+---------+----------------+
 360
 361
 362
 363
 364 Joid
 365 ----
 366
 367 +---------------------+---------+---------+---------------+
 368 |  Scenario           | Scoring | Success |  Results      |
 369 |                     |         | rate    |               |
 370 +=====================+=========+=========+===============+
 371 | nosdn               |  30/30  |  100%   | `joid-res-1`_ |
 372 +---------------------+---------+---------+---------------+
 373 | nosdn-noha          |  10/30  |   33%   | `joid-res-2`_ |
 374 +---------------------+---------+---------+---------------+
 375 | nosdn-lxd           |  19/21  |   90%   | `joid-res-3`_ |
 376 +---------------------+---------+---------+---------------+
 377 | nosdn-lxd-noha      |  15/21  |   71%   | `joid-res-4`_ |
 378 +---------------------+---------+---------+---------------+
 379 | odl_l2              |   6/33  |   18%   | `joid-res-5`_ |
 380 +---------------------+---------+---------+---------------+
 381
 382 It is highly recommended to install a json viewer in your browser
 383 (e.g. https://addons.mozilla.org/fr/firefox/addon/jsonview/)
 384
 385 You can get additional details through test logs on http://artifacts.opnfv.org/.
 386 As no search engine is available on the OPNFV artifact web site you must
 387 retrieve the pod identifier on which the tests have been executed (see
 388 field pod in any of the results) then click on the selected POD and look
 389 for the date of the test you are interested in.
 390
 391 The reporting pages can be found at:
 392
 393  * apex: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-apex.html
 394  * compass: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-compass.html
 395  * fuel: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-fuel.html
 396  * joid: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-joid.html
 397
 398 Danube known restrictions/issues
 399 ==================================
 400
 401 ************* TODO *****************************************
 402
 403 +-----------+-----------+----------------------------------------------+
 404 | Installer | Scenario  |  Issue                                       |
 405 +===========+===========+==============================================+
 406 | fuel      | odl_-*    | Tempest test case "TestServerBasicOps"       |
 407 |           |           | disabled due to bug `tempest-bug`_           |
 408 +-----------+-----------+----------------------------------------------+
 409 | apex/fuel | *-bgpvpn  | Due to some instabilities in the bgpvpn      |
 410 |           |           | test case, the scenario has been postponed   |
 411 |           |           | to Danube 2.0                                |
 412 +-----------+-----------+----------------------------------------------+
 413 | apex      | *-gluon   | vPing_ssh disabled due to floating ips       |
 414 |           |           | not working 100% of the times.               |
 415 |           |           | Tempest test "test_reboot_server_hard"       |
 416 |           |           | disabled due to bug `gluon-bug`_             |
 417 +-----------+-----------+----------------------------------------------+
 418 | joid      | any       | Tempest cases related to object storage      |
 419 |           |           | excluded                                     |
 420 +-----------+-----------+----------------------------------------------+
 421 | any       | any       | The VNF tier has not been fully tested       |
 422 |           |           | since it has not been run in daily loops     |
 423 |           |           | in CI. Weekly jobs have been activated       |
 424 |           |           | a bit late in the process and have not been  |
 425 |           |           | used to validate the scenarios.              |
 426 +-----------+-----------+----------------------------------------------+
 427
 428 Test and installer/scenario dependencies
 429 ========================================
 430
 431 It is not always possible to run all the test cases on all the scenarios.
 432 The following table details the dependencies of the test cases per
 433 scenario. The scenario dependencies (installer or scenario) are detailed
 434 in https://git.opnfv.org/cgit/functest/tree/ci/testcases.yaml
 435
 436 Test results
 437 ============
 438
 439 Test results are available in:
 440
 441  - test results document: http://artifacts.opnfv.org/functest
 442
 443  - jenkins logs on CI: https://build.opnfv.org/ci/view/functest/
 444
 445  - jenkins logs on ARM CI: https://build.opnfv.org/ci/view/armband/
 446
 447
 448
 449 Open JIRA tickets
 450 =================
 451
 452 +------------------+-----------------------------------------------+
 453 |   JIRA           |         Description                           |
 454 +==================+===============================================+
 455 |                  |                                               |
 456 |                  |                                               |
 457 +------------------+-----------------------------------------------+
 458
 459 All the tickets that are not blocking have been fixed or postponed
 460 the next release.
 461
 462 Functest Danube 2.0 is released without known bugs.
 463
 464
 465
 466 Useful links
 467 ============
 468
 469  - wiki project page: https://wiki.opnfv.org/opnfv_functional_testing
 470
 471  - wiki Functest Danube page: https://wiki.opnfv.org/display/functest/Functest+Danube
 472
 473  - Functest repo: https://git.opnfv.org/cgit/functest
 474
 475  - Functest CI dashboard: https://build.opnfv.org/ci/view/functest/
 476
 477  - JIRA dashboard: https://jira.opnfv.org/secure/Dashboard.jspa?selectPageId=10611
 478
 479  - Functest IRC chan: #opnfv-functest
 480
 481  - Reporting page: http://testresults.opnfv.org/reporting/danube.html
 482
 483  - Functest test configuration: https://git.opnfv.org/cgit/functest/tree/functest/ci/testcases.yaml
 484
 485 .. _`tempest-bug`: https://bugs.launchpad.net/tempest/+bug/1577632
 486
 487 .. _`gluon-bug`: https://bugs.opendaylight.org/show_bug.cgi?id=5586
 488
 489
 490 .. _`apex-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-455
 491
 492 .. _`apex-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-466
 493
 494 .. _`apex-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-449
 495
 496 .. _`apex-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-450
 497
 498 .. _`apex-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-461
 499
 500 .. _`apex-res-6`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-485
 501
 502
 503 .. _`compass-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-danube-484
 504
 505 .. _`compass-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-danube-454
 506
 507 .. _`compass-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-baremetal-daily-danube-482
 508
 509 .. _`compass-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-danube-481
 510
 511 .. _`compass-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-baremetal-daily-danube-453
 512
 513
 514 .. _`fuel-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-54
 515
 516 .. _`fuel-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-937
 517
 518 .. _`fuel-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-873
 519
 520 .. _`fuel-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-935
 521
 522 .. _`fuel-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-875
 523
 524 .. _`fuel-res-6`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-936
 525
 526 .. _`fuel-res-7`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-867
 527
 528 .. _`fuel-res-8`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-939
 529
 530 .. _`fuel-res-9`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-870
 531
 532 .. _`fuel-res-10`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-933
 533
 534 .. _`fuel-res-11`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-868
 535
 536 .. _`fuel-res-12`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-940
 537
 538 .. _`fuel-res-13`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-871
 539
 540 .. _`fuel-res-14`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-938
 541
 542 .. _`fuel-res-15`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-869
 543
 544 .. _`fuel-res-16`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-941
 545
 546
 547 .. _`joid-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-298
 548
 549 .. _`joid-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-260
 550
 551 .. _`joid-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-281
 552
 553 .. _`joid-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-301
 554
 555 .. _`joid-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-291