docs/release/release-notes/functest-release.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
   2 .. SPDX-License-Identifier: CC-BY-4.0
   3
   4 =======
   5 License
   6 =======
   7
   8 OPNFV Danube release note for Functest Docs
   9 (c) by Jose Lausuch (Ericsson)
  10
  11 OPNFV Danube release note for Functest Docs
  12 are licensed under a Creative Commons Attribution 4.0 International License.
  13 You should have received a copy of the license along with this.
  14 If not, see <http://creativecommons.org/licenses/by/4.0/>.
  15
  16 ===========================================
  17 OPNFV Danube2.0 release note for Functest
  18 ===========================================
  19
  20 Abstract
  21 ========
  22
  23 This document describes the release note of Functest project.
  24
  25
  26 Version history
  27 ===============
  28
  29 +------------+----------+------------------+------------------------+
  30 | **Date**   | **Ver.** | **Author**       | **Comment**            |
  31 |            |          |                  |                        |
  32 +------------+----------+------------------+------------------------+
  33 | 2016-08-17 | 1.0.0    | Morgan Richomme  | Functest for           |
  34 |            |          | (Orange)         | Colorado.1.0 release   |
  35 +------------+----------+------------------+------------------------+
  36 | 2016-10-24 | 2.0.0    | Morgan Richomme  | Functest for           |
  37 |            |          | (Orange)         | Colorado.2.0 release   |
  38 +------------+----------+------------------+------------------------+
  39 | 2016-08-17 | 3.0.0    | Morgan Richomme  | Functest for           |
  40 |            |          | (Orange)         | Colorado.3.0 release   |
  41 +------------+----------+------------------+------------------------+
  42 | 2017-03-29 | 4.0.0    | Jose Lausuch     | Functest for           |
  43 |            |          | (Ericsson)       | Danube.1.0 release     |
  44 +------------+----------+------------------+------------------------+
  45 | 2017-05-04 | 5.0.0    | Jose Lausuch     | Functest for           |
  46 |            |          | (Ericsson)       | Danube.2.0 release     |
  47 +------------+----------+------------------+------------------------+
  48 | 2017-07-12 | 6.0.0    | Jose Lausuch     | Functest for           |
  49 |            |          | (Ericsson)       | Danube.3.0 release     |
  50 +------------+----------+------------------+------------------------+
  51
  52 OPNFV Danube Release
  53 ======================
  54
  55 Functest deals with functional testing of the OPNFV solution.
  56 It includes test cases developed within the project and test cases developed in
  57 other OPNFV projects and other upstream communities.
  58
  59 The internal test cases are:
  60
  61  * connection_check
  62  * api_check
  63  * snaps_health_check
  64  * vping_ssh
  65  * vping_userdata
  66  * tempest_smoke_serial
  67  * refstack_defcore
  68  * snaps_smoke
  69  * rally_sanity
  70  * odl
  71  * tempest_full_parallel
  72  * rally_full
  73  * cloudify_ims
  74
  75 The OPNFV projects integrated into Functest framework for automation are:
  76
  77  * barometer
  78  * bgpvpn
  79  * doctor
  80  * domino
  81  * fds
  82  * multisite
  83  * onos
  84  * odl-sfc
  85  * odl-netvirt
  86  * parser
  87  * promise
  88  * security scan
  89  * orchestra_ims
  90  * vyos_vrouter
  91
  92 The validation of a scenario requires a subset of these tests depending
  93 on the installer and the scenario.
  94
  95 The test cases from vnf (cloudify_ims, orchestra_ims, vyos_vrouter) and
  96 component categories (tempest full parallel, Rally full) are not considered for
  97 scenario validation.
  98
  99 Release Data
 100 ============
 101
 102 +--------------------------------------+--------------------------------------+
 103 | **Project**                          | functest                             |
 104 |                                      |                                      |
 105 +--------------------------------------+--------------------------------------+
 106 | **Repo/tag**                         | danube.3.0                           |
 107 |                                      |                                      |
 108 +--------------------------------------+--------------------------------------+
 109 | **Release designation**              | Danube service release               |
 110 |                                      |                                      |
 111 +--------------------------------------+--------------------------------------+
 112 | **Release date**                     | July 14th 2017                       |
 113 |                                      |                                      |
 114 +--------------------------------------+--------------------------------------+
 115 | **Purpose of the delivery**          | Danube third release                 |
 116 |                                      |                                      |
 117 +--------------------------------------+--------------------------------------+
 118
 119 Deliverables
 120 ============
 121
 122 Software
 123 --------
 124
 125  - The Functest Docker image: https://hub.docker.com/r/opnfv/functest (tag: danube.3.0)
 126
 127  - The TestAPI Docker image: https://hub.docker.com/r/opnfv/testapi (tag:danube.3.0)
 128
 129
 130 Documents
 131 ---------
 132
 133  - Installation/configuration guide: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/configguide/index.html
 134
 135  - User Guide: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
 136
 137  - Developer Guide: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/developer/devguide/index.html
 138
 139
 140 Version change
 141 ==============
 142
 143
 144 - Bugfixes in refstack client
 145
 146 - Fixed vIMS onboarding and ims deployment
 147
 148 - Increased timeout of OpenBaton installation
 149
 150 - Switched to mitaka-eol in openstack repositories, as stable/mitake is not longer existing
 151
 152 - Minor fixes in logging and test cases
 153
 154 - 3 scenarios have been successfully executed on aarch64 architectures on Fuel deployments (nosdn, odl_l2 and odl_l3)
 155
 156
 157 Scenario Matrix
 158 ===============
 159
 160 For Danube 3.0, Functest was tested on the following HA scenarios (new
 161 dabube scenarios in bold):
 162
 163 +---------------------+---------+---------+---------+---------+
 164 |    Scenario         |  Apex   | Compass |  Fuel   |   Joid  |
 165 +=====================+=========+=========+=========+=========+
 166 |   nosdn             |    X    |    X    |    X    |    X    |
 167 +---------------------+---------+---------+---------+---------+
 168 | **fdio**            |    X    |         |         |         |
 169 +---------------------+---------+---------+---------+---------+
 170 |   kvm               |    X    |         |    X    |         |
 171 +---------------------+---------+---------+---------+---------+
 172 | **kvm_ovs_dpdk**    |         |         |    X    |         |
 173 +---------------------+---------+---------+---------+---------+
 174 | **kvm_ovs_dpdk-bar**|         |         |    X    |         |
 175 +---------------------+---------+---------+---------+---------+
 176 |   lxd               |         |         |         |    X    |
 177 +---------------------+---------+---------+---------+---------+
 178 | **ovs**             |         |         |    X    |         |
 179 +---------------------+---------+---------+---------+---------+
 180 | **openo**           |         |    X    |         |         |
 181 +---------------------+---------+---------+---------+---------+
 182 |   odl_l2            |         |    X    |   X     |    X    |
 183 +---------------------+---------+---------+---------+---------+
 184 |   odl-bgpvpn        |   X     |         |         |         |
 185 +---------------------+---------+---------+---------+---------+
 186 |   odl_l2-bgpvpn     |         |         |   X     |         |
 187 +---------------------+---------+---------+---------+---------+
 188 | **odl_l2-fdio**     |    X    |         |         |         |
 189 +---------------------+---------+---------+---------+---------+
 190 |   odl_l2-sfc        |         |         |    X    |         |
 191 +---------------------+---------+---------+---------+---------+
 192 |   odl_l3            |    X    |    X    |    X    |         |
 193 +---------------------+---------+---------+---------+---------+
 194 | **ocl**             |         |   X     |         |         |
 195 +---------------------+---------+---------+---------+---------+
 196 |   onos              |         |   X     |         |         |
 197 +---------------------+---------+---------+---------+---------+
 198 |   multisite         |         |         |    X    |         |
 199 +---------------------+---------+---------+---------+---------+
 200
 201 Non HA scenarios:
 202
 203 +---------------------+---------+---------+---------+---------+
 204 |    Scenario         |  Apex   | Compass |  Fuel   |   Joid  |
 205 +=====================+=========+=========+=========+=========+
 206 |   nosdn             |         |         |    X    |    X    |
 207 +---------------------+---------+---------+---------+---------+
 208 |   kvm               |         |         |    X    |         |
 209 +---------------------+---------+---------+---------+---------+
 210 | **kvm_ovs_dpdk**    |         |         |    X    |         |
 211 +---------------------+---------+---------+---------+---------+
 212 | **kvm_ovs_dpdk-bar**|         |         |    X    |         |
 213 +---------------------+---------+---------+---------+---------+
 214 |   lxd               |         |         |         |    X    |
 215 +---------------------+---------+---------+---------+---------+
 216 |   ovs               |         |         |    X    |         |
 217 +---------------------+---------+---------+---------+---------+
 218 |   odl_l2            |         |         |   X     |         |
 219 +---------------------+---------+---------+---------+---------+
 220 |   odl_l2-bgpvpn     |         |         |   X     |         |
 221 +---------------------+---------+---------+---------+---------+
 222 |   odl_l2-fdio       |    X    |         |         |         |
 223 +---------------------+---------+---------+---------+---------+
 224 | **odl_l3-fdio**     |    X    |         |         |         |
 225 +---------------------+---------+---------+---------+---------+
 226 |   odl_l2-sfc        |         |         |    X    |         |
 227 +---------------------+---------+---------+---------+---------+
 228 |   odl_l3            |         |         |    X    |         |
 229 +---------------------+---------+---------+---------+---------+
 230 | **odl_gluon**       |    X    |         |         |         |
 231 +---------------------+---------+---------+---------+---------+
 232
 233 Colorado deprecated scenarios:
 234
 235  * odl_l2-moon
 236  * onos-sfc
 237  * onos-noha
 238  * onos-sfc-noha
 239
 240 For the scenario validation, we consider only the healthcheck, smoke and
 241 features categories. These tests are run systematically in the CI daily loops.
 242
 243 Success criterias have been defined for these test cases, they can be
 244 PASS/FAIL or a success rate may be declared (100%, > 90%).
 245 All the criteria, as well as the test dependencies are declared in the
 246 testcases.yaml file located in the ci directory of the repository.
 247
 248 The scoring for the Danube release per installer can be described as
 249 follows.
 250
 251 The scoring is an indicator showing how many feature project test suites
 252 have been integrated in the scenario.
 253
 254 The scoring equals the number of tests * successful iteration of each
 255 test [0-3]. The scoring method is described in https://wiki.opnfv.org/pages/viewpage.action?pageId=6828617
 256
 257  e.g.
 258  apex/odl_l2-nofeature-ha
 259  tests = vping_ssh+vping_userdata+tempest+rally+odl+doctor
 260  Scoring = 18/18 = 6 * 3
 261
 262 By default, if not specified, the scenarios are HA.
 263 HA means OpenStack High Availability (main services). Note that not
 264 all VIM (e.g. OpenStack) services are deployed in HA mode, as that
 265 depends upon support of the specific service for HA deployment.
 266 For example, in the Danube release, the Congress service
 267 is deployed in non-HA mode even for HA OPNFV scenarios, as explicit
 268 support for HA operation has not yet been verified.
 269 See the release notes of the installers for more details.
 270
 271
 272 Apex
 273 ----
 274
 275 +------------------+---------+---------+-----------------+
 276 |  Scenario        | Scoring | Success |    Results      |
 277 |                  |         | rate    |                 |
 278 +==================+=========+=========+=================+
 279 | nosdn            |  20/30  |    67%  | `apex-res-1`_   |
 280 +------------------+---------+---------+-----------------+
 281 | odl_l3           |  16/30  |    53%  | `apex-res-2`_   |
 282 +------------------+---------+---------+-----------------+
 283 | odl-bgpvpn       |  18/33  |    56%  | `apex-res-3`_   |
 284 +------------------+---------+---------+-----------------+
 285 | odl-gluon        |  20/33  |    61%  | `apex-res-4`_   |
 286 +------------------+---------+---------+-----------------+
 287 | kvm              |  20/30  |    67%  | `apex-res-5`_   |
 288 +------------------+---------+---------+-----------------+
 289 | odl_l2-fdio      |  25/33  |    76%  | `apex-res-6`_   |
 290 +------------------+---------+---------+-----------------+
 291 | odl_l2-fdio-noha |  24/33  |    73%  |   local push    |
 292 +------------------+---------+---------+-----------------+
 293 | odl_l3-fdio-noha |  18/30  |    60%  |   local push    |
 294 +------------------+---------+---------+-----------------+
 295 | fdio             |  25/27  |    93%  |   local push    |
 296 +------------------+---------+---------+-----------------+
 297
 298 Compass
 299 -------
 300
 301 +------------------+---------+---------+------------------+
 302 |  Scenario        | Scoring | Success |  Results         |
 303 |                  |         | rate    |                  |
 304 +==================+=========+=========+==================+
 305 | nosdn            |  29/30  |    97%  | `compass-res-1`_ |
 306 +------------------+---------+---------+------------------+
 307 | odl_l2           |  29/33  |    88%  | `compass-res-2`_ |
 308 +------------------+---------+---------+------------------+
 309 | odl_l3           |  23/30  |    77%  | `compass-res-3`_ |
 310 +------------------+---------+---------+------------------+
 311 | onos             |  29/30  |    97%  | `compass-res-4`_ |
 312 +------------------+---------+---------+------------------+
 313 | onos-sfc         |  20/36  |    56%  | `compass-res-5`_ |
 314 +------------------+---------+---------+------------------+
 315
 316 Note: all the Compass tests for Danube have been executed on virtual
 317 environment. Bare metal resources were used for Master branch.
 318
 319
 320 Fuel
 321 ----
 322
 323 +----------------------+---------+---------+----------------+
 324 |  Scenario            | Scoring | Success |  Results       |
 325 |                      |         | rate    |                |
 326 +======================+=========+=========+================+
 327 | nosdn                |  39/39  |  100%   | `fuel-res-1`_  |
 328 +----------------------+---------+---------+----------------+
 329 | nosdn-noha           |  36/36  |  100%   | `fuel-res-2`_  |
 330 +----------------------+---------+---------+----------------+
 331 | nosdn-kvm            |  39/39  |  100%   | `fuel-res-3`_  |
 332 +----------------------+---------+---------+----------------+
 333 | nosdn-kvm-noha       |  36/36  |  100%   | `fuel-res-4`_  |
 334 +----------------------+---------+---------+----------------+
 335 | nosdn-ovs            |  39/39  |  100%   | `fuel-res-5`_  |
 336 +----------------------+---------+---------+----------------+
 337 | nosdn-ovs-noha       |  36/36  |  100%   | `fuel-res-6`_  |
 338 +----------------------+---------+---------+----------------+
 339 | odl_l2               |  42/42  |  100%   | `fuel-res-7`_  |
 340 +----------------------+---------+---------+----------------+
 341 | odl_l2-noha          |  39/39  |  100%   | `fuel-res-8`_  |
 342 +----------------------+---------+---------+----------------+
 343 | odl_l2-sfc           |  45/45  |  100%   | `fuel-res-9`_  |
 344 +----------------------+---------+---------+----------------+
 345 | odl_l2-sfc-noha      |  35/42  |   83%   | `fuel-res-10`_ |
 346 +----------------------+---------+---------+----------------+
 347 | odl_l3               |  36/39  |   92%   | `fuel-res-11`_ |
 348 +----------------------+---------+---------+----------------+
 349 | odl_l3-noha          |  36/36  |  100%   | `fuel-res-12`_ |
 350 +----------------------+---------+---------+----------------+
 351 | kvm_ovs_dpdk         |  39/39  |  100%   | `fuel-res-13`_ |
 352 +----------------------+---------+---------+----------------+
 353 | kvm_ovs_dpdk_noha    |  35/36  |   97%   | `fuel-res-14`_ |
 354 +----------------------+---------+---------+----------------+
 355 | kvm_ovs_dpdk_bar     |  42/42  |  100%   | `fuel-res-15`_ |
 356 +----------------------+---------+---------+----------------+
 357 | kvm_ovs_dpdk_bar_noha|  36/39  |   92%   | `fuel-res-16`_ |
 358 +----------------------+---------+---------+----------------+
 359
 360
 361
 362
 363 Joid
 364 ----
 365
 366 +---------------------+---------+---------+---------------+
 367 |  Scenario           | Scoring | Success |  Results      |
 368 |                     |         | rate    |               |
 369 +=====================+=========+=========+===============+
 370 | nosdn               |  30/30  |  100%   | `joid-res-1`_ |
 371 +---------------------+---------+---------+---------------+
 372 | nosdn-noha          |  10/30  |   33%   | `joid-res-2`_ |
 373 +---------------------+---------+---------+---------------+
 374 | nosdn-lxd           |  19/21  |   90%   | `joid-res-3`_ |
 375 +---------------------+---------+---------+---------------+
 376 | nosdn-lxd-noha      |  15/21  |   71%   | `joid-res-4`_ |
 377 +---------------------+---------+---------+---------------+
 378 | odl_l2              |   6/33  |   18%   | `joid-res-5`_ |
 379 +---------------------+---------+---------+---------------+
 380
 381 It is highly recommended to install a json viewer in your browser
 382 (e.g. https://addons.mozilla.org/fr/firefox/addon/jsonview/)
 383
 384 You can get additional details through test logs on http://artifacts.opnfv.org/.
 385 As no search engine is available on the OPNFV artifact web site you must
 386 retrieve the pod identifier on which the tests have been executed (see
 387 field pod in any of the results) then click on the selected POD and look
 388 for the date of the test you are interested in.
 389
 390 The reporting pages can be found at:
 391
 392  * apex: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-apex.html
 393  * compass: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-compass.html
 394  * fuel: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-fuel.html
 395  * joid: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-joid.html
 396
 397 Danube known restrictions/issues
 398 ==================================
 399
 400 ************* TODO *****************************************
 401
 402 +-----------+-----------+----------------------------------------------+
 403 | Installer | Scenario  |  Issue                                       |
 404 +===========+===========+==============================================+
 405 | fuel      | odl_-*    | Tempest test case "TestServerBasicOps"       |
 406 |           |           | disabled due to bug `tempest-bug`_           |
 407 +-----------+-----------+----------------------------------------------+
 408 | apex/fuel | *-bgpvpn  | Due to some instabilities in the bgpvpn      |
 409 |           |           | test case, the scenario has been postponed   |
 410 |           |           | to Danube 2.0                                |
 411 +-----------+-----------+----------------------------------------------+
 412 | apex      | *-gluon   | vPing_ssh disabled due to floating ips       |
 413 |           |           | not working 100% of the times.               |
 414 |           |           | Tempest test "test_reboot_server_hard"       |
 415 |           |           | disabled due to bug `gluon-bug`_             |
 416 +-----------+-----------+----------------------------------------------+
 417 | joid      | any       | Tempest cases related to object storage      |
 418 |           |           | excluded                                     |
 419 +-----------+-----------+----------------------------------------------+
 420 | any       | any       | The VNF tier has not been fully tested       |
 421 |           |           | since it has not been run in daily loops     |
 422 |           |           | in CI. Weekly jobs have been activated       |
 423 |           |           | a bit late in the process and have not been  |
 424 |           |           | used to validate the scenarios.              |
 425 +-----------+-----------+----------------------------------------------+
 426
 427 Test and installer/scenario dependencies
 428 ========================================
 429
 430 It is not always possible to run all the test cases on all the scenarios.
 431 The following table details the dependencies of the test cases per
 432 scenario. The scenario dependencies (installer or scenario) are detailed
 433 in https://git.opnfv.org/cgit/functest/tree/ci/testcases.yaml
 434
 435 Test results
 436 ============
 437
 438 Test results are available in:
 439
 440  - test results document: http://artifacts.opnfv.org/functest
 441
 442  - jenkins logs on CI: https://build.opnfv.org/ci/view/functest/
 443
 444  - jenkins logs on ARM CI: https://build.opnfv.org/ci/view/armband/
 445
 446
 447
 448 Open JIRA tickets
 449 =================
 450
 451 +------------------+-----------------------------------------------+
 452 |   JIRA           |         Description                           |
 453 +==================+===============================================+
 454 |                  |                                               |
 455 |                  |                                               |
 456 +------------------+-----------------------------------------------+
 457
 458 All the tickets that are not blocking have been fixed or postponed
 459 the next release.
 460
 461 Functest Danube 2.0 is released without known bugs.
 462
 463
 464
 465 Useful links
 466 ============
 467
 468  - wiki project page: https://wiki.opnfv.org/opnfv_functional_testing
 469
 470  - wiki Functest Danube page: https://wiki.opnfv.org/display/functest/Functest+Danube
 471
 472  - Functest repo: https://git.opnfv.org/cgit/functest
 473
 474  - Functest CI dashboard: https://build.opnfv.org/ci/view/functest/
 475
 476  - JIRA dashboard: https://jira.opnfv.org/secure/Dashboard.jspa?selectPageId=10611
 477
 478  - Functest IRC chan: #opnfv-functest
 479
 480  - Reporting page: http://testresults.opnfv.org/reporting/danube.html
 481
 482  - Functest test configuration: https://git.opnfv.org/cgit/functest/tree/functest/ci/testcases.yaml
 483
 484 .. _`tempest-bug`: https://bugs.launchpad.net/tempest/+bug/1577632
 485
 486 .. _`gluon-bug`: https://bugs.opendaylight.org/show_bug.cgi?id=5586
 487
 488
 489 .. _`apex-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-455
 490
 491 .. _`apex-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-466
 492
 493 .. _`apex-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-449
 494
 495 .. _`apex-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-450
 496
 497 .. _`apex-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-461
 498
 499 .. _`apex-res-6`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-baremetal-daily-danube-485
 500
 501
 502 .. _`compass-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-danube-484
 503
 504 .. _`compass-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-danube-454
 505
 506 .. _`compass-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-baremetal-daily-danube-482
 507
 508 .. _`compass-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-danube-481
 509
 510 .. _`compass-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-baremetal-daily-danube-453
 511
 512
 513 .. _`fuel-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-54
 514
 515 .. _`fuel-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-937
 516
 517 .. _`fuel-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-873
 518
 519 .. _`fuel-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-935
 520
 521 .. _`fuel-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-875
 522
 523 .. _`fuel-res-6`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-936
 524
 525 .. _`fuel-res-7`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-867
 526
 527 .. _`fuel-res-8`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-939
 528
 529 .. _`fuel-res-9`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-870
 530
 531 .. _`fuel-res-10`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-933
 532
 533 .. _`fuel-res-11`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-868
 534
 535 .. _`fuel-res-12`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-940
 536
 537 .. _`fuel-res-13`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-871
 538
 539 .. _`fuel-res-14`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-938
 540
 541 .. _`fuel-res-15`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-869
 542
 543 .. _`fuel-res-16`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-941
 544
 545
 546 .. _`joid-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-298
 547
 548 .. _`joid-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-260
 549
 550 .. _`joid-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-281
 551
 552 .. _`joid-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-301
 553
 554 .. _`joid-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-291