docs/release/release-notes/functest-release.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
   2 .. SPDX-License-Identifier: CC-BY-4.0
   3
   4 =======
   5 License
   6 =======
   7
   8 OPNFV Danube release note for Functest Docs
   9 (c) by Jose Lausuch (Ericsson)
  10
  11 OPNFV Danube release note for Functest Docs
  12 are licensed under a Creative Commons Attribution 4.0 International License.
  13 You should have received a copy of the license along with this.
  14 If not, see <http://creativecommons.org/licenses/by/4.0/>.
  15
  16 ===========================================
  17 OPNFV Danube2.0 release note for Functest
  18 ===========================================
  19
  20 Abstract
  21 ========
  22
  23 This document describes the release note of Functest project.
  24
  25
  26 Version history
  27 ===============
  28
  29 +------------+----------+------------------+------------------------+
  30 | **Date**   | **Ver.** | **Author**       | **Comment**            |
  31 |            |          |                  |                        |
  32 +------------+----------+------------------+------------------------+
  33 | 2016-08-17 | 1.0.0    | Morgan Richomme  | Functest for Colorado  |
  34 |            |          | (Orange)         | release                |
  35 +------------+----------+------------------+------------------------+
  36 | 2017-03-29 | 4.0.0    | Jose Lausuch     | Functest for Danube    |
  37 |            |          | (Ericsson)       | release                |
  38 +------------+----------+------------------+------------------------+
  39 | 2017-05-04 | 5.0.0    | Jose Lausuch     | Functest for Danube.2.0|
  40 |            |          | (Ericsson)       | release                |
  41 +------------+----------+------------------+------------------------+
  42
  43
  44 OPNFV Danube Release
  45 ======================
  46
  47 Functest deals with functional testing of the OPNFV solution.
  48 It includes test cases developed within the project and test cases developed in
  49 other OPNFV projects and other upstream communities.
  50
  51 The internal test cases are:
  52
  53  * connection_check
  54  * api_check
  55  * snaps_health_check
  56  * vping_ssh
  57  * vping_userdata
  58  * tempest_smoke_serial
  59  * refstack_defcore
  60  * snaps_smoke
  61  * rally_sanity
  62  * odl
  63  * tempest_full_parallel
  64  * rally_full
  65  * cloudify_ims
  66
  67 The OPNFV projects integrated into Functest framework for automation are:
  68
  69  * barometer
  70  * bgpvpn
  71  * doctor
  72  * domino
  73  * fds
  74  * multisite
  75  * netready
  76  * onos
  77  * odl-sfc
  78  * odl-netvirt
  79  * parser
  80  * promise
  81  * security scan
  82  * orchestra_ims
  83  * vyos_vrouter
  84
  85 The validation of a scenario requires a subset of these tests depending
  86 on the installer and the scenario.
  87
  88 The test cases from vnf (cloudify_ims, orchestra_ims, vyos_vrouter) and
  89 component categories (tempest full parallel, Rally full) are not considered for
  90 scenario validation.
  91
  92 Release Data
  93 ============
  94
  95 +--------------------------------------+--------------------------------------+
  96 | **Project**                          | functest                             |
  97 |                                      |                                      |
  98 +--------------------------------------+--------------------------------------+
  99 | **Repo/tag**                         | danube.2.0                           |
 100 |                                      |                                      |
 101 +--------------------------------------+--------------------------------------+
 102 | **Release designation**              | Danube service release               |
 103 |                                      |                                      |
 104 +--------------------------------------+--------------------------------------+
 105 | **Release date**                     | May 4st 2017                         |
 106 |                                      |                                      |
 107 +--------------------------------------+--------------------------------------+
 108 | **Purpose of the delivery**          | Danube second release                |
 109 |                                      |                                      |
 110 +--------------------------------------+--------------------------------------+
 111
 112 Deliverables
 113 ============
 114
 115 Software
 116 --------
 117
 118  - The Functest Docker image: https://hub.docker.com/r/opnfv/functest (tag: danube.2.0)
 119
 120  - The TestAPI Docker image: https://hub.docker.com/r/opnfv/testapi (tag:danube.2.0)
 121
 122
 123 Documents
 124 ---------
 125
 126  - Installation/configuration guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
 127
 128  - User Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html
 129
 130  - Developer Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/developer/devguide/index.html
 131
 132
 133 Version change
 134 ==============
 135
 136 Feature evolution
 137 -----------------
 138
 139 - Adoption of SNAPS as middleware in 4 new test cases (connection_check, api_check,
 140    snaps_health_check and snaps_smoke)
 141
 142 - Introduction of refstack suite
 143
 144 - Support new odl suites (odl-netvirt, fds)
 145
 146 - Introduction of VNF onboarding capabilities
 147
 148 - Support of new feature projects (fds, netready, barometer, orchestra, vyos_vrouter)
 149
 150
 151
 152 Framework
 153 ---------
 154
 155  - Harmonization of the naming, better adoption of OpenStack coding conventions
 156
 157  - Enhanced code to be more Object Oriented, removed bash scripts
 158
 159  - Introduction of abstraction classes to ease and harmonize the integration of
 160     test cases (internal or from feature projects)
 161
 162  - New management of logger, env variables and configuration files
 163
 164  - Creation of unit tests on the whole framework to ensure stability
 165
 166  - Creation or ARM Functest docker
 167
 168
 169 Test API
 170 ---------
 171
 172 - Automatic documentation (html & pdf)
 173
 174 - Full dockerization and automation of the deployment on testresults.opnfv.org
 175
 176 - Automation of test database backup on artifact
 177
 178
 179 New internal tests cases
 180 ------------------------
 181
 182 - connection_check
 183
 184 - api_check
 185
 186 - snaps_health_check (replacing shell script healtcheck)
 187
 188 - refstack_defcore
 189
 190 - snaps_smoke
 191
 192 - vyos_vrouter
 193
 194
 195 Scenario Matrix
 196 ===============
 197
 198 For Danube 2.0, Functest was tested on the following HA scenarios (new
 199 dabube scenarios in bold):
 200
 201 +---------------------+---------+---------+---------+---------+
 202 |    Scenario         |  Apex   | Compass |  Fuel   |   Joid  |
 203 +=====================+=========+=========+=========+=========+
 204 |   nosdn             |    X    |    X    |    X    |    X    |
 205 +---------------------+---------+---------+---------+---------+
 206 | **fdio**            |    X    |         |         |         |
 207 +---------------------+---------+---------+---------+---------+
 208 |   kvm               |    X    |         |    X    |         |
 209 +---------------------+---------+---------+---------+---------+
 210 | **kvm_ovs_dpdk**    |         |         |    X    |         |
 211 +---------------------+---------+---------+---------+---------+
 212 | **kvm_ovs_dpdk-bar**|         |         |    X    |         |
 213 +---------------------+---------+---------+---------+---------+
 214 |   lxd               |         |         |         |    X    |
 215 +---------------------+---------+---------+---------+---------+
 216 | **ovs**             |         |         |    X    |         |
 217 +---------------------+---------+---------+---------+---------+
 218 | **openo**           |         |    X    |         |         |
 219 +---------------------+---------+---------+---------+---------+
 220 |   odl_l2            |         |    X    |   X     |    X    |
 221 +---------------------+---------+---------+---------+---------+
 222 |   odl-bgpvpn        |   X     |         |         |         |
 223 +---------------------+---------+---------+---------+---------+
 224 |   odl_l2-bgpvpn     |         |         |   X     |         |
 225 +---------------------+---------+---------+---------+---------+
 226 | **odl_l2-fdio**     |    X    |         |         |         |
 227 +---------------------+---------+---------+---------+---------+
 228 |   odl_l2-sfc        |         |         |    X    |         |
 229 +---------------------+---------+---------+---------+---------+
 230 |   odl_l3            |    X    |    X    |    X    |         |
 231 +---------------------+---------+---------+---------+---------+
 232 | **ocl**             |         |   X     |         |         |
 233 +---------------------+---------+---------+---------+---------+
 234 |   onos              |         |   X     |         |         |
 235 +---------------------+---------+---------+---------+---------+
 236 |   multisite         |         |         |    X    |         |
 237 +---------------------+---------+---------+---------+---------+
 238
 239 Non HA scenarios:
 240
 241 +---------------------+---------+---------+---------+---------+
 242 |    Scenario         |  Apex   | Compass |  Fuel   |   Joid  |
 243 +=====================+=========+=========+=========+=========+
 244 |   nosdn             |         |         |    X    |    X    |
 245 +---------------------+---------+---------+---------+---------+
 246 |   kvm               |         |         |    X    |         |
 247 +---------------------+---------+---------+---------+---------+
 248 | **kvm_ovs_dpdk**    |         |         |    X    |         |
 249 +---------------------+---------+---------+---------+---------+
 250 | **kvm_ovs_dpdk-bar**|         |         |    X    |         |
 251 +---------------------+---------+---------+---------+---------+
 252 |   lxd               |         |         |         |    X    |
 253 +---------------------+---------+---------+---------+---------+
 254 |   ovs               |         |         |    X    |         |
 255 +---------------------+---------+---------+---------+---------+
 256 |   odl_l2            |         |         |   X     |         |
 257 +---------------------+---------+---------+---------+---------+
 258 |   odl_l2-bgpvpn     |         |         |   X     |         |
 259 +---------------------+---------+---------+---------+---------+
 260 |   odl_l2-fdio       |    X    |         |         |         |
 261 +---------------------+---------+---------+---------+---------+
 262 | **odl_l3-fdio**     |    X    |         |         |         |
 263 +---------------------+---------+---------+---------+---------+
 264 |   odl_l2-sfc        |         |         |    X    |         |
 265 +---------------------+---------+---------+---------+---------+
 266 |   odl_l3            |         |         |    X    |         |
 267 +---------------------+---------+---------+---------+---------+
 268 | **odl_gluon**       |    X    |         |         |         |
 269 +---------------------+---------+---------+---------+---------+
 270
 271 Colorado deprecated scenarios:
 272
 273  * odl_l2-moon
 274  * onos-sfc
 275  * onos-noha
 276  * onos-sfc-noha
 277
 278 For the scenario validation, we consider only the healthcheck, smoke and
 279 features categories. These tests are run systematically in the CI daily loops.
 280
 281 Success criterias have been defined for these test cases, they can be
 282 PASS/FAIL or a success rate may be declared (100%, > 90%).
 283 All the criteria, as well as the test dependencies are declared in the
 284 testcases.yaml file located in the ci directory of the repository.
 285
 286 The scoring for the Danube release per installer can be described as
 287 follows.
 288
 289 The scoring is an indicator showing how many feature project test suites
 290 have been integrated in the scenario.
 291
 292 The scoring equals the number of tests * successful iteration of each
 293 test [0-3]. The scoring method is described in https://wiki.opnfv.org/pages/viewpage.action?pageId=6828617
 294
 295  e.g.
 296  apex/odl_l2-nofeature-ha
 297  tests = vping_ssh+vping_userdata+tempest+rally+odl+doctor+copper
 298  Scoring = 21/21 = 7 * 3
 299
 300 By default, if not specified, the scenarios are HA.
 301 HA means OpenStack High Availability (main services). Note that not
 302 all VIM (e.g. OpenStack) services are deployed in HA mode, as that
 303 depends upon support of the specific service for HA deployment.
 304 For example, in the Danube release, the Congress service
 305 is deployed in non-HA mode even for HA OPNFV scenarios, as explicit
 306 support for HA operation has not yet been verified.
 307 See the release notes of the installers for more details.
 308
 309
 310 Apex
 311 ----
 312
 313 +------------------+---------+---------+-----------------+
 314 |  Scenario        | Scoring | Success |    Results      |
 315 |                  |         | rate    |                 |
 316 +==================+=========+=========+=================+
 317 | nosdn            |  30/33  |    91%  | `apex-res-1`_   |
 318 +------------------+---------+---------+-----------------+
 319 | odl_l3           |  23/33  |    70%  | `apex-res-2`_   |
 320 +------------------+---------+---------+-----------------+
 321 | odl-bgpvpn       |  16/36  |    44%  | `apex-res-3`_   |
 322 +------------------+---------+---------+-----------------+
 323 | odl-gluon        |  18/36  |    50%  | `apex-res-4`_   |
 324 +------------------+---------+---------+-----------------+
 325 | kvm              |  30/33  |    91%  | `apex-res-5`_   |
 326 +------------------+---------+---------+-----------------+
 327 | odl_l2-fdio      |  30/36  |    83%  | `apex-res-6`_   |
 328 +------------------+---------+---------+-----------------+
 329 | odl_l2-fdio-noha |  29/36  |    81%  | `apex-res-7`_   |
 330 +------------------+---------+---------+-----------------+
 331 | odl_l3-fdio-noha |  23/30  |    77%  | `apex-res-8`_   |
 332 +------------------+---------+---------+-----------------+
 333 | fdio             |   6/30  |    20%  | `apex-res-9`_   |
 334 +------------------+---------+---------+-----------------+
 335
 336 Compass
 337 -------
 338
 339 +------------------+---------+---------+------------------+
 340 |  Scenario        | Scoring | Success |  Results         |
 341 |                  |         | rate    |                  |
 342 +==================+=========+=========+==================+
 343 | nosdn            |  29/30  |    97%  | `compass-res-1`_ |
 344 +------------------+---------+---------+------------------+
 345 | odl_l2           |  29/33  |    88%  | `compass-res-2`_ |
 346 +------------------+---------+---------+------------------+
 347 | odl_l3           |  25/30  |    83%  | `compass-res-3`_ |
 348 +------------------+---------+---------+------------------+
 349 | onos             |  28/33  |    84%  | `compass-res-4`_ |
 350 +------------------+---------+---------+------------------+
 351 | openo            |  29/30  |    97%  | `compass-res-5`_ |
 352 +------------------+---------+---------+------------------+
 353 | ocl              |   3/30  |    10%  | `compass-res-6`_ |
 354 +------------------+---------+---------+------------------+
 355
 356 Note: all the Compass tests for Danube have been executed on virtual
 357 environment. Bare metal resources were used for Master branch.
 358
 359
 360 Fuel
 361 ----
 362
 363 +----------------------+---------+---------+----------------+
 364 |  Scenario            | Scoring | Success |  Results       |
 365 |                      |         | rate    |                |
 366 +======================+=========+=========+================+
 367 | nosdn                |  38/39  |   97%   | `fuel-res-1`_  |
 368 +----------------------+---------+---------+----------------+
 369 | nosdn-noha           |  36/36  |  100%   | `fuel-res-2`_  |
 370 +----------------------+---------+---------+----------------+
 371 | nosdn-kvm            |  39/39  |  100%   | `fuel-res-3`_  |
 372 +----------------------+---------+---------+----------------+
 373 | nosdn-kvm-noha       |  36/36  |  100%   | `fuel-res-4`_  |
 374 +----------------------+---------+---------+----------------+
 375 | nosdn-ovs            |  39/39  |  100%   | `fuel-res-5`_  |
 376 +----------------------+---------+---------+----------------+
 377 | nosdn-ovs-noha       |  36/36  |  100%   | `fuel-res-6`_  |
 378 +----------------------+---------+---------+----------------+
 379 | odl_l2               |  40/42  |   95%   | `fuel-res-7`_  |
 380 +----------------------+---------+---------+----------------+
 381 | odl_l2-noha          |  37/39  |   95%   | `fuel-res-8`_  |
 382 +----------------------+---------+---------+----------------+
 383 | odl_l2-sfc           |  45/45  |  100%   | `fuel-res-9`_  |
 384 +----------------------+---------+---------+----------------+
 385 | odl_l2-sfc-noha      |  36/42  |   86%   | `fuel-res-10`_ |
 386 +----------------------+---------+---------+----------------+
 387 | odl_l3               |  38/39  |   97%   | `fuel-res-11`_ |
 388 +----------------------+---------+---------+----------------+
 389 | odl_l3-noha          |  36/36  |  100%   | `fuel-res-12`_ |
 390 +----------------------+---------+---------+----------------+
 391 | kvm_ovs_dpdk         |   6/39  |   15%   | `fuel-res-13`_ |
 392 +----------------------+---------+---------+----------------+
 393 | kvm_ovs_dpdk_noha    |  36/36  |  100%   | `fuel-res-14`_ |
 394 +----------------------+---------+---------+----------------+
 395 | kvm_ovs_dpdk_bar     |   6/42  |   14%   | `fuel-res-15`_ |
 396 +----------------------+---------+---------+----------------+
 397 | kvm_ovs_dpdk_bar_noha|  38/39  |   97%   | `fuel-res-16`_ |
 398 +----------------------+---------+---------+----------------+
 399
 400
 401
 402
 403 Joid
 404 ----
 405
 406 +---------------------+---------+---------+---------------+
 407 |  Scenario           | Scoring | Success |  Results      |
 408 |                     |         | rate    |               |
 409 +=====================+=========+=========+===============+
 410 | nosdn               |  29/33  |   97%   | `joid-res-1`_ |
 411 +---------------------+---------+---------+---------------+
 412 | nosdn-noha          |  33/33  |  100%   | `joid-res-2`_ |
 413 +---------------------+---------+---------+---------------+
 414 | nosdn-lxd           |  18/24  |   75%   | `joid-res-3`_ |
 415 +---------------------+---------+---------+---------------+
 416 | nosdn-lxd-noha      |  18/24  |   71%   | `joid-res-4`_ |
 417 +---------------------+---------+---------+---------------+
 418 | odl_l2              |   9/36  |   25%   | `joid-res-5`_ |
 419 +---------------------+---------+---------+---------------+
 420
 421 It is highly recommended to install a json viewer in your browser
 422 (e.g. https://addons.mozilla.org/fr/firefox/addon/jsonview/)
 423
 424 You can get additional details through test logs on http://artifacts.opnfv.org/.
 425 As no search engine is available on the OPNFV artifact web site you must
 426 retrieve the pod identifier on which the tests have been executed (see
 427 field pod in any of the results) then click on the selected POD and look
 428 for the date of the test you are interested in.
 429
 430 The reporting pages can be found at:
 431
 432  * apex: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-apex.html
 433  * compass: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-compass.html
 434  * fuel: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-fuel.html
 435  * joid: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-joid.html
 436
 437 Danube known restrictions/issues
 438 ==================================
 439
 440 ************* TODO *****************************************
 441
 442 +-----------+-----------+----------------------------------------------+
 443 | Installer | Scenario  |  Issue                                       |
 444 +===========+===========+==============================================+
 445 | fuel      | odl_-*    | Tempest test case "TestServerBasicOps"       |
 446 |           |           | disabled due to bug `tempest-bug`_           |
 447 +-----------+-----------+----------------------------------------------+
 448 | apex/fuel | *-bgpvpn  | Due to some instabilities in the bgpvpn      |
 449 |           |           | test case, the scenario has been postponed   |
 450 |           |           | to Danube 2.0                                |
 451 +-----------+-----------+----------------------------------------------+
 452 | apex      | *-gluon   | vPing_ssh disabled due to floating ips       |
 453 |           |           | not working 100% of the times.               |
 454 |           |           | Tempest test "test_reboot_server_hard"       |
 455 |           |           | disabled due to bug `gluon-bug`_             |
 456 +-----------+-----------+----------------------------------------------+
 457 | joid      | any       | Tempest cases related to object storage      |
 458 |           |           | excluded                                     |
 459 +-----------+-----------+----------------------------------------------+
 460 | any       | any       | The VNF tier has not been fully tested       |
 461 |           |           | since it has not been run in daily loops     |
 462 |           |           | in CI. Weekly jobs have been activated       |
 463 |           |           | a bit late in the process and have not been  |
 464 |           |           | used to validate the scenarios.              |
 465 +-----------+-----------+----------------------------------------------+
 466
 467 Test and installer/scenario dependencies
 468 ========================================
 469
 470 It is not always possible to run all the test cases on all the scenarios.
 471 The following table details the dependencies of the test cases per
 472 scenario. The scenario dependencies (installer or scenario) are detailed
 473 in https://git.opnfv.org/cgit/functest/tree/ci/testcases.yaml
 474
 475 Test results
 476 ============
 477
 478 Test results are available in:
 479
 480  - test results document: http://artifacts.opnfv.org/functest
 481
 482  - jenkins logs on CI: https://build.opnfv.org/ci/view/functest/
 483
 484  - jenkins logs on ARM CI: https://build.opnfv.org/ci/view/armband/
 485
 486
 487
 488 Open JIRA tickets
 489 =================
 490
 491 +------------------+-----------------------------------------------+
 492 |   JIRA           |         Description                           |
 493 +==================+===============================================+
 494 |                  |                                               |
 495 |                  |                                               |
 496 +------------------+-----------------------------------------------+
 497
 498 All the tickets that are not blocking have been fixed or postponed
 499 the next release.
 500
 501 Functest Danube 2.0 is released without known bugs.
 502
 503
 504
 505 Useful links
 506 ============
 507
 508  - wiki project page: https://wiki.opnfv.org/opnfv_functional_testing
 509
 510  - wiki Functest Danube page: https://wiki.opnfv.org/display/functest/Functest+Danube
 511
 512  - Functest repo: https://git.opnfv.org/cgit/functest
 513
 514  - Functest CI dashboard: https://build.opnfv.org/ci/view/functest/
 515
 516  - JIRA dashboard: https://jira.opnfv.org/secure/Dashboard.jspa?selectPageId=10611
 517
 518  - Functest IRC chan: #opnfv-functest
 519
 520  - Reporting page: http://testresults.opnfv.org/reporting/danube.html
 521
 522  - Functest test configuration: https://git.opnfv.org/cgit/functest/tree/functest/ci/testcases.yaml
 523
 524 .. _`tempest-bug`: https://bugs.launchpad.net/tempest/+bug/1577632
 525
 526 .. _`gluon-bug`: https://bugs.opendaylight.org/show_bug.cgi?id=5586
 527
 528 .. _`apex-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-danube-daily-danube-222
 529
 530 .. _`apex-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-danube-daily-danube-211
 531
 532 .. _`apex-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-danube-daily-danube-224
 533
 534 .. _`apex-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-danube-daily-danube-225
 535
 536 .. _`apex-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-danube-daily-danube-216
 537
 538 .. _`apex-res-6`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-danube-daily-danube-215
 539
 540 .. _`apex-res-7`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-danube-daily-danube-214
 541
 542 .. _`apex-res-8`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-danube-daily-danube-217
 543
 544 .. _`apex-res-9`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-apex-apex-daily-danube-daily-danube-218
 545
 546 .. _`compass-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-danube-216
 547
 548 .. _`compass-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-danube-221
 549
 550 .. _`compass-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-baremetal-daily-danube-214
 551
 552 .. _`compass-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-virtual-daily-danube-213
 553
 554 .. _`compass-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-baremetal-daily-danube-212
 555
 556 .. _`compass-res-6`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-compass-baremetal-daily-danube-222
 557
 558 .. _`fuel-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-54
 559
 560 .. _`fuel-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-357
 561
 562 .. _`fuel-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-53
 563
 564 .. _`fuel-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-355
 565
 566 .. _`fuel-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-335
 567
 568 .. _`fuel-res-6`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-356
 569
 570 .. _`fuel-res-7`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-zte-pod1-daily-danube-337
 571
 572 .. _`fuel-res-8`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-358
 573
 574 .. _`fuel-res-9`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-340
 575
 576 .. _`fuel-res-10`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-353
 577
 578 .. _`fuel-res-11`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-338
 579
 580 .. _`fuel-res-12`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-360
 581
 582 .. _`fuel-res-13`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-341
 583
 584 .. _`fuel-res-14`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-359
 585
 586 .. _`fuel-res-15`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-baremetal-daily-danube-339
 587
 588 .. _`fuel-res-16`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-fuel-virtual-daily-danube-361
 589
 590
 591 .. _`joid-res-1`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-164
 592
 593 .. _`joid-res-2`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-150
 594
 595 .. _`joid-res-3`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-162
 596
 597 .. _`joid-res-4`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-167
 598
 599 .. _`joid-res-5`: http://testresults.opnfv.org/test/api/v1/results?build_tag=jenkins-functest-joid-baremetal-daily-danube-165