docs/release/release-notes.rst

   1 =======
   2 License
   3 =======
   4
   5 OPNFV Colorado release note for Yardstick Docs
   6 are licensed under a Creative Commons Attribution 4.0 International License.
   7 You should have received a copy of the license along with this.
   8 If not, see <http://creativecommons.org/licenses/by/4.0/>.
   9
  10 The *Yardstick framework*, the *Yardstick test cases* and the *ApexLake*
  11 experimental framework are opensource software, licensed under the terms of the
  12 Apache License, Version 2.0.
  13
  14 =========================================
  15 OPNFV Colorado Release Note for Yardstick
  16 =========================================
  17
  18 .. toctree::
  19    :maxdepth: 2
  20
  21 .. _Yardstick: https://wiki.opnfv.org/yardstick
  22
  23 .. _Dashboard: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
  24
  25 .. _NFV-TST001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf
  26
  27
  28 Abstract
  29 ========
  30
  31 This document describes the release note of Yardstick project.
  32
  33
  34 Version History
  35 ===============
  36
  37 +----------------+--------------------+---------------------------------+
  38 | *Date*         | *Version*          | *Comment*                       |
  39 |                |                    |                                 |
  40 +----------------+--------------------+---------------------------------+
  41 | Aug 22nd, 2016 |  1.0               | Yardstick for Colorado release  |
  42 |                |                    |                                 |
  43 +----------------+--------------------+---------------------------------+
  44
  45
  46 Important Notes
  47 ===============
  48
  49 The software delivered in the OPNFV Yardstick_ Project, comprising the
  50 *Yardstick framework*, the *Yardstick test cases* and the experimental
  51 framework *Apex Lake* is a realization of the methodology in ETSI-ISG
  52 NFV-TST001_.
  53
  54 The *Yardstick* framework is *installer*, *infrastructure* and *application*
  55 independent.
  56
  57
  58 OPNFV Colorado Release
  59 ======================
  60
  61 This Colorado release provides *Yardstick* as a framework for NFVI testing
  62 and OPNFV feature testing, automated in the OPNFV CI pipeline, including:
  63
  64 * Documentation generated with Sphinx
  65
  66   * User Guide
  67
  68   * Code Documentation
  69
  70   * Release notes (this document)
  71
  72   * Results
  73
  74 * Automated Yardstick test suite (daily, weekly)
  75
  76   * Jenkins Jobs for OPNFV community labs
  77
  78 * Automated Yardstick test results visualization
  79
  80   * Dashboard_ using Grafana (user:opnfv/password: opnfv), influxDB is used as
  81     backend
  82
  83 * Yardstick framework source code
  84
  85 * Yardstick test cases yaml files
  86
  87 * Yardstick pliug-in configration yaml files, plug-in install/remove scripts
  88
  89 For Colorado release, the *Yardstick framework* is used for the following
  90 testing:
  91
  92 * OPNFV platform testing - generic test cases to measure the categories:
  93
  94   * Compute
  95
  96   * Network
  97
  98   * Storage
  99
 100 * Test cases for the following OPNFV Projects:
 101
 102   * High Availability
 103
 104   * IPv6
 105
 106   * KVM
 107
 108   * Parser
 109
 110   * StorPerf
 111
 112   * VSperf
 113
 114   * virtual Traffic Classifier
 115
 116 The *Yardstick framework* is developed in the OPNFV community, by the
 117 Yardstick_ team. The *virtual Traffic Classifier* is a part of the Yardstick
 118 Project.
 119
 120 .. note:: The test case description template used for the Yardstick test cases
 121   is based on the document ETSI-ISG NFV-TST001_; the results report template
 122   used for the Yardstick results is based on the IEEE Std 829-2008.
 123
 124
 125 Release Data
 126 ============
 127
 128 +--------------------------------------+--------------------------------------+
 129 | **Project**                          | Yardstick                            |
 130 |                                      |                                      |
 131 +--------------------------------------+--------------------------------------+
 132 | **Repo/tag**                         | yardstick/colorado.1.0               |
 133 |                                      |                                      |
 134 +--------------------------------------+--------------------------------------+
 135 | **Yardstick Docker image tag**       | colorado.1.0                         |
 136 |                                      |                                      |
 137 +--------------------------------------+--------------------------------------+
 138 | **Release designation**              | Colorado base release                |
 139 |                                      |                                      |
 140 +--------------------------------------+--------------------------------------+
 141 | **Release date**                     | September 22 2016                    |
 142 |                                      |                                      |
 143 +--------------------------------------+--------------------------------------+
 144 | **Purpose of the delivery**          | Colorado base release                |
 145 |                                      |                                      |
 146 +--------------------------------------+--------------------------------------+
 147
 148
 149 Deliverables
 150 ============
 151
 152 Documents
 153 ---------
 154
 155  - User Guide: http://artifacts.opnfv.org/yardstick/colorado/docs/userguide/index.html
 156
 157  - Test Results: http://artifacts.opnfv.org/yardstick/colorado/docs/results/overview.html
 158
 159
 160 Software Deliverables
 161 ---------------------
 162
 163 **Yardstick framework source code <colorado.1.0>**
 164
 165 +--------------------------------------+--------------------------------------+
 166 | **Project**                          | Yardstick                            |
 167 |                                      |                                      |
 168 +--------------------------------------+--------------------------------------+
 169 | **Repo/tag**                         | yardstick/colorado.1.0               |
 170 |                                      |                                      |
 171 +--------------------------------------+--------------------------------------+
 172 | **Yardstick Docker image tag**       | colorado.1.0                         |
 173 |                                      |                                      |
 174 +--------------------------------------+--------------------------------------+
 175 | **Release designation**              | colorado                             |
 176 |                                      |                                      |
 177 +--------------------------------------+--------------------------------------+
 178 | **Release date**                     | September 22th, 2016                 |
 179 |                                      |                                      |
 180 +--------------------------------------+--------------------------------------+
 181 | **Purpose of the delivery**          | OPNFV Colorado release               |
 182 |                                      |                                      |
 183 +--------------------------------------+--------------------------------------+
 184
 185
 186 **Contexts**
 187
 188 +---------------------+-------------------------------------------------------+
 189 | **Context**         | **Description**                                       |
 190 |                     |                                                       |
 191 +---------------------+-------------------------------------------------------+
 192 | *Heat*              | Models orchestration using OpenStack Heat             |
 193 |                     |                                                       |
 194 +---------------------+-------------------------------------------------------+
 195 | *Node*              | Models Baremetal, Controller, Compute                 |
 196 |                     |                                                       |
 197 +---------------------+-------------------------------------------------------+
 198
 199
 200 **Runners**
 201
 202 +---------------------+-------------------------------------------------------+
 203 | **Runner**          | **Description**                                       |
 204 |                     |                                                       |
 205 +---------------------+-------------------------------------------------------+
 206 | *Arithmetic*        | Steps every run arithmetically according to specified |
 207 |                     | input value                                           |
 208 |                     |                                                       |
 209 +---------------------+-------------------------------------------------------+
 210 | *Duration*          | Runs for a specified period of time                   |
 211 |                     |                                                       |
 212 +---------------------+-------------------------------------------------------+
 213 | *Iteration*         | Runs for a specified number of iterations             |
 214 |                     |                                                       |
 215 +---------------------+-------------------------------------------------------+
 216 | *Sequence*          | Selects input value to a scenario from an input file  |
 217 |                     | and runs all entries sequentially                     |
 218 |                     |                                                       |
 219 +---------------------+-------------------------------------------------------+
 220
 221
 222 **Scenarios**
 223
 224 +---------------------+-------------------------------------------------------+
 225 | **Category**        | **Delivered**                                         |
 226 |                     |                                                       |
 227 +---------------------+-------------------------------------------------------+
 228 | *Availability*      | Attacker:                                             |
 229 |                     |                                                       |
 230 |                     | * baremetal, process                                  |
 231 |                     |                                                       |
 232 |                     | HA tools:                                             |
 233 |                     |                                                       |
 234 |                     | * check host, openstack, process, service             |
 235 |                     | * kill process                                        |
 236 |                     | * start/stop service                                  |
 237 |                     |                                                       |
 238 |                     | Monitor:                                              |
 239 |                     |                                                       |
 240 |                     | * command, process                                    |
 241 |                     |                                                       |
 242 +---------------------+-------------------------------------------------------+
 243 | *Compute*           | * cpuload                                             |
 244 |                     |                                                       |
 245 |                     | * cyclictest                                          |
 246 |                     |                                                       |
 247 |                     | * lmbench                                             |
 248 |                     |                                                       |
 249 |                     | * lmbench_cache                                       |
 250 |                     |                                                       |
 251 |                     | * perf                                                |
 252 |                     |                                                       |
 253 |                     | * unixbench                                           |
 254 |                     |                                                       |
 255 |                     | * ramspeed                                            |
 256 |                     |                                                       |
 257 |                     | * cachestat                                           |
 258 |                     |                                                       |
 259 |                     | * memeoryload                                         |
 260 |                     |                                                       |
 261 |                     | * computecapacity                                     |
 262 |                     |                                                       |
 263 +---------------------+-------------------------------------------------------+
 264 | *Networking*        | * iperf3                                              |
 265 |                     |                                                       |
 266 |                     | * netperf                                             |
 267 |                     |                                                       |
 268 |                     | * netperf_node                                        |
 269 |                     |                                                       |
 270 |                     | * ping                                                |
 271 |                     |                                                       |
 272 |                     | * ping6                                               |
 273 |                     |                                                       |
 274 |                     | * pktgen                                              |
 275 |                     |                                                       |
 276 |                     | * sfc                                                 |
 277 |                     |                                                       |
 278 |                     | * sfc with tacker                                     |
 279 |                     |                                                       |
 280 |                     | * vtc instantion validation                           |
 281 |                     |                                                       |
 282 |                     | * vtc instantion validation with noisy neighbors      |
 283 |                     |                                                       |
 284 |                     | * vtc throughput                                      |
 285 |                     |                                                       |
 286 |                     | * vtc throughput in the presence of noisy neighbors   |
 287 |                     |                                                       |
 288 |                     | * networkcapacity                                     |
 289 |                     |                                                       |
 290 |                     | * netutilization                                      |
 291 |                     |                                                       |
 292 +---------------------+-------------------------------------------------------+
 293 | *Parser*            | Tosca2Heat                                            |
 294 |                     |                                                       |
 295 +---------------------+-------------------------------------------------------+
 296 | *Storage*           | fio                                                   |
 297 |                     |                                                       |
 298 |                     | storagecapacity                                       |
 299 |                     |                                                       |
 300 +---------------------+-------------------------------------------------------+
 301 | *StorPerf*          | storperf                                              |
 302 |                     |                                                       |
 303 +---------------------+-------------------------------------------------------+
 304
 305
 306 **API to Other Frameworks**
 307
 308 +---------------------+-------------------------------------------------------+
 309 | **Framework**       | **Description**                                       |
 310 |                     |                                                       |
 311 +---------------------+-------------------------------------------------------+
 312 | *ApexLake*          | Experimental framework that enables the user to       |
 313 |                     | validate NFVI from the perspective of a VNF.          |
 314 |                     | A virtual Traffic Classifier is utilized as VNF.      |
 315 |                     | Enables experiments with SR-IOV on Compute Node.      |
 316 |                     |                                                       |
 317 +---------------------+-------------------------------------------------------+
 318
 319
 320 **Test Results Output**
 321
 322 +-----------------------------+-----------------------------------------------+
 323 | **Dispatcher**              | **Description**                               |
 324 |                             |                                               |
 325 +-----------------------------+-----------------------------------------------+
 326 |  file                       | Log to a file.                                |
 327 |                             |                                               |
 328 +-----------------------------+-----------------------------------------------+
 329 |  http                       | Post data to html.                            |
 330 |                             |                                               |
 331 +-----------------------------+-----------------------------------------------+
 332 |  influxdb                   | Post data to influxDB.                        |
 333 |                             |                                               |
 334 +-----------------------------+-----------------------------------------------+
 335
 336
 337 Delivered Test cases
 338 --------------------
 339
 340 * Generic NFVI test cases
 341
 342   * OPNFV_YARDSTICK_TCOO1 - NW Performance
 343
 344   * OPNFV_YARDSTICK_TCOO2 - NW Latency
 345
 346   * OPNFV_YARDSTICK_TCOO4 - Cache Utilization
 347
 348   * OPNFV_YARDSTICK_TCOO5 - Storage Performance
 349
 350   * OPNFV_YARDSTICK_TCOO8 - Packet Loss Extended Test
 351
 352   * OPNFV_YARDSTICK_TCOO9 - Packet Loss
 353
 354   * OPNFV_YARDSTICK_TCO10 - Memory Latency
 355
 356   * OPNFV_YARDSTICK_TCO11 - Packet Delay Variation Between VMs
 357
 358   * OPNFV_YARDSTICK_TCO12 - Memory Bandwidth
 359
 360   * OPNFV_YARDSTICK_TCO14 - Processing Speed
 361
 362   * OPNFV_YARDSTICK_TCO24 - CPU Load
 363
 364   * OPNFV_YARDSTICK_TCO37 - Latency, CPU Load, Throughput, Packet Loss
 365
 366   * OPNFV_YARDSTICK_TCO38 - Latency, CPU Load, Throughput, Packet Loss Extended
 367     Test
 368
 369   * OPNFV_YARDSTICK_TCO42 - Network Performance
 370
 371   * OPNFV_YARDSTICK_TCO43 - Network Latency Between NFVI Nodes
 372
 373   * OPNFV_YARDSTICK_TCO44 - Memory Utilization
 374
 375   * OPNFV_YARDSTICK_TCO55 - Compute Capacity
 376
 377   * OPNFV_YARDSTICK_TCO61 - Network Utilization
 378
 379   * OPNFV_YARDSTICK_TCO63 - Storage Capacity
 380
 381   * OPNFV_YARDSTICK_TCO69 - Memory Bandwidth
 382
 383   * OPNFV_YARDSTICK_TCO70 - Latency, Memory Utilization, Throughput, Packet
 384     Loss
 385
 386   * OPNFV_YARDSTICK_TCO71 - Latency, Cache Utilization, Throughput, Packet Loss
 387
 388   * OPNFV_YARDSTICK_TCO72 - Latency, Network Utilization, Throughput, Packet
 389     Loss
 390
 391   * OPNFV_YARDSTICK_TC073 - Network Latency and Throughput Between Nodes
 392
 393   * OPNFV_YARDSTICK_TCO75 - Network Capacity and Scale
 394
 395 * Test Cases for OPNFV HA Project:
 396
 397   * OPNFV_YARDSTICK_TCO19 - HA: Control node Openstack service down
 398
 399   * OPNFV_YARDSTICK_TC025 - HA: OpenStacK Controller Node abnormally down
 400
 401   * OPNFV_YARDSTICK_TCO45 - HA: Control node Openstack service down - neutron
 402     server
 403
 404   * OPNFV_YARDSTICK_TC046 - HA: Control node Openstack service down - keystone
 405
 406   * OPNFV_YARDSTICK_TCO47 - HA: Control node Openstack service down - glance
 407     api
 408
 409   * OPNFV_YARDSTICK_TC048 - HA: Control node Openstack service down - cinder
 410     api
 411
 412   * OPNFV_YARDSTICK_TCO49 - HA: Control node Openstack service down - swift
 413     proxy
 414
 415   * OPNFV_YARDSTICK_TC050 - HA: OpenStack Controller Node Network High
 416     Availability
 417
 418   * OPNFV_YARDSTICK_TCO51 - HA: OpenStack Controller Node CPU Overload High
 419     Availability
 420
 421   * OPNFV_YARDSTICK_TC052 - HA: OpenStack Controller Node Disk I/O Block High
 422     Availability
 423
 424   * OPNFV_YARDSTICK_TCO53 - HA: OpenStack Controller Load Balance Service High
 425     Availability
 426
 427   * OPNFV_YARDSTICK_TC054 - HA: OpenStack Virtual IP High Availability
 428
 429 * Test Case for OPNFV IPv6 Project:
 430
 431   * OPNFV_YARDSTICK_TCO27 - IPv6 connectivity
 432
 433 * Test Case for OPNFV KVM Project:
 434
 435   * OPNFV_YARDSTICK_TCO28 - KVM Latency measurements
 436
 437 * Test Case for OPNFV Parser Project:
 438
 439   * OPNFV_YARDSTICK_TCO40 - Verify Parser Yang-to-Tosca
 440
 441 * Test Case for OPNFV StorPerf Project:
 442
 443   * OPNFV_YARDSTICK_TCO74 - Storperf
 444
 445 * Test Cases for Virtual Traffic Classifier:
 446
 447   * OPNFV_YARDSTICK_TC006 - Virtual Traffic Classifier Data Plane Throughput
 448 Benchmarking Test
 449
 450   * OPNFV_YARDSTICK_TC007 - Virtual Traffic Classifier Data Plane Throughput
 451 Benchmarking in presence of noisy neighbors Test
 452
 453   * OPNFV_YARDSTICK_TC020 - Virtual Traffic Classifier Instantiation Test
 454
 455   * OPNFV_YARDSTICK_TC021 - Virtual Traffic Classifier Instantiation in
 456 presence of noisy neighbors Test
 457
 458
 459 Version Change
 460 ==============
 461
 462 Module Version Changes
 463 ----------------------
 464
 465 This is the third tracked release of Yardstick. It is based on following
 466 upstream versions:
 467
 468 - ONOS Goldeneye
 469
 470 - OpenStack Mitaka
 471
 472 - OpenDaylight Beryllium
 473
 474
 475 Document Version Changes
 476 ------------------------
 477
 478 This is the first tracked version of the Yardstick framework in OPNFV.
 479 It includes the following documentation updates:
 480
 481 - Yardstick User Guide: added yardstick plugin chapter; added Store Other
 482 Project's Test Results in InfluxDB chapter; Refine yardstick instantion chapter.
 483
 484 - Yardstick Code Documentation: no changes
 485
 486 - Yardstick Release Notes for Yardstick: this document
 487
 488 - Test Results report for Colorado testing with Yardstick: updated listed of
 489 verified scenarios and limitations
 490
 491
 492 Feature additions
 493 -----------------
 494  - Yardstick plugin
 495
 496
 497 Scenario Matrix
 498 ===============
 499
 500 For Colorado 1.0, Yardstick was tested on the following scenarios:
 501
 502 +-------------------------+---------+---------+---------+---------+
 503 |         Scenario        |  Apex   | Compass |  Fuel   |   Joid  |
 504 +=========================+=========+=========+=========+=========+
 505 | os-nosdn-nofeature-noha |         |         |         |    X    |
 506 +-------------------------+---------+---------+---------+---------+
 507 | os-nosdn-nofeature-ha   |    X    |         |    X    |    X    |
 508 +-------------------------+---------+---------+---------+---------+
 509 | os-odl_l2-nofeature-ha  |    X    |    X    |    X    |    X    |
 510 +-------------------------+---------+---------+---------+---------+
 511 | os-odl_l2-nofeature-noha|         |    X    |         |         |
 512 +-------------------------+---------+---------+---------+---------+
 513 | os-odl_l3-nofeature-ha  |    X    |         |    X    |         |
 514 +-------------------------+---------+---------+---------+---------+
 515 | os-odl_l3-nofeature-ha  |         |    X    |         |         |
 516 +-------------------------+---------+---------+---------+---------+
 517 | os-onos-sfc-ha          |    X    |         |    X    |    X    |
 518 +-------------------------+---------+---------+---------+---------+
 519 | os-onos-nofeature-ha    |    X    |         |    X    |    X    |
 520 +-------------------------+---------+---------+---------+---------+
 521 | os-onos-nofeature-noha  |         |    X    |         |         |
 522 +-------------------------+---------+---------+---------+---------+
 523 | os-odl_l2-sfc-ha        |         |         |    X    |         |
 524 +-------------------------+---------+---------+---------+---------+
 525 | os-odl_l2-sfc-noha      |    X    |    X    |         |         |
 526 +-------------------------+---------+---------+---------+---------+
 527 | os-odl_l2-bgpvpn-ha     |    X    |         |    X    |         |
 528 +-------------------------+---------+---------+---------+---------+
 529 | os-odl_l2-bgpvpn-noha   |         |    X    |         |         |
 530 +-------------------------+---------+---------+---------+---------+
 531 | os-nosdn-kvm-ha         |         |         |    X    |         |
 532 +-------------------------+---------+---------+---------+---------+
 533 | os-nosdn-kvm-noha       |         |    X    |         |         |
 534 +-------------------------+---------+---------+---------+---------+
 535 | os-nosdn-ovs-ha         |         |         |         |         |
 536 +-------------------------+---------+---------+---------+---------+
 537 | os-nosdn-ovs-noha       |    X    |    X    |         |         |
 538 +-------------------------+---------+---------+---------+---------+
 539 | os-ocl-nofeature-ha     |         |         |         |         |
 540 +-------------------------+---------+---------+---------+---------+
 541 | os-nosdn-lxd-ha         |         |         |         |    X    |
 542 +-------------------------+---------+---------+---------+---------+
 543 | os-nosdn-lxd-noha       |         |         |         |    X    |
 544 +-------------------------+---------+---------+---------+---------+
 545 | os-odl_l2-fdio-noha     |    X    |         |         |         |
 546 +-------------------------+---------+---------+---------+---------+
 547
 548
 549 Test results
 550 ============
 551
 552 Test results are available in:
 553
 554  - jenkins logs on CI: https://build.opnfv.org/ci/view/yardstick/
 555
 556 The reporting pages can be found at:
 557
 558  * apex: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-apex.html
 559  * compass: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-compass.html
 560  * fuel: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-fuel.html
 561  * joid: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-joid.html
 562
 563 You can get additional details through test logs on http://artifacts.opnfv.org/.
 564 As no search engine is available on the OPNFV artifact web site you must
 565 retrieve the pod identifier on which the tests have been executed (see
 566 field pod in any of the results) then click on the selected POD and look
 567 for the date of the test you are interested in.
 568
 569
 570 Known Issues/Faults
 571 ------------
 572  - Boot up VM failed in joid-os-nosdn-lxd-ha and joid-os-nosdn-lxd-noha scenarios
 573  - Yardstick CI job timeout in fuel-os-onos-nofeature-ha scenario
 574  - SSH timeout in apex-os-onos-sfc-ha, apex-os-onos-nofeature-ha scenarios
 575  - Floating IP not supported in apex-os-odl_l3-nofeature-ha scenario
 576  - Scp /home/stack/overcloudrc failed in apex-os-nosdn-ovs-noha and apex-os-odl_l2-sfc-noha scenarios
 577
 578 .. note:: The faults not related to *Yardstick* framework, addressing scenarios
 579   which were not fully verified, are listed in the OPNFV installer's release
 580   notes.
 581
 582
 583 Corrected Faults
 584 ----------------
 585 * TODO *
 586
 587
 588 Colorado known restrictions/issues
 589 ==================================
 590 +-----------+-----------+----------------------------------------------+
 591 | Installer | Scenario  |  Issue                                       |
 592 +===========+===========+==============================================+
 593 | any       | *-bgpvpn  | Floating ips not supported. Some Test cases  |
 594 |           |           | related to floating ips are excluded.        |
 595 +-----------+-----------+----------------------------------------------+
 596 | any       | odl_l3-*  | Some test cases related to using floating IP |
 597 |           |           | addresses fail because of a known ODL bug.   |
 598 |           |           | https://jira.opnfv.org/browse/APEX-112       |
 599 +-----------+-----------+----------------------------------------------+
 600 | apex      | *-fdio    | Due to late integration, fdio scenarios'     |
 601 |           |           | test suite file is not provided.             |
 602 +-----------+-----------+----------------------------------------------+
 603 | joid      | *-lxd     | In the LXD scenarios, nova-lxd does not      |
 604 |           |           | support qcow2 Images.                        |
 605 |           |           | https://jira.opnfv.org/browse/YARDSTICK-325  |
 606 +-----------+-----------+----------------------------------------------+
 607
 608
 609 Open JIRA tickets
 610 =================
 611
 612 +------------------+-----------------------------------------------+
 613 |   JIRA           |         Description                           |
 614 +==================+===============================================+
 615 | `YARDSTICK-325`_ |  Add imge format support for LXD scenario     |
 616 |                  |                                               |
 617 +------------------+-----------------------------------------------+
 618
 619
 620 Useful links
 621 ============
 622
 623  - wiki project page: https://wiki.opnfv.org/display/yardstick/Yardstick
 624
 625  - wiki Yardstick Colorado release planing page: https://wiki.opnfv.org/display/yardstick/Yardstick+Colorado+Release+Planning
 626
 627  - wiki Yardstick Colorado release jira page: https://wiki.opnfv.org/display/yardstick/Jira+Yardstick-Colorado
 628
 629  - Yardstick repo: https://git.opnfv.org/cgit/yardstick
 630
 631  - Yardstick CI dashboard: https://build.opnfv.org/ci/view/yardstick
 632
 633  - Yardstick grafana dashboard: http://testresults.opnfv.org/grafana/
 634
 635  - Yardstick IRC chanel: #opnfv-yardstick
 636
 637 .. _`YARDSTICK-325` : https://jira.opnfv.org/browse/YARDSTICK-325
 638