docs/release/release-notes.rst

   1 =======
   2 License
   3 =======
   4
   5 OPNFV Colorado release note for Yardstick Docs
   6 are licensed under a Creative Commons Attribution 4.0 International License.
   7 You should have received a copy of the license along with this.
   8 If not, see <http://creativecommons.org/licenses/by/4.0/>.
   9
  10 The *Yardstick framework*, the *Yardstick test cases* and the *ApexLake*
  11 experimental framework are opensource software, licensed under the terms of the
  12 Apache License, Version 2.0.
  13
  14 =========================================
  15 OPNFV Colorado Release Note for Yardstick
  16 =========================================
  17
  18 .. toctree::
  19    :maxdepth: 2
  20
  21 .. _Yardstick: https://wiki.opnfv.org/yardstick
  22
  23 .. _Dashboard: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
  24
  25 .. _NFV-TST001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf
  26
  27
  28 Abstract
  29 ========
  30
  31 This document describes the release note of Yardstick project.
  32
  33
  34 Version History
  35 ===============
  36
  37 +----------------+--------------------+---------------------------------+
  38 | *Date*         | *Version*          | *Comment*                       |
  39 |                |                    |                                 |
  40 +----------------+--------------------+---------------------------------+
  41 | Oct 27nd, 2016 |  2.0               | Yardstick for Colorado release  |
  42 |                |                    |                                 |
  43 +----------------+--------------------+---------------------------------+
  44 | Aug 22nd, 2016 |  1.0               | Yardstick for Colorado release  |
  45 |                |                    |                                 |
  46 +----------------+--------------------+---------------------------------+
  47
  48
  49 Important Notes
  50 ===============
  51
  52 The software delivered in the OPNFV Yardstick_ Project, comprising the
  53 *Yardstick framework*, the *Yardstick test cases* and the experimental
  54 framework *Apex Lake* is a realization of the methodology in ETSI-ISG
  55 NFV-TST001_.
  56
  57 The *Yardstick* framework is *installer*, *infrastructure* and *application*
  58 independent.
  59
  60
  61 OPNFV Colorado Release
  62 ======================
  63
  64 This Colorado release provides *Yardstick* as a framework for NFVI testing
  65 and OPNFV feature testing, automated in the OPNFV CI pipeline, including:
  66
  67 * Documentation generated with Sphinx
  68
  69   * User Guide
  70
  71   * Code Documentation
  72
  73   * Release notes (this document)
  74
  75   * Results
  76
  77 * Automated Yardstick test suite (daily, weekly)
  78
  79   * Jenkins Jobs for OPNFV community labs
  80
  81 * Automated Yardstick test results visualization
  82
  83   * Dashboard_ using Grafana (user:opnfv/password: opnfv), influxDB is used as
  84     backend
  85
  86 * Yardstick framework source code
  87
  88 * Yardstick test cases yaml files
  89
  90 * Yardstick pliug-in configration yaml files, plug-in install/remove scripts
  91
  92 For Colorado release, the *Yardstick framework* is used for the following
  93 testing:
  94
  95 * OPNFV platform testing - generic test cases to measure the categories:
  96
  97   * Compute
  98
  99   * Network
 100
 101   * Storage
 102
 103 * Test cases for the following OPNFV Projects:
 104
 105   * High Availability
 106
 107   * IPv6
 108
 109   * KVM
 110
 111   * Parser
 112
 113   * StorPerf
 114
 115   * VSperf
 116
 117   * virtual Traffic Classifier
 118
 119 The *Yardstick framework* is developed in the OPNFV community, by the
 120 Yardstick_ team. The *virtual Traffic Classifier* is a part of the Yardstick
 121 Project.
 122
 123 .. note:: The test case description template used for the Yardstick test cases
 124   is based on the document ETSI-ISG NFV-TST001_; the results report template
 125   used for the Yardstick results is based on the IEEE Std 829-2008.
 126
 127
 128 Release Data
 129 ============
 130
 131 +--------------------------------------+--------------------------------------+
 132 | **Project**                          | Yardstick                            |
 133 |                                      |                                      |
 134 +--------------------------------------+--------------------------------------+
 135 | **Repo/tag**                         | yardstick/colorado.2.0               |
 136 |                                      |                                      |
 137 +--------------------------------------+--------------------------------------+
 138 | **Yardstick Docker image tag**       | colorado.2.0                         |
 139 |                                      |                                      |
 140 +--------------------------------------+--------------------------------------+
 141 | **Release designation**              | Colorado                             |
 142 |                                      |                                      |
 143 +--------------------------------------+--------------------------------------+
 144 | **Release date**                     | October 27 2016                      |
 145 |                                      |                                      |
 146 +--------------------------------------+--------------------------------------+
 147 | **Purpose of the delivery**          | OPNFV Colorado release 2.0           |
 148 |                                      |                                      |
 149 +--------------------------------------+--------------------------------------+
 150
 151
 152 Deliverables
 153 ============
 154
 155 Documents
 156 ---------
 157
 158  - User Guide: http://artifacts.opnfv.org/yardstick/colorado/docs/userguide/index.html
 159
 160  - Test Results: http://artifacts.opnfv.org/yardstick/colorado/docs/results/overview.html
 161
 162
 163 Software Deliverables
 164 ---------------------
 165
 166 **Yardstick framework source code <colorado.2.0>**
 167
 168 +--------------------------------------+--------------------------------------+
 169 | **Project**                          | Yardstick                            |
 170 |                                      |                                      |
 171 +--------------------------------------+--------------------------------------+
 172 | **Repo/tag**                         | yardstick/colorado.2.0               |
 173 |                                      |                                      |
 174 +--------------------------------------+--------------------------------------+
 175 | **Yardstick Docker image tag**       | colorado.2.0                         |
 176 |                                      |                                      |
 177 +--------------------------------------+--------------------------------------+
 178 | **Release designation**              | Colorado                             |
 179 |                                      |                                      |
 180 +--------------------------------------+--------------------------------------+
 181 | **Release date**                     | October 27th, 2016                   |
 182 |                                      |                                      |
 183 +--------------------------------------+--------------------------------------+
 184 | **Purpose of the delivery**          | OPNFV Colorado release               |
 185 |                                      |                                      |
 186 +--------------------------------------+--------------------------------------+
 187
 188
 189 **Contexts**
 190
 191 +---------------------+-------------------------------------------------------+
 192 | **Context**         | **Description**                                       |
 193 |                     |                                                       |
 194 +---------------------+-------------------------------------------------------+
 195 | *Heat*              | Models orchestration using OpenStack Heat             |
 196 |                     |                                                       |
 197 +---------------------+-------------------------------------------------------+
 198 | *Node*              | Models Baremetal, Controller, Compute                 |
 199 |                     |                                                       |
 200 +---------------------+-------------------------------------------------------+
 201
 202
 203 **Runners**
 204
 205 +---------------------+-------------------------------------------------------+
 206 | **Runner**          | **Description**                                       |
 207 |                     |                                                       |
 208 +---------------------+-------------------------------------------------------+
 209 | *Arithmetic*        | Steps every run arithmetically according to specified |
 210 |                     | input value                                           |
 211 |                     |                                                       |
 212 +---------------------+-------------------------------------------------------+
 213 | *Duration*          | Runs for a specified period of time                   |
 214 |                     |                                                       |
 215 +---------------------+-------------------------------------------------------+
 216 | *Iteration*         | Runs for a specified number of iterations             |
 217 |                     |                                                       |
 218 +---------------------+-------------------------------------------------------+
 219 | *Sequence*          | Selects input value to a scenario from an input file  |
 220 |                     | and runs all entries sequentially                     |
 221 |                     |                                                       |
 222 +---------------------+-------------------------------------------------------+
 223
 224
 225 **Scenarios**
 226
 227 +---------------------+-------------------------------------------------------+
 228 | **Category**        | **Delivered**                                         |
 229 |                     |                                                       |
 230 +---------------------+-------------------------------------------------------+
 231 | *Availability*      | Attacker:                                             |
 232 |                     |                                                       |
 233 |                     | * baremetal, process                                  |
 234 |                     |                                                       |
 235 |                     | HA tools:                                             |
 236 |                     |                                                       |
 237 |                     | * check host, openstack, process, service             |
 238 |                     | * kill process                                        |
 239 |                     | * start/stop service                                  |
 240 |                     |                                                       |
 241 |                     | Monitor:                                              |
 242 |                     |                                                       |
 243 |                     | * command, process                                    |
 244 |                     |                                                       |
 245 +---------------------+-------------------------------------------------------+
 246 | *Compute*           | * cpuload                                             |
 247 |                     |                                                       |
 248 |                     | * cyclictest                                          |
 249 |                     |                                                       |
 250 |                     | * lmbench                                             |
 251 |                     |                                                       |
 252 |                     | * lmbench_cache                                       |
 253 |                     |                                                       |
 254 |                     | * perf                                                |
 255 |                     |                                                       |
 256 |                     | * unixbench                                           |
 257 |                     |                                                       |
 258 |                     | * ramspeed                                            |
 259 |                     |                                                       |
 260 |                     | * cachestat                                           |
 261 |                     |                                                       |
 262 |                     | * memeoryload                                         |
 263 |                     |                                                       |
 264 |                     | * computecapacity                                     |
 265 |                     |                                                       |
 266 +---------------------+-------------------------------------------------------+
 267 | *Networking*        | * iperf3                                              |
 268 |                     |                                                       |
 269 |                     | * netperf                                             |
 270 |                     |                                                       |
 271 |                     | * netperf_node                                        |
 272 |                     |                                                       |
 273 |                     | * ping                                                |
 274 |                     |                                                       |
 275 |                     | * ping6                                               |
 276 |                     |                                                       |
 277 |                     | * pktgen                                              |
 278 |                     |                                                       |
 279 |                     | * sfc                                                 |
 280 |                     |                                                       |
 281 |                     | * sfc with tacker                                     |
 282 |                     |                                                       |
 283 |                     | * vtc instantion validation                           |
 284 |                     |                                                       |
 285 |                     | * vtc instantion validation with noisy neighbors      |
 286 |                     |                                                       |
 287 |                     | * vtc throughput                                      |
 288 |                     |                                                       |
 289 |                     | * vtc throughput in the presence of noisy neighbors   |
 290 |                     |                                                       |
 291 |                     | * networkcapacity                                     |
 292 |                     |                                                       |
 293 |                     | * netutilization                                      |
 294 |                     |                                                       |
 295 +---------------------+-------------------------------------------------------+
 296 | *Parser*            | Tosca2Heat                                            |
 297 |                     |                                                       |
 298 +---------------------+-------------------------------------------------------+
 299 | *Storage*           | fio                                                   |
 300 |                     |                                                       |
 301 |                     | storagecapacity                                       |
 302 |                     |                                                       |
 303 +---------------------+-------------------------------------------------------+
 304 | *StorPerf*          | storperf                                              |
 305 |                     |                                                       |
 306 +---------------------+-------------------------------------------------------+
 307
 308
 309 **API to Other Frameworks**
 310
 311 +---------------------+-------------------------------------------------------+
 312 | **Framework**       | **Description**                                       |
 313 |                     |                                                       |
 314 +---------------------+-------------------------------------------------------+
 315 | *ApexLake*          | Experimental framework that enables the user to       |
 316 |                     | validate NFVI from the perspective of a VNF.          |
 317 |                     | A virtual Traffic Classifier is utilized as VNF.      |
 318 |                     | Enables experiments with SR-IOV on Compute Node.      |
 319 |                     |                                                       |
 320 +---------------------+-------------------------------------------------------+
 321
 322
 323 **Test Results Output**
 324
 325 +-----------------------------+-----------------------------------------------+
 326 | **Dispatcher**              | **Description**                               |
 327 |                             |                                               |
 328 +-----------------------------+-----------------------------------------------+
 329 |  file                       | Log to a file.                                |
 330 |                             |                                               |
 331 +-----------------------------+-----------------------------------------------+
 332 |  http                       | Post data to html.                            |
 333 |                             |                                               |
 334 +-----------------------------+-----------------------------------------------+
 335 |  influxdb                   | Post data to influxDB.                        |
 336 |                             |                                               |
 337 +-----------------------------+-----------------------------------------------+
 338
 339
 340 Delivered Test cases
 341 --------------------
 342
 343 * Generic NFVI test cases
 344
 345   * OPNFV_YARDSTICK_TCOO1 - NW Performance
 346
 347   * OPNFV_YARDSTICK_TCOO2 - NW Latency
 348
 349   * OPNFV_YARDSTICK_TCOO4 - Cache Utilization
 350
 351   * OPNFV_YARDSTICK_TCOO5 - Storage Performance
 352
 353   * OPNFV_YARDSTICK_TCOO8 - Packet Loss Extended Test
 354
 355   * OPNFV_YARDSTICK_TCOO9 - Packet Loss
 356
 357   * OPNFV_YARDSTICK_TCO10 - Memory Latency
 358
 359   * OPNFV_YARDSTICK_TCO11 - Packet Delay Variation Between VMs
 360
 361   * OPNFV_YARDSTICK_TCO12 - Memory Bandwidth
 362
 363   * OPNFV_YARDSTICK_TCO14 - Processing Speed
 364
 365   * OPNFV_YARDSTICK_TCO24 - CPU Load
 366
 367   * OPNFV_YARDSTICK_TCO37 - Latency, CPU Load, Throughput, Packet Loss
 368
 369   * OPNFV_YARDSTICK_TCO38 - Latency, CPU Load, Throughput, Packet Loss Extended
 370     Test
 371
 372   * OPNFV_YARDSTICK_TCO42 - Network Performance
 373
 374   * OPNFV_YARDSTICK_TCO43 - Network Latency Between NFVI Nodes
 375
 376   * OPNFV_YARDSTICK_TCO44 - Memory Utilization
 377
 378   * OPNFV_YARDSTICK_TCO55 - Compute Capacity
 379
 380   * OPNFV_YARDSTICK_TCO61 - Network Utilization
 381
 382   * OPNFV_YARDSTICK_TCO63 - Storage Capacity
 383
 384   * OPNFV_YARDSTICK_TCO69 - Memory Bandwidth
 385
 386   * OPNFV_YARDSTICK_TCO70 - Latency, Memory Utilization, Throughput, Packet
 387     Loss
 388
 389   * OPNFV_YARDSTICK_TCO71 - Latency, Cache Utilization, Throughput, Packet Loss
 390
 391   * OPNFV_YARDSTICK_TCO72 - Latency, Network Utilization, Throughput, Packet
 392     Loss
 393
 394   * OPNFV_YARDSTICK_TC073 - Network Latency and Throughput Between Nodes
 395
 396   * OPNFV_YARDSTICK_TCO75 - Network Capacity and Scale
 397
 398 * Test Cases for OPNFV HA Project:
 399
 400   * OPNFV_YARDSTICK_TCO19 - HA: Control node Openstack service down
 401
 402   * OPNFV_YARDSTICK_TC025 - HA: OpenStacK Controller Node abnormally down
 403
 404   * OPNFV_YARDSTICK_TCO45 - HA: Control node Openstack service down - neutron
 405     server
 406
 407   * OPNFV_YARDSTICK_TC046 - HA: Control node Openstack service down - keystone
 408
 409   * OPNFV_YARDSTICK_TCO47 - HA: Control node Openstack service down - glance
 410     api
 411
 412   * OPNFV_YARDSTICK_TC048 - HA: Control node Openstack service down - cinder
 413     api
 414
 415   * OPNFV_YARDSTICK_TCO49 - HA: Control node Openstack service down - swift
 416     proxy
 417
 418   * OPNFV_YARDSTICK_TC050 - HA: OpenStack Controller Node Network High
 419     Availability
 420
 421   * OPNFV_YARDSTICK_TCO51 - HA: OpenStack Controller Node CPU Overload High
 422     Availability
 423
 424   * OPNFV_YARDSTICK_TC052 - HA: OpenStack Controller Node Disk I/O Block High
 425     Availability
 426
 427   * OPNFV_YARDSTICK_TCO53 - HA: OpenStack Controller Load Balance Service High
 428     Availability
 429
 430   * OPNFV_YARDSTICK_TC054 - HA: OpenStack Virtual IP High Availability
 431
 432 * Test Case for OPNFV IPv6 Project:
 433
 434   * OPNFV_YARDSTICK_TCO27 - IPv6 connectivity
 435
 436 * Test Case for OPNFV KVM Project:
 437
 438   * OPNFV_YARDSTICK_TCO28 - KVM Latency measurements
 439
 440 * Test Case for OPNFV Parser Project:
 441
 442   * OPNFV_YARDSTICK_TCO40 - Verify Parser Yang-to-Tosca
 443
 444 * Test Case for OPNFV StorPerf Project:
 445
 446   * OPNFV_YARDSTICK_TCO74 - Storperf
 447
 448 * Test Cases for Virtual Traffic Classifier:
 449
 450   * OPNFV_YARDSTICK_TC006 - Virtual Traffic Classifier Data Plane Throughput
 451 Benchmarking Test
 452
 453   * OPNFV_YARDSTICK_TC007 - Virtual Traffic Classifier Data Plane Throughput
 454 Benchmarking in presence of noisy neighbors Test
 455
 456   * OPNFV_YARDSTICK_TC020 - Virtual Traffic Classifier Instantiation Test
 457
 458   * OPNFV_YARDSTICK_TC021 - Virtual Traffic Classifier Instantiation in
 459 presence of noisy neighbors Test
 460
 461
 462 Version Change
 463 ==============
 464
 465 Module Version Changes
 466 ----------------------
 467
 468 This is the second tracked release of Yardstick. It is based on following
 469 upstream versions:
 470
 471 - ONOS Goldeneye
 472
 473 - OpenStack Mitaka
 474
 475 - OpenDaylight Beryllium
 476
 477
 478 Document Version Changes
 479 ------------------------
 480
 481 This is the second tracked version of the Yardstick framework in OPNFV.
 482 It includes the following documentation updates:
 483
 484 - Yardstick User Guide: added yardstick plugin chapter; added Store Other
 485 Project's Test Results in InfluxDB chapter; Refine yardstick instantion chapter.
 486
 487 - Yardstick Code Documentation: no changes
 488
 489 - Yardstick Release Notes for Yardstick: this document
 490
 491 - Test Results report for Colorado testing with Yardstick: updated listed of
 492 verified scenarios and limitations
 493
 494
 495 Feature additions
 496 -----------------
 497  - Yardstick plugin
 498  - Yardstick reporting
 499  - StorPerf Integration
 500
 501
 502 Scenario Matrix
 503 ===============
 504
 505 For Colorado 2.0, Yardstick was tested on the following scenarios:
 506
 507 +-------------------------+---------+---------+---------+---------+
 508 |         Scenario        |  Apex   | Compass |  Fuel   |   Joid  |
 509 +=========================+=========+=========+=========+=========+
 510 | os-nosdn-nofeature-noha |         |         |         |    X    |
 511 +-------------------------+---------+---------+---------+---------+
 512 | os-nosdn-nofeature-ha   |    X    |         |    X    |    X    |
 513 +-------------------------+---------+---------+---------+---------+
 514 | os-odl_l2-nofeature-ha  |    X    |    X    |    X    |    X    |
 515 +-------------------------+---------+---------+---------+---------+
 516 | os-odl_l2-nofeature-noha|         |    X    |         |         |
 517 +-------------------------+---------+---------+---------+---------+
 518 | os-odl_l3-nofeature-ha  |    X    |         |    X    |         |
 519 +-------------------------+---------+---------+---------+---------+
 520 | os-odl_l3-nofeature-ha  |         |    X    |         |         |
 521 +-------------------------+---------+---------+---------+---------+
 522 | os-onos-sfc-ha          |    X    |         |    X    |    X    |
 523 +-------------------------+---------+---------+---------+---------+
 524 | os-onos-nofeature-ha    |    X    |         |    X    |    X    |
 525 +-------------------------+---------+---------+---------+---------+
 526 | os-onos-nofeature-noha  |         |    X    |         |         |
 527 +-------------------------+---------+---------+---------+---------+
 528 | os-odl_l2-sfc-ha        |         |         |    X    |         |
 529 +-------------------------+---------+---------+---------+---------+
 530 | os-odl_l2-sfc-noha      |    X    |    X    |         |         |
 531 +-------------------------+---------+---------+---------+---------+
 532 | os-odl_l2-bgpvpn-ha     |    X    |         |    X    |         |
 533 +-------------------------+---------+---------+---------+---------+
 534 | os-odl_l2-bgpvpn-noha   |         |    X    |         |         |
 535 +-------------------------+---------+---------+---------+---------+
 536 | os-nosdn-kvm-ha         |         |         |    X    |         |
 537 +-------------------------+---------+---------+---------+---------+
 538 | os-nosdn-kvm-noha       |         |    X    |         |         |
 539 +-------------------------+---------+---------+---------+---------+
 540 | os-nosdn-ovs-ha         |         |         |    X    |         |
 541 +-------------------------+---------+---------+---------+---------+
 542 | os-nosdn-ovs-noha       |    X    |         |    X    |         |
 543 +-------------------------+---------+---------+---------+---------+
 544 | os-ocl-nofeature-ha     |         |         |         |         |
 545 +-------------------------+---------+---------+---------+---------+
 546 | os-nosdn-lxd-ha         |         |         |         |    X    |
 547 +-------------------------+---------+---------+---------+---------+
 548 | os-nosdn-lxd-noha       |         |         |         |    X    |
 549 +-------------------------+---------+---------+---------+---------+
 550 | os-odl_l2-fdio-noha     |    X    |         |         |         |
 551 +-------------------------+---------+---------+---------+---------+
 552
 553
 554 Test results
 555 ============
 556
 557 Test results are available in:
 558
 559  - jenkins logs on CI: https://build.opnfv.org/ci/view/yardstick/
 560
 561 The reporting pages can be found at:
 562
 563  * apex: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-apex.html
 564  * compass: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-compass.html
 565  * fuel: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-fuel.html
 566  * joid: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-joid.html
 567
 568 You can get additional details through test logs on http://artifacts.opnfv.org/.
 569 As no search engine is available on the OPNFV artifact web site you must
 570 retrieve the pod identifier on which the tests have been executed (see
 571 field pod in any of the results) then click on the selected POD and look
 572 for the date of the test you are interested in.
 573
 574
 575 Known Issues/Faults
 576 ------------
 577  - Floating IP not supported in bgpvpn scenario
 578  - Floating IP not supported in apex-os-odl_l3-nofeature-ha scenario
 579
 580 .. note:: The faults not related to *Yardstick* framework, addressing scenarios
 581   which were not fully verified, are listed in the OPNFV installer's release
 582   notes.
 583
 584
 585 Corrected Faults
 586 ----------------
 587
 588 Colorado.2.0:
 589
 590 +----------------------------+------------------------------------------------+
 591 | **JIRA REFERENCE**         | **SLOGAN**                                     |
 592 |                            |                                                |
 593 +----------------------------+------------------------------------------------+
 594 | JIRA: YARDSTICK-325        | Provide raw format yardstick vm image for      |
 595 |                            | nova-lxd scenario.                             |
 596 |                            |                                                |
 597 +----------------------------+------------------------------------------------+
 598 | JIRA: YARDSTICK-358        | tc027 ipv6 test case to de-coupling to the     |
 599 |                            | installers.                                    |
 600 |                            |                                                |
 601 +----------------------------+------------------------------------------------+
 602 | JIRA: YARDSTICK-359        | ipv6 testcase disable port-security on         |
 603 |                            | vRouter.                                       |
 604 |                            |                                                |
 605 +----------------------------+------------------------------------------------+
 606 | JIRA: YARDSTICK-363        | ipv6 testcase to support fuel.                 |
 607 |                            |                                                |
 608 +----------------------------+------------------------------------------------+
 609 | JIRA: YARDSTICK-367        | Add d3 graph presentation to yardstick         |
 610 |                            | reporting.                                     |
 611 |                            |                                                |
 612 +----------------------------+------------------------------------------------+
 613 | JIRA: YARDSTICK-371        | Provide raw format yardstick vm image for      |
 614 |                            | nova-lxd scenario.                             |
 615 |                            |                                                |
 616 +----------------------------+------------------------------------------------+
 617 | JIRA: YARDSTICK-372        | cannot find yardstick-img-dpdk-modify and      |
 618 |                            | yardstick-img-lxd-modify in environment        |
 619 |                            | varibales.                                     |
 620 |                            |                                                |
 621 +----------------------------+------------------------------------------------+
 622
 623
 624 Colorado 2.0 known restrictions/issues
 625 ==================================
 626 +-----------+-----------+----------------------------------------------+
 627 | Installer | Scenario  |  Issue                                       |
 628 +===========+===========+==============================================+
 629 | any       | *-bgpvpn  | Floating ips not supported. Some Test cases  |
 630 |           |           | related to floating ips are excluded.        |
 631 +-----------+-----------+----------------------------------------------+
 632 | any       | odl_l3-*  | Some test cases related to using floating IP |
 633 |           |           | addresses fail because of a known ODL bug.   |
 634 |           |           | https://jira.opnfv.org/browse/APEX-112       |
 635 +-----------+-----------+----------------------------------------------+
 636
 637
 638 Open JIRA tickets
 639 =================
 640
 641
 642 Useful links
 643 ============
 644
 645  - wiki project page: https://wiki.opnfv.org/display/yardstick/Yardstick
 646
 647  - wiki Yardstick Colorado release planing page: https://wiki.opnfv.org/display/yardstick/Yardstick+Colorado+Release+Planning
 648
 649  - wiki Yardstick Colorado release jira page: https://wiki.opnfv.org/display/yardstick/Jira+Yardstick-Colorado
 650
 651  - Yardstick repo: https://git.opnfv.org/cgit/yardstick
 652
 653  - Yardstick CI dashboard: https://build.opnfv.org/ci/view/yardstick
 654
 655  - Yardstick grafana dashboard: http://testresults.opnfv.org/grafana/
 656
 657  - Yardstick IRC chanel: #opnfv-yardstick
 658
 659 .. _`YARDSTICK-325` : https://jira.opnfv.org/browse/YARDSTICK-325
 660
 661 .. _`YARDSTICK-358` : https://jira.opnfv.org/browse/YARDSTICK-358
 662
 663 .. _`YARDSTICK-359` : https://jira.opnfv.org/browse/YARDSTICK-359
 664
 665 .. _`YARDSTICK-363` : https://jira.opnfv.org/browse/YARDSTICK-363
 666
 667 .. _`YARDSTICK-367` : https://jira.opnfv.org/browse/YARDSTICK-367
 668
 669 .. _`YARDSTICK-371` : https://jira.opnfv.org/browse/YARDSTICK-371
 670
 671 .. _`YARDSTICK-372` : https://jira.opnfv.org/browse/YARDSTICK-372