docs/release/release-notes.rst

   1 =======
   2 License
   3 =======
   4
   5 OPNFV Colorado release note for Yardstick Docs
   6 are licensed under a Creative Commons Attribution 4.0 International License.
   7 You should have received a copy of the license along with this.
   8 If not, see <http://creativecommons.org/licenses/by/4.0/>.
   9
  10 The *Yardstick framework*, the *Yardstick test cases* and the *ApexLake*
  11 experimental framework are opensource software, licensed under the terms of the
  12 Apache License, Version 2.0.
  13
  14 =========================================
  15 OPNFV Colorado Release Note for Yardstick
  16 =========================================
  17
  18 .. toctree::
  19    :maxdepth: 2
  20
  21 .. _Yardstick: https://wiki.opnfv.org/yardstick
  22
  23 .. _Dashboard: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
  24
  25 .. _NFV-TST001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf
  26
  27
  28 Abstract
  29 ========
  30
  31 This document describes the release note of Yardstick project.
  32
  33
  34 Version History
  35 ===============
  36
  37 +----------------+--------------------+---------------------------------+
  38 | *Date*         | *Version*          | *Comment*                       |
  39 |                |                    |                                 |
  40 +----------------+--------------------+---------------------------------+
  41 | Dec 5th, 2016  |  3.0               | Yardstick for Colorado release  |
  42 |                |                    |                                 |
  43 +----------------+--------------------+---------------------------------+
  44 | Oct 27th, 2016 |  2.0               | Yardstick for Colorado release  |
  45 |                |                    |                                 |
  46 +----------------+--------------------+---------------------------------+
  47 | Aug 22nd, 2016 |  1.0               | Yardstick for Colorado release  |
  48 |                |                    |                                 |
  49 +----------------+--------------------+---------------------------------+
  50
  51
  52 Important Notes
  53 ===============
  54
  55 The software delivered in the OPNFV Yardstick_ Project, comprising the
  56 *Yardstick framework*, the *Yardstick test cases* and the experimental
  57 framework *Apex Lake* is a realization of the methodology in ETSI-ISG
  58 NFV-TST001_.
  59
  60 The *Yardstick* framework is *installer*, *infrastructure* and *application*
  61 independent.
  62
  63
  64 OPNFV Colorado Release
  65 ======================
  66
  67 This Colorado release provides *Yardstick* as a framework for NFVI testing
  68 and OPNFV feature testing, automated in the OPNFV CI pipeline, including:
  69
  70 * Documentation generated with Sphinx
  71
  72   * User Guide
  73
  74   * Code Documentation
  75
  76   * Release notes (this document)
  77
  78   * Results
  79
  80 * Automated Yardstick test suite (daily, weekly)
  81
  82   * Jenkins Jobs for OPNFV community labs
  83
  84 * Automated Yardstick test results visualization
  85
  86   * Dashboard_ using Grafana (user:opnfv/password: opnfv), influxDB is used as
  87     backend
  88
  89 * Yardstick framework source code
  90
  91 * Yardstick test cases yaml files
  92
  93 * Yardstick pliug-in configration yaml files, plug-in install/remove scripts
  94
  95 For Colorado release, the *Yardstick framework* is used for the following
  96 testing:
  97
  98 * OPNFV platform testing - generic test cases to measure the categories:
  99
 100   * Compute
 101
 102   * Network
 103
 104   * Storage
 105
 106 * Test cases for the following OPNFV Projects:
 107
 108   * High Availability
 109
 110   * IPv6
 111
 112   * KVM
 113
 114   * Parser
 115
 116   * StorPerf
 117
 118   * VSperf
 119
 120   * virtual Traffic Classifier
 121
 122 The *Yardstick framework* is developed in the OPNFV community, by the
 123 Yardstick_ team. The *virtual Traffic Classifier* is a part of the Yardstick
 124 Project.
 125
 126 .. note:: The test case description template used for the Yardstick test cases
 127   is based on the document ETSI-ISG NFV-TST001_; the results report template
 128   used for the Yardstick results is based on the IEEE Std 829-2008.
 129
 130
 131 Release Data
 132 ============
 133
 134 +--------------------------------------+--------------------------------------+
 135 | **Project**                          | Yardstick                            |
 136 |                                      |                                      |
 137 +--------------------------------------+--------------------------------------+
 138 | **Repo/tag**                         | yardstick/colorado.3.0               |
 139 |                                      |                                      |
 140 +--------------------------------------+--------------------------------------+
 141 | **Yardstick Docker image tag**       | colorado.3.0                         |
 142 |                                      |                                      |
 143 +--------------------------------------+--------------------------------------+
 144 | **Release designation**              | Colorado                             |
 145 |                                      |                                      |
 146 +--------------------------------------+--------------------------------------+
 147 | **Release date**                     | December 5th, 2016                   |
 148 |                                      |                                      |
 149 +--------------------------------------+--------------------------------------+
 150 | **Purpose of the delivery**          | OPNFV Colorado release 3.0           |
 151 |                                      |                                      |
 152 +--------------------------------------+--------------------------------------+
 153
 154
 155 Deliverables
 156 ============
 157
 158 Documents
 159 ---------
 160
 161  - User Guide: http://artifacts.opnfv.org/yardstick/colorado/docs/userguide/index.html
 162
 163  - Test Results: http://artifacts.opnfv.org/yardstick/colorado/docs/results/overview.html
 164
 165
 166 Software Deliverables
 167 ---------------------
 168
 169 **Yardstick framework source code <colorado.3.0>**
 170
 171 +--------------------------------------+--------------------------------------+
 172 | **Project**                          | Yardstick                            |
 173 |                                      |                                      |
 174 +--------------------------------------+--------------------------------------+
 175 | **Repo/tag**                         | yardstick/colorado.3.0               |
 176 |                                      |                                      |
 177 +--------------------------------------+--------------------------------------+
 178 | **Yardstick Docker image tag**       | colorado.3.0                         |
 179 |                                      |                                      |
 180 +--------------------------------------+--------------------------------------+
 181 | **Release designation**              | Colorado                             |
 182 |                                      |                                      |
 183 +--------------------------------------+--------------------------------------+
 184 | **Release date**                     | December 5th, 2016                   |
 185 |                                      |                                      |
 186 +--------------------------------------+--------------------------------------+
 187 | **Purpose of the delivery**          | OPNFV Colorado release               |
 188 |                                      |                                      |
 189 +--------------------------------------+--------------------------------------+
 190
 191
 192 **Contexts**
 193
 194 +---------------------+-------------------------------------------------------+
 195 | **Context**         | **Description**                                       |
 196 |                     |                                                       |
 197 +---------------------+-------------------------------------------------------+
 198 | *Heat*              | Models orchestration using OpenStack Heat             |
 199 |                     |                                                       |
 200 +---------------------+-------------------------------------------------------+
 201 | *Node*              | Models Baremetal, Controller, Compute                 |
 202 |                     |                                                       |
 203 +---------------------+-------------------------------------------------------+
 204
 205
 206 **Runners**
 207
 208 +---------------------+-------------------------------------------------------+
 209 | **Runner**          | **Description**                                       |
 210 |                     |                                                       |
 211 +---------------------+-------------------------------------------------------+
 212 | *Arithmetic*        | Steps every run arithmetically according to specified |
 213 |                     | input value                                           |
 214 |                     |                                                       |
 215 +---------------------+-------------------------------------------------------+
 216 | *Duration*          | Runs for a specified period of time                   |
 217 |                     |                                                       |
 218 +---------------------+-------------------------------------------------------+
 219 | *Iteration*         | Runs for a specified number of iterations             |
 220 |                     |                                                       |
 221 +---------------------+-------------------------------------------------------+
 222 | *Sequence*          | Selects input value to a scenario from an input file  |
 223 |                     | and runs all entries sequentially                     |
 224 |                     |                                                       |
 225 +---------------------+-------------------------------------------------------+
 226
 227
 228 **Scenarios**
 229
 230 +---------------------+-------------------------------------------------------+
 231 | **Category**        | **Delivered**                                         |
 232 |                     |                                                       |
 233 +---------------------+-------------------------------------------------------+
 234 | *Availability*      | Attacker:                                             |
 235 |                     |                                                       |
 236 |                     | * baremetal, process                                  |
 237 |                     |                                                       |
 238 |                     | HA tools:                                             |
 239 |                     |                                                       |
 240 |                     | * check host, openstack, process, service             |
 241 |                     | * kill process                                        |
 242 |                     | * start/stop service                                  |
 243 |                     |                                                       |
 244 |                     | Monitor:                                              |
 245 |                     |                                                       |
 246 |                     | * command, process                                    |
 247 |                     |                                                       |
 248 +---------------------+-------------------------------------------------------+
 249 | *Compute*           | * cpuload                                             |
 250 |                     |                                                       |
 251 |                     | * cyclictest                                          |
 252 |                     |                                                       |
 253 |                     | * lmbench                                             |
 254 |                     |                                                       |
 255 |                     | * lmbench_cache                                       |
 256 |                     |                                                       |
 257 |                     | * perf                                                |
 258 |                     |                                                       |
 259 |                     | * unixbench                                           |
 260 |                     |                                                       |
 261 |                     | * ramspeed                                            |
 262 |                     |                                                       |
 263 |                     | * cachestat                                           |
 264 |                     |                                                       |
 265 |                     | * memeoryload                                         |
 266 |                     |                                                       |
 267 |                     | * computecapacity                                     |
 268 |                     |                                                       |
 269 +---------------------+-------------------------------------------------------+
 270 | *Networking*        | * iperf3                                              |
 271 |                     |                                                       |
 272 |                     | * netperf                                             |
 273 |                     |                                                       |
 274 |                     | * netperf_node                                        |
 275 |                     |                                                       |
 276 |                     | * ping                                                |
 277 |                     |                                                       |
 278 |                     | * ping6                                               |
 279 |                     |                                                       |
 280 |                     | * pktgen                                              |
 281 |                     |                                                       |
 282 |                     | * sfc                                                 |
 283 |                     |                                                       |
 284 |                     | * sfc with tacker                                     |
 285 |                     |                                                       |
 286 |                     | * vtc instantion validation                           |
 287 |                     |                                                       |
 288 |                     | * vtc instantion validation with noisy neighbors      |
 289 |                     |                                                       |
 290 |                     | * vtc throughput                                      |
 291 |                     |                                                       |
 292 |                     | * vtc throughput in the presence of noisy neighbors   |
 293 |                     |                                                       |
 294 |                     | * networkcapacity                                     |
 295 |                     |                                                       |
 296 |                     | * netutilization                                      |
 297 |                     |                                                       |
 298 +---------------------+-------------------------------------------------------+
 299 | *Parser*            | Tosca2Heat                                            |
 300 |                     |                                                       |
 301 +---------------------+-------------------------------------------------------+
 302 | *Storage*           | fio                                                   |
 303 |                     |                                                       |
 304 |                     | storagecapacity                                       |
 305 |                     |                                                       |
 306 +---------------------+-------------------------------------------------------+
 307 | *StorPerf*          | storperf                                              |
 308 |                     |                                                       |
 309 +---------------------+-------------------------------------------------------+
 310
 311
 312 **API to Other Frameworks**
 313
 314 +---------------------+-------------------------------------------------------+
 315 | **Framework**       | **Description**                                       |
 316 |                     |                                                       |
 317 +---------------------+-------------------------------------------------------+
 318 | *ApexLake*          | Experimental framework that enables the user to       |
 319 |                     | validate NFVI from the perspective of a VNF.          |
 320 |                     | A virtual Traffic Classifier is utilized as VNF.      |
 321 |                     | Enables experiments with SR-IOV on Compute Node.      |
 322 |                     |                                                       |
 323 +---------------------+-------------------------------------------------------+
 324
 325
 326 **Test Results Output**
 327
 328 +-----------------------------+-----------------------------------------------+
 329 | **Dispatcher**              | **Description**                               |
 330 |                             |                                               |
 331 +-----------------------------+-----------------------------------------------+
 332 |  file                       | Log to a file.                                |
 333 |                             |                                               |
 334 +-----------------------------+-----------------------------------------------+
 335 |  http                       | Post data to html.                            |
 336 |                             |                                               |
 337 +-----------------------------+-----------------------------------------------+
 338 |  influxdb                   | Post data to influxDB.                        |
 339 |                             |                                               |
 340 +-----------------------------+-----------------------------------------------+
 341
 342
 343 Delivered Test cases
 344 --------------------
 345
 346 * Generic NFVI test cases
 347
 348   * OPNFV_YARDSTICK_TCOO1 - NW Performance
 349
 350   * OPNFV_YARDSTICK_TCOO2 - NW Latency
 351
 352   * OPNFV_YARDSTICK_TCOO4 - Cache Utilization
 353
 354   * OPNFV_YARDSTICK_TCOO5 - Storage Performance
 355
 356   * OPNFV_YARDSTICK_TCOO8 - Packet Loss Extended Test
 357
 358   * OPNFV_YARDSTICK_TCOO9 - Packet Loss
 359
 360   * OPNFV_YARDSTICK_TCO10 - Memory Latency
 361
 362   * OPNFV_YARDSTICK_TCO11 - Packet Delay Variation Between VMs
 363
 364   * OPNFV_YARDSTICK_TCO12 - Memory Bandwidth
 365
 366   * OPNFV_YARDSTICK_TCO14 - Processing Speed
 367
 368   * OPNFV_YARDSTICK_TCO24 - CPU Load
 369
 370   * OPNFV_YARDSTICK_TCO37 - Latency, CPU Load, Throughput, Packet Loss
 371
 372   * OPNFV_YARDSTICK_TCO38 - Latency, CPU Load, Throughput, Packet Loss Extended
 373     Test
 374
 375   * OPNFV_YARDSTICK_TCO42 - Network Performance
 376
 377   * OPNFV_YARDSTICK_TCO43 - Network Latency Between NFVI Nodes
 378
 379   * OPNFV_YARDSTICK_TCO44 - Memory Utilization
 380
 381   * OPNFV_YARDSTICK_TCO55 - Compute Capacity
 382
 383   * OPNFV_YARDSTICK_TCO61 - Network Utilization
 384
 385   * OPNFV_YARDSTICK_TCO63 - Storage Capacity
 386
 387   * OPNFV_YARDSTICK_TCO69 - Memory Bandwidth
 388
 389   * OPNFV_YARDSTICK_TCO70 - Latency, Memory Utilization, Throughput, Packet
 390     Loss
 391
 392   * OPNFV_YARDSTICK_TCO71 - Latency, Cache Utilization, Throughput, Packet Loss
 393
 394   * OPNFV_YARDSTICK_TCO72 - Latency, Network Utilization, Throughput, Packet
 395     Loss
 396
 397   * OPNFV_YARDSTICK_TC073 - Network Latency and Throughput Between Nodes
 398
 399   * OPNFV_YARDSTICK_TCO75 - Network Capacity and Scale
 400
 401 * Test Cases for OPNFV HA Project:
 402
 403   * OPNFV_YARDSTICK_TCO19 - HA: Control node Openstack service down
 404
 405   * OPNFV_YARDSTICK_TC025 - HA: OpenStacK Controller Node abnormally down
 406
 407   * OPNFV_YARDSTICK_TCO45 - HA: Control node Openstack service down - neutron
 408     server
 409
 410   * OPNFV_YARDSTICK_TC046 - HA: Control node Openstack service down - keystone
 411
 412   * OPNFV_YARDSTICK_TCO47 - HA: Control node Openstack service down - glance
 413     api
 414
 415   * OPNFV_YARDSTICK_TC048 - HA: Control node Openstack service down - cinder
 416     api
 417
 418   * OPNFV_YARDSTICK_TCO49 - HA: Control node Openstack service down - swift
 419     proxy
 420
 421   * OPNFV_YARDSTICK_TC050 - HA: OpenStack Controller Node Network High
 422     Availability
 423
 424   * OPNFV_YARDSTICK_TCO51 - HA: OpenStack Controller Node CPU Overload High
 425     Availability
 426
 427   * OPNFV_YARDSTICK_TC052 - HA: OpenStack Controller Node Disk I/O Block High
 428     Availability
 429
 430   * OPNFV_YARDSTICK_TCO53 - HA: OpenStack Controller Load Balance Service High
 431     Availability
 432
 433   * OPNFV_YARDSTICK_TC054 - HA: OpenStack Virtual IP High Availability
 434
 435 * Test Case for OPNFV IPv6 Project:
 436
 437   * OPNFV_YARDSTICK_TCO27 - IPv6 connectivity
 438
 439 * Test Case for OPNFV KVM Project:
 440
 441   * OPNFV_YARDSTICK_TCO28 - KVM Latency measurements
 442
 443 * Test Case for OPNFV Parser Project:
 444
 445   * OPNFV_YARDSTICK_TCO40 - Verify Parser Yang-to-Tosca
 446
 447 * Test Case for OPNFV StorPerf Project:
 448
 449   * OPNFV_YARDSTICK_TCO74 - Storperf
 450
 451 * Test Cases for Virtual Traffic Classifier:
 452
 453   * OPNFV_YARDSTICK_TC006 - Virtual Traffic Classifier Data Plane Throughput
 454 Benchmarking Test
 455
 456   * OPNFV_YARDSTICK_TC007 - Virtual Traffic Classifier Data Plane Throughput
 457 Benchmarking in presence of noisy neighbors Test
 458
 459   * OPNFV_YARDSTICK_TC020 - Virtual Traffic Classifier Instantiation Test
 460
 461   * OPNFV_YARDSTICK_TC021 - Virtual Traffic Classifier Instantiation in
 462 presence of noisy neighbors Test
 463
 464
 465 Version Change
 466 ==============
 467
 468 Module Version Changes
 469 ----------------------
 470
 471 This is the second tracked release of Yardstick. It is based on following
 472 upstream versions:
 473
 474 - ONOS Goldeneye
 475
 476 - OpenStack Mitaka
 477
 478 - OpenDaylight Beryllium
 479
 480
 481 Document Version Changes
 482 ------------------------
 483
 484 This is the second tracked version of the Yardstick framework in OPNFV.
 485 It includes the following documentation updates:
 486
 487 - Yardstick User Guide: added yardstick plugin chapter; added Store Other
 488 Project's Test Results in InfluxDB chapter; Refine yardstick instantion chapter.
 489
 490 - Yardstick Code Documentation: no changes
 491
 492 - Yardstick Release Notes for Yardstick: this document
 493
 494 - Test Results report for Colorado testing with Yardstick: updated listed of
 495 verified scenarios and limitations
 496
 497
 498 Feature additions
 499 -----------------
 500  - Yardstick plugin
 501  - Yardstick reporting
 502  - StorPerf Integration
 503
 504
 505 Scenario Matrix
 506 ===============
 507
 508 For Colorado 3.0, Yardstick was tested on the following scenarios:
 509
 510 +-------------------------+---------+---------+---------+---------+
 511 |         Scenario        |  Apex   | Compass |  Fuel   |   Joid  |
 512 +=========================+=========+=========+=========+=========+
 513 | os-nosdn-nofeature-noha |         |         |         |    X    |
 514 +-------------------------+---------+---------+---------+---------+
 515 | os-nosdn-nofeature-ha   |    X    |         |    X    |    X    |
 516 +-------------------------+---------+---------+---------+---------+
 517 | os-odl_l2-nofeature-ha  |    X    |    X    |    X    |    X    |
 518 +-------------------------+---------+---------+---------+---------+
 519 | os-odl_l2-nofeature-noha|         |    X    |         |         |
 520 +-------------------------+---------+---------+---------+---------+
 521 | os-odl_l3-nofeature-ha  |    X    |         |    X    |         |
 522 +-------------------------+---------+---------+---------+---------+
 523 | os-odl_l3-nofeature-ha  |         |    X    |         |         |
 524 +-------------------------+---------+---------+---------+---------+
 525 | os-onos-sfc-ha          |    X    |         |    X    |    X    |
 526 +-------------------------+---------+---------+---------+---------+
 527 | os-onos-nofeature-ha    |    X    |         |    X    |    X    |
 528 +-------------------------+---------+---------+---------+---------+
 529 | os-onos-nofeature-noha  |         |    X    |         |         |
 530 +-------------------------+---------+---------+---------+---------+
 531 | os-odl_l2-sfc-ha        |         |         |    X    |         |
 532 +-------------------------+---------+---------+---------+---------+
 533 | os-odl_l2-sfc-noha      |    X    |    X    |         |         |
 534 +-------------------------+---------+---------+---------+---------+
 535 | os-odl_l2-bgpvpn-ha     |    X    |         |    X    |         |
 536 +-------------------------+---------+---------+---------+---------+
 537 | os-odl_l2-bgpvpn-noha   |         |    X    |         |         |
 538 +-------------------------+---------+---------+---------+---------+
 539 | os-nosdn-kvm-ha         |         |         |    X    |         |
 540 +-------------------------+---------+---------+---------+---------+
 541 | os-nosdn-kvm-noha       |         |    X    |         |         |
 542 +-------------------------+---------+---------+---------+---------+
 543 | os-nosdn-ovs-ha         |         |         |    X    |         |
 544 +-------------------------+---------+---------+---------+---------+
 545 | os-nosdn-ovs-noha       |    X    |         |    X    |         |
 546 +-------------------------+---------+---------+---------+---------+
 547 | os-ocl-nofeature-ha     |         |         |         |         |
 548 +-------------------------+---------+---------+---------+---------+
 549 | os-nosdn-lxd-ha         |         |         |         |    X    |
 550 +-------------------------+---------+---------+---------+---------+
 551 | os-nosdn-lxd-noha       |         |         |         |    X    |
 552 +-------------------------+---------+---------+---------+---------+
 553 | os-odl_l2-fdio-noha     |    X    |         |         |         |
 554 +-------------------------+---------+---------+---------+---------+
 555
 556
 557 Test results
 558 ============
 559
 560 Test results are available in:
 561
 562  - jenkins logs on CI: https://build.opnfv.org/ci/view/yardstick/
 563
 564 The reporting pages can be found at:
 565
 566  * apex: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-apex.html
 567  * compass: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-compass.html
 568  * fuel: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-fuel.html
 569  * joid: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-joid.html
 570
 571 You can get additional details through test logs on http://artifacts.opnfv.org/.
 572 As no search engine is available on the OPNFV artifact web site you must
 573 retrieve the pod identifier on which the tests have been executed (see
 574 field pod in any of the results) then click on the selected POD and look
 575 for the date of the test you are interested in.
 576
 577
 578 Known Issues/Faults
 579 ------------
 580  - Floating IP not supported in bgpvpn scenario
 581  - Floating IP not supported in apex-os-odl_l3-nofeature-ha scenario
 582
 583 .. note:: The faults not related to *Yardstick* framework, addressing scenarios
 584   which were not fully verified, are listed in the OPNFV installer's release
 585   notes.
 586
 587
 588 Corrected Faults
 589 ----------------
 590
 591 Colorado.3.0:
 592
 593 +----------------------------+------------------------------------------------+
 594 | **JIRA REFERENCE**         | **SLOGAN**                                     |
 595 |                            |                                                |
 596 +----------------------------+------------------------------------------------+
 597 | JIRA: YARDSTICK-239        | Define process for working with Yardstick      |
 598 |                            | Grafana dashboard.                             |
 599 |                            |                                                |
 600 +----------------------------+------------------------------------------------+
 601 | JIRA: YARDSTICK-373        | Add os-odl_l2-fdio-ha scenario support.        |
 602 |                            |                                                |
 603 +----------------------------+------------------------------------------------+
 604
 605
 606 Colorado.2.0:
 607
 608 +----------------------------+------------------------------------------------+
 609 | **JIRA REFERENCE**         | **SLOGAN**                                     |
 610 |                            |                                                |
 611 +----------------------------+------------------------------------------------+
 612 | JIRA: YARDSTICK-325        | Provide raw format yardstick vm image for      |
 613 |                            | nova-lxd scenario.                             |
 614 |                            |                                                |
 615 +----------------------------+------------------------------------------------+
 616 | JIRA: YARDSTICK-358        | tc027 ipv6 test case to de-coupling to the     |
 617 |                            | installers.                                    |
 618 |                            |                                                |
 619 +----------------------------+------------------------------------------------+
 620 | JIRA: YARDSTICK-359        | ipv6 testcase disable port-security on         |
 621 |                            | vRouter.                                       |
 622 |                            |                                                |
 623 +----------------------------+------------------------------------------------+
 624 | JIRA: YARDSTICK-363        | ipv6 testcase to support fuel.                 |
 625 |                            |                                                |
 626 +----------------------------+------------------------------------------------+
 627 | JIRA: YARDSTICK-367        | Add d3 graph presentation to yardstick         |
 628 |                            | reporting.                                     |
 629 |                            |                                                |
 630 +----------------------------+------------------------------------------------+
 631 | JIRA: YARDSTICK-371        | Provide raw format yardstick vm image for      |
 632 |                            | nova-lxd scenario.                             |
 633 |                            |                                                |
 634 +----------------------------+------------------------------------------------+
 635 | JIRA: YARDSTICK-372        | cannot find yardstick-img-dpdk-modify and      |
 636 |                            | yardstick-img-lxd-modify in environment        |
 637 |                            | varibales.                                     |
 638 |                            |                                                |
 639 +----------------------------+------------------------------------------------+
 640
 641
 642 Colorado 3.0 known restrictions/issues
 643 ==================================
 644 +-----------+-----------+----------------------------------------------+
 645 | Installer | Scenario  |  Issue                                       |
 646 +===========+===========+==============================================+
 647 | any       | *-bgpvpn  | Floating ips not supported. Some Test cases  |
 648 |           |           | related to floating ips are excluded.        |
 649 +-----------+-----------+----------------------------------------------+
 650 | any       | odl_l3-*  | Some test cases related to using floating IP |
 651 |           |           | addresses fail because of a known ODL bug.   |
 652 |           |           | https://jira.opnfv.org/browse/APEX-112       |
 653 +-----------+-----------+----------------------------------------------+
 654
 655
 656 Open JIRA tickets
 657 =================
 658
 659
 660 Useful links
 661 ============
 662
 663  - wiki project page: https://wiki.opnfv.org/display/yardstick/Yardstick
 664
 665  - wiki Yardstick Colorado release planing page: https://wiki.opnfv.org/display/yardstick/Yardstick+Colorado+Release+Planning
 666
 667  - wiki Yardstick Colorado release jira page: https://wiki.opnfv.org/display/yardstick/Jira+Yardstick-Colorado
 668
 669  - Yardstick repo: https://git.opnfv.org/cgit/yardstick
 670
 671  - Yardstick CI dashboard: https://build.opnfv.org/ci/view/yardstick
 672
 673  - Yardstick grafana dashboard: http://testresults.opnfv.org/grafana/
 674
 675  - Yardstick IRC chanel: #opnfv-yardstick
 676
 677 .. _`YARDSTICK-239` : https://jira.opnfv.org/browse/YARDSTICK-239
 678
 679 .. _`YARDSTICK-325` : https://jira.opnfv.org/browse/YARDSTICK-325
 680
 681 .. _`YARDSTICK-358` : https://jira.opnfv.org/browse/YARDSTICK-358
 682
 683 .. _`YARDSTICK-359` : https://jira.opnfv.org/browse/YARDSTICK-359
 684
 685 .. _`YARDSTICK-363` : https://jira.opnfv.org/browse/YARDSTICK-363
 686
 687 .. _`YARDSTICK-367` : https://jira.opnfv.org/browse/YARDSTICK-367
 688
 689 .. _`YARDSTICK-371` : https://jira.opnfv.org/browse/YARDSTICK-371
 690
 691 .. _`YARDSTICK-372` : https://jira.opnfv.org/browse/YARDSTICK-372
 692
 693 .. _`YARDSTICK-373` : https://jira.opnfv.org/browse/YARDSTICK-373