docs/release/release-notes/release-notes.rst

   1 =======
   2 License
   3 =======
   4
   5 OPNFV Danube release note for Yardstick Docs
   6 are licensed under a Creative Commons Attribution 4.0 International License.
   7 You should have received a copy of the license along with this.
   8 If not, see <http://creativecommons.org/licenses/by/4.0/>.
   9
  10 The *Yardstick framework*, the *Yardstick test cases* and the *ApexLake*
  11 experimental framework are opensource software, licensed under the terms of the
  12 Apache License, Version 2.0.
  13
  14 =========================================
  15 OPNFV Danube Release Note for Yardstick
  16 =========================================
  17
  18 .. toctree::
  19    :maxdepth: 2
  20
  21 .. _Yardstick: https://wiki.opnfv.org/yardstick
  22
  23 .. _Dashboard: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
  24
  25 .. _NFV-TST001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf
  26
  27
  28 Abstract
  29 ========
  30
  31 This document describes the release note of Yardstick project.
  32
  33
  34 Version History
  35 ===============
  36
  37 +----------------+--------------------+---------------------------------+
  38 | *Date*         | *Version*          | *Comment*                       |
  39 |                |                    |                                 |
  40 +----------------+--------------------+---------------------------------+
  41 |                |  3.1               | Yardstick for Danube release    |
  42 |                |                    |                                 |
  43 |                |                    | Note: The 3.1 tag is due to git |
  44 |                |                    | tag issue during Danube 3.0     |
  45 |                |                    | release                         |
  46 |                |                    |                                 |
  47 +----------------+--------------------+---------------------------------+
  48 | May 4th, 2017  |  2.0               | Yardstick for Danube release    |
  49 |                |                    |                                 |
  50 +----------------+--------------------+---------------------------------+
  51 | Mar 31st, 2017 |  1.0               | Yardstick for Danube release    |
  52 |                |                    |                                 |
  53 +----------------+--------------------+---------------------------------+
  54
  55
  56 Important Notes
  57 ===============
  58
  59 The software delivered in the OPNFV Yardstick_ Project, comprising the
  60 *Yardstick framework*, the *Yardstick test cases* and the experimental
  61 framework *Apex Lake* is a realization of the methodology in ETSI-ISG
  62 NFV-TST001_.
  63
  64 The *Yardstick* framework is *installer*, *infrastructure* and *application*
  65 independent.
  66
  67
  68 OPNFV Danube Release
  69 ======================
  70
  71 This Danube release provides *Yardstick* as a framework for NFVI testing
  72 and OPNFV feature testing, automated in the OPNFV CI pipeline, including:
  73
  74 * Documentation generated with Sphinx
  75
  76   * User Guide
  77
  78   * Developer Guide
  79
  80   * Release notes (this document)
  81
  82   * Results
  83
  84 * Automated Yardstick test suite (daily, weekly)
  85
  86   * Jenkins Jobs for OPNFV community labs
  87
  88 * Automated Yardstick test results visualization
  89
  90   * Dashboard_ using Grafana (user:opnfv/password: opnfv), influxDB is used as
  91     backend
  92
  93 * Yardstick framework source code
  94
  95 * Yardstick test cases yaml files
  96
  97 * Yardstick plug-in configration yaml files, plug-in install/remove scripts
  98
  99 For Danube release, the *Yardstick framework* is used for the following
 100 testing:
 101
 102 * OPNFV platform testing - generic test cases to measure the categories:
 103
 104   * Compute
 105
 106   * Network
 107
 108   * Storage
 109
 110 * OPNFV platform network service benchmarking(NSB)
 111
 112   * NSB
 113
 114 * Test cases for the following OPNFV Projects:
 115
 116   * High Availability
 117
 118   * IPv6
 119
 120   * KVM
 121
 122   * Parser
 123
 124   * StorPerf
 125
 126   * VSperf
 127
 128   * virtual Traffic Classifier
 129
 130 The *Yardstick framework* is developed in the OPNFV community, by the
 131 Yardstick_ team. The *virtual Traffic Classifier* is a part of the Yardstick
 132 Project.
 133
 134 .. note:: The test case description template used for the Yardstick test cases
 135   is based on the document ETSI-ISG NFV-TST001_; the results report template
 136   used for the Yardstick results is based on the IEEE Std 829-2008.
 137
 138
 139 Release Data
 140 ============
 141
 142 +--------------------------------------+--------------------------------------+
 143 | **Project**                          | Yardstick                            |
 144 |                                      |                                      |
 145 +--------------------------------------+--------------------------------------+
 146 | **Repo/tag**                         | yardstick/Danube.3.1                 |
 147 |                                      |                                      |
 148 +--------------------------------------+--------------------------------------+
 149 | **Yardstick Docker image tag**       | Danube.3.1                           |
 150 |                                      |                                      |
 151 +--------------------------------------+--------------------------------------+
 152 | **Release designation**              | Danube                               |
 153 |                                      |                                      |
 154 +--------------------------------------+--------------------------------------+
 155 | **Release date**                     | July 14th, 2017                      |
 156 |                                      |                                      |
 157 +--------------------------------------+--------------------------------------+
 158 | **Purpose of the delivery**          | OPNFV Danube release 3.0             |
 159 |                                      |                                      |
 160 +--------------------------------------+--------------------------------------+
 161
 162
 163 Deliverables
 164 ============
 165
 166 Documents
 167 ---------
 168
 169  - User Guide: http://docs.opnfv.org/en/stable-danube/submodules/yardstick/docs/testing/user/userguide/index.html
 170
 171  - Developer Guide: http://docs.opnfv.org/en/stable-danube/submodules/yardstick/docs/testing/developer/devguide/index.html
 172
 173
 174 Software Deliverables
 175 ---------------------
 176
 177
 178  - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: danube.3.1)
 179
 180
 181 **Contexts**
 182
 183 +---------------------+-------------------------------------------------------+
 184 | **Context**         | **Description**                                       |
 185 |                     |                                                       |
 186 +---------------------+-------------------------------------------------------+
 187 | *Heat*              | Models orchestration using OpenStack Heat             |
 188 |                     |                                                       |
 189 +---------------------+-------------------------------------------------------+
 190 | *Node*              | Models Baremetal, Controller, Compute                 |
 191 |                     |                                                       |
 192 +---------------------+-------------------------------------------------------+
 193 | *Standalone*        | Models VM running on Non-Managed NFVi                 |
 194 |                     |                                                       |
 195 +---------------------+-------------------------------------------------------+
 196
 197
 198 **Runners**
 199
 200 +---------------------+-------------------------------------------------------+
 201 | **Runner**          | **Description**                                       |
 202 |                     |                                                       |
 203 +---------------------+-------------------------------------------------------+
 204 | *Arithmetic*        | Steps every run arithmetically according to specified |
 205 |                     | input value                                           |
 206 |                     |                                                       |
 207 +---------------------+-------------------------------------------------------+
 208 | *Duration*          | Runs for a specified period of time                   |
 209 |                     |                                                       |
 210 +---------------------+-------------------------------------------------------+
 211 | *Iteration*         | Runs for a specified number of iterations             |
 212 |                     |                                                       |
 213 +---------------------+-------------------------------------------------------+
 214 | *Sequence*          | Selects input value to a scenario from an input file  |
 215 |                     | and runs all entries sequentially                     |
 216 |                     |                                                       |
 217 +---------------------+-------------------------------------------------------+
 218
 219
 220 **Scenarios**
 221
 222 +---------------------+-------------------------------------------------------+
 223 | **Category**        | **Delivered**                                         |
 224 |                     |                                                       |
 225 +---------------------+-------------------------------------------------------+
 226 | *Availability*      | Attacker:                                             |
 227 |                     |                                                       |
 228 |                     | * baremetal, process                                  |
 229 |                     |                                                       |
 230 |                     | HA tools:                                             |
 231 |                     |                                                       |
 232 |                     | * check host, openstack, process, service             |
 233 |                     | * kill process                                        |
 234 |                     | * start/stop service                                  |
 235 |                     |                                                       |
 236 |                     | Monitor:                                              |
 237 |                     |                                                       |
 238 |                     | * command, process                                    |
 239 |                     |                                                       |
 240 +---------------------+-------------------------------------------------------+
 241 | *Compute*           | * cpuload                                             |
 242 |                     |                                                       |
 243 |                     | * cyclictest                                          |
 244 |                     |                                                       |
 245 |                     | * lmbench                                             |
 246 |                     |                                                       |
 247 |                     | * lmbench_cache                                       |
 248 |                     |                                                       |
 249 |                     | * perf                                                |
 250 |                     |                                                       |
 251 |                     | * unixbench                                           |
 252 |                     |                                                       |
 253 |                     | * ramspeed                                            |
 254 |                     |                                                       |
 255 |                     | * cachestat                                           |
 256 |                     |                                                       |
 257 |                     | * memeoryload                                         |
 258 |                     |                                                       |
 259 |                     | * computecapacity                                     |
 260 |                     |                                                       |
 261 +---------------------+-------------------------------------------------------+
 262 | *Networking*        | * iperf3                                              |
 263 |                     |                                                       |
 264 |                     | * netperf                                             |
 265 |                     |                                                       |
 266 |                     | * netperf_node                                        |
 267 |                     |                                                       |
 268 |                     | * ping                                                |
 269 |                     |                                                       |
 270 |                     | * ping6                                               |
 271 |                     |                                                       |
 272 |                     | * pktgen                                              |
 273 |                     |                                                       |
 274 |                     | * sfc                                                 |
 275 |                     |                                                       |
 276 |                     | * sfc with tacker                                     |
 277 |                     |                                                       |
 278 |                     | * vtc instantion validation                           |
 279 |                     |                                                       |
 280 |                     | * vtc instantion validation with noisy neighbors      |
 281 |                     |                                                       |
 282 |                     | * vtc throughput                                      |
 283 |                     |                                                       |
 284 |                     | * vtc throughput in the presence of noisy neighbors   |
 285 |                     |                                                       |
 286 |                     | * networkcapacity                                     |
 287 |                     |                                                       |
 288 |                     | * netutilization                                      |
 289 |                     |                                                       |
 290 |                     | * nstat                                               |
 291 |                     |                                                       |
 292 +---------------------+-------------------------------------------------------+
 293 | *Parser*            | Tosca2Heat                                            |
 294 |                     |                                                       |
 295 +---------------------+-------------------------------------------------------+
 296 | *Storage*           | fio                                                   |
 297 |                     |                                                       |
 298 |                     | storagecapacity                                       |
 299 |                     |                                                       |
 300 +---------------------+-------------------------------------------------------+
 301 | *StorPerf*          | storperf                                              |
 302 |                     |                                                       |
 303 +---------------------+-------------------------------------------------------+
 304 | *NSB*               | vPE thoughput test case                               |
 305 |                     |                                                       |
 306 +---------------------+-------------------------------------------------------+
 307
 308
 309 **API to Other Frameworks**
 310
 311 +---------------------+-------------------------------------------------------+
 312 | **Framework**       | **Description**                                       |
 313 |                     |                                                       |
 314 +---------------------+-------------------------------------------------------+
 315 | *ApexLake*          | Experimental framework that enables the user to       |
 316 |                     | validate NFVI from the perspective of a VNF.          |
 317 |                     | A virtual Traffic Classifier is utilized as VNF.      |
 318 |                     | Enables experiments with SR-IOV on Compute Node.      |
 319 |                     |                                                       |
 320 +---------------------+-------------------------------------------------------+
 321
 322
 323 **Test Results Output**
 324
 325 +-----------------------------+-----------------------------------------------+
 326 | **Dispatcher**              | **Description**                               |
 327 |                             |                                               |
 328 +-----------------------------+-----------------------------------------------+
 329 |  file                       | Log to a file.                                |
 330 |                             |                                               |
 331 +-----------------------------+-----------------------------------------------+
 332 |  http                       | Post data to html.                            |
 333 |                             |                                               |
 334 +-----------------------------+-----------------------------------------------+
 335 |  influxdb                   | Post data to influxDB.                        |
 336 |                             |                                               |
 337 +-----------------------------+-----------------------------------------------+
 338
 339
 340 Delivered Test cases
 341 --------------------
 342
 343 * Generic NFVI test cases
 344
 345   * OPNFV_YARDSTICK_TCOO1 - NW Performance
 346
 347   * OPNFV_YARDSTICK_TCOO2 - NW Latency
 348
 349   * OPNFV_YARDSTICK_TCOO4 - Cache Utilization
 350
 351   * OPNFV_YARDSTICK_TCOO5 - Storage Performance
 352
 353   * OPNFV_YARDSTICK_TCOO8 - Packet Loss Extended Test
 354
 355   * OPNFV_YARDSTICK_TCOO9 - Packet Loss
 356
 357   * OPNFV_YARDSTICK_TCO10 - Memory Latency
 358
 359   * OPNFV_YARDSTICK_TCO11 - Packet Delay Variation Between VMs
 360
 361   * OPNFV_YARDSTICK_TCO12 - Memory Bandwidth
 362
 363   * OPNFV_YARDSTICK_TCO14 - Processing Speed
 364
 365   * OPNFV_YARDSTICK_TCO24 - CPU Load
 366
 367   * OPNFV_YARDSTICK_TCO37 - Latency, CPU Load, Throughput, Packet Loss
 368
 369   * OPNFV_YARDSTICK_TCO38 - Latency, CPU Load, Throughput, Packet Loss Extended
 370     Test
 371
 372   * OPNFV_YARDSTICK_TCO42 - Network Performance
 373
 374   * OPNFV_YARDSTICK_TCO43 - Network Latency Between NFVI Nodes
 375
 376   * OPNFV_YARDSTICK_TCO44 - Memory Utilization
 377
 378   * OPNFV_YARDSTICK_TCO55 - Compute Capacity
 379
 380   * OPNFV_YARDSTICK_TCO61 - Network Utilization
 381
 382   * OPNFV_YARDSTICK_TCO63 - Storage Capacity
 383
 384   * OPNFV_YARDSTICK_TCO69 - Memory Bandwidth
 385
 386   * OPNFV_YARDSTICK_TCO70 - Latency, Memory Utilization, Throughput, Packet
 387     Loss
 388
 389   * OPNFV_YARDSTICK_TCO71 - Latency, Cache Utilization, Throughput, Packet Loss
 390
 391   * OPNFV_YARDSTICK_TCO72 - Latency, Network Utilization, Throughput, Packet
 392     Loss
 393
 394   * OPNFV_YARDSTICK_TC073 - Network Latency and Throughput Between Nodes
 395
 396   * OPNFV_YARDSTICK_TCO75 - Network Capacity and Scale
 397
 398   * OPNFV_YARDSTICK_TCO76 - Network frame error rate
 399
 400 * Test Cases for OPNFV HA Project:
 401
 402   * OPNFV_YARDSTICK_TCO19 - HA: Control node Openstack service down
 403
 404   * OPNFV_YARDSTICK_TC025 - HA: OpenStacK Controller Node abnormally down
 405
 406   * OPNFV_YARDSTICK_TCO45 - HA: Control node Openstack service down - neutron
 407     server
 408
 409   * OPNFV_YARDSTICK_TC046 - HA: Control node Openstack service down - keystone
 410
 411   * OPNFV_YARDSTICK_TCO47 - HA: Control node Openstack service down - glance
 412     api
 413
 414   * OPNFV_YARDSTICK_TC048 - HA: Control node Openstack service down - cinder
 415     api
 416
 417   * OPNFV_YARDSTICK_TCO49 - HA: Control node Openstack service down - swift
 418     proxy
 419
 420   * OPNFV_YARDSTICK_TC050 - HA: OpenStack Controller Node Network High
 421     Availability
 422
 423   * OPNFV_YARDSTICK_TCO51 - HA: OpenStack Controller Node CPU Overload High
 424     Availability
 425
 426   * OPNFV_YARDSTICK_TC052 - HA: OpenStack Controller Node Disk I/O Block High
 427     Availability
 428
 429   * OPNFV_YARDSTICK_TCO53 - HA: OpenStack Controller Load Balance Service High
 430     Availability
 431
 432   * OPNFV_YARDSTICK_TC054 - HA: OpenStack Virtual IP High Availability
 433
 434 * Test Case for OPNFV IPv6 Project:
 435
 436   * OPNFV_YARDSTICK_TCO27 - IPv6 connectivity
 437
 438 * Test Case for OPNFV KVM Project:
 439
 440   * OPNFV_YARDSTICK_TCO28 - KVM Latency measurements
 441
 442 * Test Case for OPNFV Parser Project:
 443
 444   * OPNFV_YARDSTICK_TCO40 - Verify Parser Yang-to-Tosca
 445
 446 * Test Case for OPNFV StorPerf Project:
 447
 448   * OPNFV_YARDSTICK_TCO74 - Storperf
 449
 450 * Test Cases for Virtual Traffic Classifier:
 451
 452   * OPNFV_YARDSTICK_TC006 - Virtual Traffic Classifier Data Plane Throughput
 453
 454 * Benchmarking Test:
 455
 456   * OPNFV_YARDSTICK_TC007 - Virtual Traffic Classifier Data Plane Throughput
 457
 458 * Benchmarking in presence of noisy neighbors Test:
 459
 460   * OPNFV_YARDSTICK_TC020 - Virtual Traffic Classifier Instantiation Test
 461
 462   * OPNFV_YARDSTICK_TC021 - Virtual Traffic Classifier Instantiation in
 463     presence of noisy neighbors Test
 464
 465
 466 Version Change
 467 ==============
 468
 469 Module Version Changes
 470 ----------------------
 471
 472 This is the fourth tracked release of Yardstick. It is based on following
 473 upstream versions:
 474
 475 - ONOS Ibis
 476
 477 - OpenStack Newton
 478
 479 - OpenDaylight Boron
 480
 481
 482 Document Version Changes
 483 ------------------------
 484
 485 This is the fourth tracked version of the Yardstick framework in OPNFV.
 486 It includes the following documentation updates:
 487
 488 - Yardstick User Guide: add "network service benchmarking(NSB)" chapter;
 489   add "Yardstick - NSB Testing -Installation" chapter; add "Yardstick API" chapter;
 490   add "Yardstick user interface" chapter; Update Yardstick installation chapter;
 491
 492 - Yardstick Developer Guide
 493
 494 - Yardstick Release Notes for Yardstick: this document
 495
 496
 497 Feature additions
 498 -----------------
 499
 500 - Yardstick RESTful API support
 501
 502 - Introduce Network service benchmarking
 503
 504 - Introduce stress testing with Bottlenecks team
 505
 506 - Yardstick framework improvement:
 507
 508   - Parellel test cases execution support
 509
 510   - yardstick report CLI
 511
 512   - Node context support openstack configuration via Ansible
 513
 514   - Https support
 515
 516 - Python 3 support
 517
 518
 519 Scenario Matrix
 520 ===============
 521
 522 For Danube 3.0, Yardstick was tested on the following scenarios:
 523
 524 +-------------------------+---------+---------+---------+---------+
 525 |         Scenario        |  Apex   | Compass |  Fuel   |   Joid  |
 526 +=========================+=========+=========+=========+=========+
 527 | os-nosdn-nofeature-noha |         |         |    X    |    X    |
 528 +-------------------------+---------+---------+---------+---------+
 529 | os-nosdn-nofeature-ha   |    X    |    X    |    X    |    X    |
 530 +-------------------------+---------+---------+---------+---------+
 531 | os-odl_l2-nofeature-ha  |         |    X    |    X    |    X    |
 532 +-------------------------+---------+---------+---------+---------+
 533 | os-odl_l2-nofeature-noha|         |         |    X    |         |
 534 +-------------------------+---------+---------+---------+---------+
 535 | os-odl_l3-nofeature-ha  |    X    |    X    |    X    |         |
 536 +-------------------------+---------+---------+---------+---------+
 537 | os-odl_l3-nofeature-noha|         |         |    X    |         |
 538 +-------------------------+---------+---------+---------+---------+
 539 | os-onos-sfc-ha          |         |         |         |         |
 540 +-------------------------+---------+---------+---------+---------+
 541 | os-onos-nofeature-ha    |         |    X    |         |    X    |
 542 +-------------------------+---------+---------+---------+---------+
 543 | os-onos-nofeature-noha  |         |         |         |         |
 544 +-------------------------+---------+---------+---------+---------+
 545 | os-odl_l2-sfc-ha        |         |         |    X    |         |
 546 +-------------------------+---------+---------+---------+---------+
 547 | os-odl_l2-sfc-noha      |         |         |    X    |         |
 548 +-------------------------+---------+---------+---------+---------+
 549 | os-odl_l2-bgpvpn-ha     |    X    |         |    X    |         |
 550 +-------------------------+---------+---------+---------+---------+
 551 | os-odl_l2-bgpvpn-noha   |         |         |    X    |         |
 552 +-------------------------+---------+---------+---------+---------+
 553 | os-nosdn-kvm-ha         |    X    |         |    X    |         |
 554 +-------------------------+---------+---------+---------+---------+
 555 | os-nosdn-kvm-noha       |         |         |    X    |         |
 556 +-------------------------+---------+---------+---------+---------+
 557 | os-nosdn-ovs-ha         |         |         |    X    |         |
 558 +-------------------------+---------+---------+---------+---------+
 559 | os-nosdn-ovs-noha       |         |         |    X    |         |
 560 +-------------------------+---------+---------+---------+---------+
 561 | os-ocl-nofeature-ha     |         |    X    |         |         |
 562 +-------------------------+---------+---------+---------+---------+
 563 | os-nosdn-lxd-ha         |         |         |         |    X    |
 564 +-------------------------+---------+---------+---------+---------+
 565 | os-nosdn-lxd-noha       |         |         |         |    X    |
 566 +-------------------------+---------+---------+---------+---------+
 567 | os-nosdn-fdio-ha        |    X    |         |         |         |
 568 +-------------------------+---------+---------+---------+---------+
 569 | os-odl_l2-fdio-noha     |    X    |         |         |         |
 570 +-------------------------+---------+---------+---------+---------+
 571 | os-odl-gluon-noha       |    X    |         |         |         |
 572 +-------------------------+---------+---------+---------+---------+
 573 | os-nosdn-openo-ha       |         |    X    |         |         |
 574 +-------------------------+---------+---------+---------+---------+
 575 | os-nosdn-kvm_ovs_dpdk   |         |         |    X    |         |
 576 | -noha                   |         |         |         |         |
 577 +-------------------------+---------+---------+---------+---------+
 578 | os-nosdn-kvm_ovs_dpdk-ha|         |         |    X    |         |
 579 +-------------------------+---------+---------+---------+---------+
 580 | os-nosdn-kvm_ovs_dpdk   |         |         |    X    |         |
 581 | _bar-ha                 |         |         |         |         |
 582 +-------------------------+---------+---------+---------+---------+
 583 | os-nosdn-kvm_ovs_dpdk   |         |         |    X    |         |
 584 | _bar-noha               |         |         |         |         |
 585 +-------------------------+---------+---------+---------+---------+
 586 | opnfv_os-ovn-nofeature- |    X    |         |         |         |
 587 | noha_daily              |         |         |         |         |
 588 +-------------------------+---------+---------+---------+---------+
 589
 590 Test results
 591 ============
 592
 593 Test results are available in:
 594
 595  - jenkins logs on CI: https://build.opnfv.org/ci/view/yardstick/
 596
 597 The reporting pages can be found at:
 598
 599  * apex: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-apex.html
 600  * compass: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-compass.html
 601  * fuel: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-fuel.html
 602  * joid: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-joid.html
 603
 604
 605 Known Issues/Faults
 606 ------------
 607
 608  - Floating IP not supported in bgpvpn scenario
 609
 610  - VM instance cannot get floating IP in compass-os-odl_l2-nofeature-ha scenario
 611
 612 .. note:: The faults not related to *Yardstick* framework, addressing scenarios
 613   which were not fully verified, are listed in the OPNFV installer's release
 614   notes.
 615
 616
 617 Corrected Faults
 618 ----------------
 619
 620 Danube.3.1:
 621
 622 +----------------------------+------------------------------------------------+
 623 | **JIRA REFERENCE**         | **DESCRIPTION**                                |
 624 |                            |                                                |
 625 +----------------------------+------------------------------------------------+
 626 | JIRA: YARDSTICK-714        | Add yardstick env influxdb/grafana command for |
 627 |                            | CentOS                                         |
 628 +----------------------------+------------------------------------------------+
 629 | JIRA: YARDSTICK-655        | Monitor command in tc019 may not show the      |
 630 |                            | real nova-api service status                   |
 631 +----------------------------+------------------------------------------------+
 632 | JIRA: YARDSTICK-397        | HA testing framework improvement               |
 633 |                            |                                                |
 634 +----------------------------+------------------------------------------------+
 635 | JIRA: YARDSTICK-660        | Improve monitor_process pass criteria          |
 636 |                            |                                                |
 637 +----------------------------+------------------------------------------------+
 638 | JIRA: YARDSTICK-657        | HA monitor_multi bug,                          |
 639 |                            | KeyError: 'max_outage_time'                    |
 640 +----------------------------+------------------------------------------------+
 641 | JIRA: YARDSTICK-647        | TC025 fault_type value is wrong when using     |
 642 |                            | baremetal pod scripts                          |
 643 +----------------------------+------------------------------------------------+
 644 | JIRA: YARDSTICK-659        | Terminate openstack service process using kill |
 645 |                            | command in HA test cases                       |
 646 +----------------------------+------------------------------------------------+
 647 | JIRA: ARMBAND-275          | Yardstick TC005 fails with                     |
 648 |                            | "Cannot map zero-fill pages" error             |
 649 +----------------------------+------------------------------------------------+
 650 | JIRA: YARDSTICK-561        | Bugfix: AttributeError: 'dict' object has no   |
 651 |                            | attribute 'split' if run sample/ping-hot.yaml  |
 652 +----------------------------+------------------------------------------------+
 653 | JIRA: ARMBAND-268          | ERROR No JSON object could be decoded from     |
 654 |                            | LMBENCH in TC010                               |
 655 +----------------------------+------------------------------------------------+
 656 | JIRA: YARDSTICK-680        | storperf test case tc074 do not get results    |
 657 |                            |                                                |
 658 +----------------------------+------------------------------------------------+
 659
 660 Danube.2.0:
 661
 662 +----------------------------+------------------------------------------------+
 663 | **JIRA REFERENCE**         | **DESCRIPTION**                                |
 664 |                            |                                                |
 665 +----------------------------+------------------------------------------------+
 666 | JIRA: YARDSTICK-608        | Set work directory in Yardstick container      |
 667 |                            |                                                |
 668 +----------------------------+------------------------------------------------+
 669 | JIRA: YARDSTICK-616        | Bugfix: https support should adapt insecure    |
 670 |                            | situation                                      |
 671 +----------------------------+------------------------------------------------+
 672 | JIRA: YARDSTICK-620        | Yardstick virtualenv support                   |
 673 |                            |                                                |
 674 +----------------------------+------------------------------------------------+
 675 | JIRA: YARDSTICK-621        | Bugfix: fix query job status in TC074          |
 676 |                            |                                                |
 677 +----------------------------+------------------------------------------------+
 678 | JIRA: YARDSTICK-622        | Bugfix: take test case modification into       |
 679 |                            | effect in load_images.sh                       |
 680 +----------------------------+------------------------------------------------+
 681 | JIRA: YARDSTICK-623        | change openrc file path to                     |
 682 |                            | /etc/yardstick/openstack.creds                 |
 683 +----------------------------+------------------------------------------------+
 684 | JIRA: YARDSTICK-624        | Add opnfv_os-ovn-nofeature-noha_daily test     |
 685 |                            | suite                                          |
 686 +----------------------------+------------------------------------------------+
 687 | JIRA: YARDSTICK-628        | Bugfix: Make tc019 and tc025 accept            |
 688 |                            | --task-args options                            |
 689 +----------------------------+------------------------------------------------+
 690 | JIRA: YARDSTICK-629        | Bugfix: yardstick env prepare cmd do not       |
 691 |                            | support other installer                        |
 692 +----------------------------+------------------------------------------------+
 693 | JIRA: YARDSTICK-632        | Bugfix: KeyError when using http dispatcher    |
 694 |                            |                                                |
 695 +----------------------------+------------------------------------------------+
 696 | JIRA: YARDSTICK-633        | Bugfix: Environment Compatibility Issues in HA |
 697 |                            | Test Cases                                     |
 698 +----------------------------+------------------------------------------------+
 699 | JIRA: YARDSTICK-634        | fix ha issue when run tc050~tc054 in ci        |
 700 |                            |                                                |
 701 +----------------------------+------------------------------------------------+
 702 | JIRA: YARDSTICK-635        | Bugfix: Local Openstack Operation in HA test   |
 703 |                            | frameworks                                     |
 704 +----------------------------+------------------------------------------------+
 705
 706 Danube.1.0:
 707
 708 +----------------------------+------------------------------------------------+
 709 | **JIRA REFERENCE**         | **DESCRIPTION**                                |
 710 |                            |                                                |
 711 +----------------------------+------------------------------------------------+
 712 | JIRA: YARDSTICK-599        | Could not load EntryPoint.parse when using     |
 713 |                            | 'openstack -h'                                 |
 714 +----------------------------+------------------------------------------------+
 715 | JIRA: YARDSTICK-602        | Don't rely on staic ip addresses as they       |
 716 |                            | are dynamic                                    |
 717 +----------------------------+------------------------------------------------+
 718
 719
 720 Danube 3.1 known restrictions/issues
 721 ====================================
 722 +-----------+-----------+----------------------------------------------+
 723 | Installer | Scenario  |  Issue                                       |
 724 +===========+===========+==============================================+
 725 | any       | *-bgpvpn  | Floating ips not supported. Some Test cases  |
 726 |           |           | related to floating ips are excluded.        |
 727 +-----------+-----------+----------------------------------------------+
 728 | any       | odl_l3-*  | Some test cases related to using floating IP |
 729 |           |           | addresses fail because of a known ODL bug.   |
 730 |           |           |                                              |
 731 +-----------+-----------+----------------------------------------------+
 732 | compass   | odl_l2-*  | In some test cases, VM instance will failed  |
 733 |           |           | raising network interfaces.                  |
 734 |           |           |                                              |
 735 +-----------+-----------+----------------------------------------------+
 736
 737
 738 Open JIRA tickets
 739 =================
 740
 741 +----------------------------+------------------------------------------------+
 742 | **JIRA REFERENCE**         | **DESCRIPTION**                                |
 743 |                            |                                                |
 744 +----------------------------+------------------------------------------------+
 745 | JIRA: YARDSTICK-626        | Fio and Lmbench don't work in Ubuntu-arm64     |
 746 |                            | image                                          |
 747 +----------------------------+------------------------------------------------+
 748 | JIRA: YARDSTICK-603        | Timeout waiting for floating ip                |
 749 |                            | (which actually pingable)                      |
 750 +----------------------------+------------------------------------------------+
 751 | JIRA: YARDSTICK-412        | IPv6 test case should add support for newton   |
 752 |                            |                                                |
 753 +----------------------------+------------------------------------------------+
 754
 755
 756 Useful links
 757 ============
 758
 759  - wiki project page: https://wiki.opnfv.org/display/yardstick/Yardstick
 760
 761  - wiki Yardstick Danube release planing page: https://wiki.opnfv.org/display/yardstick/Yardstick+Danube+Release+Planning
 762
 763  - Yardstick repo: https://git.opnfv.org/cgit/yardstick
 764
 765  - Yardstick CI dashboard: https://build.opnfv.org/ci/view/yardstick
 766
 767  - Yardstick grafana dashboard: http://testresults.opnfv.org/grafana/
 768
 769  - Yardstick IRC chanel: #opnfv-yardstick