docs/release/release-notes/release-notes.rst

   1 =======
   2 License
   3 =======
   4
   5 OPNFV Danube release note for Yardstick Docs
   6 are licensed under a Creative Commons Attribution 4.0 International License.
   7 You should have received a copy of the license along with this.
   8 If not, see <http://creativecommons.org/licenses/by/4.0/>.
   9
  10 The *Yardstick framework*, the *Yardstick test cases* are opensource software,
  11  licensed under the terms of the Apache License, Version 2.0.
  12
  13 =========================================
  14 OPNFV Danube Release Note for Yardstick
  15 =========================================
  16
  17 .. toctree::
  18    :maxdepth: 2
  19
  20 .. _Yardstick: https://wiki.opnfv.org/yardstick
  21
  22 .. _Dashboard: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
  23
  24 .. _NFV-TST001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf
  25
  26
  27 Abstract
  28 ========
  29
  30 This document describes the release note of Yardstick project.
  31
  32
  33 Version History
  34 ===============
  35
  36 +----------------+--------------------+---------------------------------+
  37 | *Date*         | *Version*          | *Comment*                       |
  38 |                |                    |                                 |
  39 +----------------+--------------------+---------------------------------+
  40 |                |  3.2               | Yardstick for Danube release    |
  41 |                |                    |                                 |
  42 |                |                    | Note: The 3.2 tag is due to a   |
  43 |                |                    | code issue during Danube 3.1    |
  44 |                |                    | release                         |
  45 |                |                    |                                 |
  46 +----------------+--------------------+---------------------------------+
  47 | May 4th, 2017  |  2.0               | Yardstick for Danube release    |
  48 |                |                    |                                 |
  49 +----------------+--------------------+---------------------------------+
  50 | Mar 31st, 2017 |  1.0               | Yardstick for Danube release    |
  51 |                |                    |                                 |
  52 +----------------+--------------------+---------------------------------+
  53
  54
  55 Important Notes
  56 ===============
  57
  58 The software delivered in the OPNFV Yardstick_ Project, comprising the
  59 *Yardstick framework*, the *Yardstick test cases* and the experimental
  60 framework *Apex Lake* is a realization of the methodology in ETSI-ISG
  61 NFV-TST001_.
  62
  63 The *Yardstick* framework is *installer*, *infrastructure* and *application*
  64 independent.
  65
  66
  67 OPNFV Danube Release
  68 ======================
  69
  70 This Danube release provides *Yardstick* as a framework for NFVI testing
  71 and OPNFV feature testing, automated in the OPNFV CI pipeline, including:
  72
  73 * Documentation generated with Sphinx
  74
  75   * User Guide
  76
  77   * Developer Guide
  78
  79   * Release notes (this document)
  80
  81   * Results
  82
  83 * Automated Yardstick test suite (daily, weekly)
  84
  85   * Jenkins Jobs for OPNFV community labs
  86
  87 * Automated Yardstick test results visualization
  88
  89   * Dashboard_ using Grafana (user:opnfv/password: opnfv), influxDB is used as
  90     backend
  91
  92 * Yardstick framework source code
  93
  94 * Yardstick test cases yaml files
  95
  96 * Yardstick plug-in configration yaml files, plug-in install/remove scripts
  97
  98 For Danube release, the *Yardstick framework* is used for the following
  99 testing:
 100
 101 * OPNFV platform testing - generic test cases to measure the categories:
 102
 103   * Compute
 104
 105   * Network
 106
 107   * Storage
 108
 109 * OPNFV platform network service benchmarking(NSB)
 110
 111   * NSB
 112
 113 * Test cases for the following OPNFV Projects:
 114
 115   * High Availability
 116
 117   * IPv6
 118
 119   * KVM
 120
 121   * Parser
 122
 123   * StorPerf
 124
 125   * VSperf
 126
 127   * virtual Traffic Classifier
 128
 129 The *Yardstick framework* is developed in the OPNFV community, by the
 130 Yardstick_ team. The *virtual Traffic Classifier* is a part of the Yardstick
 131 Project.
 132
 133 .. note:: The test case description template used for the Yardstick test cases
 134   is based on the document ETSI-ISG NFV-TST001_; the results report template
 135   used for the Yardstick results is based on the IEEE Std 829-2008.
 136
 137
 138 Release Data
 139 ============
 140
 141 +--------------------------------------+--------------------------------------+
 142 | **Project**                          | Yardstick                            |
 143 |                                      |                                      |
 144 +--------------------------------------+--------------------------------------+
 145 | **Repo/tag**                         | yardstick/Danube.3.2                 |
 146 |                                      |                                      |
 147 +--------------------------------------+--------------------------------------+
 148 | **Yardstick Docker image tag**       | Danube.3.2                           |
 149 |                                      |                                      |
 150 +--------------------------------------+--------------------------------------+
 151 | **Release designation**              | Danube                               |
 152 |                                      |                                      |
 153 +--------------------------------------+--------------------------------------+
 154 | **Release date**                     | August 15th, 2017                    |
 155 |                                      |                                      |
 156 +--------------------------------------+--------------------------------------+
 157 | **Purpose of the delivery**          | OPNFV Danube release 3.0             |
 158 |                                      |                                      |
 159 +--------------------------------------+--------------------------------------+
 160
 161
 162 Deliverables
 163 ============
 164
 165 Documents
 166 ---------
 167
 168  - User Guide: http://docs.opnfv.org/en/stable-danube/submodules/yardstick/docs/testing/user/userguide/index.html
 169
 170  - Developer Guide: http://docs.opnfv.org/en/stable-danube/submodules/yardstick/docs/testing/developer/devguide/index.html
 171
 172
 173 Software Deliverables
 174 ---------------------
 175
 176
 177  - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: danube.3.2)
 178
 179
 180 **Contexts**
 181
 182 +---------------------+-------------------------------------------------------+
 183 | **Context**         | **Description**                                       |
 184 |                     |                                                       |
 185 +---------------------+-------------------------------------------------------+
 186 | *Heat*              | Models orchestration using OpenStack Heat             |
 187 |                     |                                                       |
 188 +---------------------+-------------------------------------------------------+
 189 | *Node*              | Models Baremetal, Controller, Compute                 |
 190 |                     |                                                       |
 191 +---------------------+-------------------------------------------------------+
 192 | *Standalone*        | Models VM running on Non-Managed NFVi                 |
 193 |                     |                                                       |
 194 +---------------------+-------------------------------------------------------+
 195
 196
 197 **Runners**
 198
 199 +---------------------+-------------------------------------------------------+
 200 | **Runner**          | **Description**                                       |
 201 |                     |                                                       |
 202 +---------------------+-------------------------------------------------------+
 203 | *Arithmetic*        | Steps every run arithmetically according to specified |
 204 |                     | input value                                           |
 205 |                     |                                                       |
 206 +---------------------+-------------------------------------------------------+
 207 | *Duration*          | Runs for a specified period of time                   |
 208 |                     |                                                       |
 209 +---------------------+-------------------------------------------------------+
 210 | *Iteration*         | Runs for a specified number of iterations             |
 211 |                     |                                                       |
 212 +---------------------+-------------------------------------------------------+
 213 | *Sequence*          | Selects input value to a scenario from an input file  |
 214 |                     | and runs all entries sequentially                     |
 215 |                     |                                                       |
 216 +---------------------+-------------------------------------------------------+
 217
 218
 219 **Scenarios**
 220
 221 +---------------------+-------------------------------------------------------+
 222 | **Category**        | **Delivered**                                         |
 223 |                     |                                                       |
 224 +---------------------+-------------------------------------------------------+
 225 | *Availability*      | Attacker:                                             |
 226 |                     |                                                       |
 227 |                     | * baremetal, process                                  |
 228 |                     |                                                       |
 229 |                     | HA tools:                                             |
 230 |                     |                                                       |
 231 |                     | * check host, openstack, process, service             |
 232 |                     | * kill process                                        |
 233 |                     | * start/stop service                                  |
 234 |                     |                                                       |
 235 |                     | Monitor:                                              |
 236 |                     |                                                       |
 237 |                     | * command, process                                    |
 238 |                     |                                                       |
 239 +---------------------+-------------------------------------------------------+
 240 | *Compute*           | * cpuload                                             |
 241 |                     |                                                       |
 242 |                     | * cyclictest                                          |
 243 |                     |                                                       |
 244 |                     | * lmbench                                             |
 245 |                     |                                                       |
 246 |                     | * lmbench_cache                                       |
 247 |                     |                                                       |
 248 |                     | * perf                                                |
 249 |                     |                                                       |
 250 |                     | * unixbench                                           |
 251 |                     |                                                       |
 252 |                     | * ramspeed                                            |
 253 |                     |                                                       |
 254 |                     | * cachestat                                           |
 255 |                     |                                                       |
 256 |                     | * memeoryload                                         |
 257 |                     |                                                       |
 258 |                     | * computecapacity                                     |
 259 |                     |                                                       |
 260 +---------------------+-------------------------------------------------------+
 261 | *Networking*        | * iperf3                                              |
 262 |                     |                                                       |
 263 |                     | * netperf                                             |
 264 |                     |                                                       |
 265 |                     | * netperf_node                                        |
 266 |                     |                                                       |
 267 |                     | * ping                                                |
 268 |                     |                                                       |
 269 |                     | * ping6                                               |
 270 |                     |                                                       |
 271 |                     | * pktgen                                              |
 272 |                     |                                                       |
 273 |                     | * sfc                                                 |
 274 |                     |                                                       |
 275 |                     | * sfc with tacker                                     |
 276 |                     |                                                       |
 277 |                     | * vtc instantion validation                           |
 278 |                     |                                                       |
 279 |                     | * vtc instantion validation with noisy neighbors      |
 280 |                     |                                                       |
 281 |                     | * vtc throughput                                      |
 282 |                     |                                                       |
 283 |                     | * vtc throughput in the presence of noisy neighbors   |
 284 |                     |                                                       |
 285 |                     | * networkcapacity                                     |
 286 |                     |                                                       |
 287 |                     | * netutilization                                      |
 288 |                     |                                                       |
 289 |                     | * nstat                                               |
 290 |                     |                                                       |
 291 +---------------------+-------------------------------------------------------+
 292 | *Parser*            | Tosca2Heat                                            |
 293 |                     |                                                       |
 294 +---------------------+-------------------------------------------------------+
 295 | *Storage*           | fio                                                   |
 296 |                     |                                                       |
 297 |                     | storagecapacity                                       |
 298 |                     |                                                       |
 299 +---------------------+-------------------------------------------------------+
 300 | *StorPerf*          | storperf                                              |
 301 |                     |                                                       |
 302 +---------------------+-------------------------------------------------------+
 303 | *NSB*               | vPE thoughput test case                               |
 304 |                     |                                                       |
 305 +---------------------+-------------------------------------------------------+
 306
 307
 308 **Test Results Output**
 309
 310 +-----------------------------+-----------------------------------------------+
 311 | **Dispatcher**              | **Description**                               |
 312 |                             |                                               |
 313 +-----------------------------+-----------------------------------------------+
 314 |  file                       | Log to a file.                                |
 315 |                             |                                               |
 316 +-----------------------------+-----------------------------------------------+
 317 |  http                       | Post data to html.                            |
 318 |                             |                                               |
 319 +-----------------------------+-----------------------------------------------+
 320 |  influxdb                   | Post data to influxDB.                        |
 321 |                             |                                               |
 322 +-----------------------------+-----------------------------------------------+
 323
 324
 325 Delivered Test cases
 326 --------------------
 327
 328 * Generic NFVI test cases
 329
 330   * OPNFV_YARDSTICK_TCOO1 - NW Performance
 331
 332   * OPNFV_YARDSTICK_TCOO2 - NW Latency
 333
 334   * OPNFV_YARDSTICK_TCOO4 - Cache Utilization
 335
 336   * OPNFV_YARDSTICK_TCOO5 - Storage Performance
 337
 338   * OPNFV_YARDSTICK_TCOO8 - Packet Loss Extended Test
 339
 340   * OPNFV_YARDSTICK_TCOO9 - Packet Loss
 341
 342   * OPNFV_YARDSTICK_TCO10 - Memory Latency
 343
 344   * OPNFV_YARDSTICK_TCO11 - Packet Delay Variation Between VMs
 345
 346   * OPNFV_YARDSTICK_TCO12 - Memory Bandwidth
 347
 348   * OPNFV_YARDSTICK_TCO14 - Processing Speed
 349
 350   * OPNFV_YARDSTICK_TCO24 - CPU Load
 351
 352   * OPNFV_YARDSTICK_TCO37 - Latency, CPU Load, Throughput, Packet Loss
 353
 354   * OPNFV_YARDSTICK_TCO38 - Latency, CPU Load, Throughput, Packet Loss Extended
 355     Test
 356
 357   * OPNFV_YARDSTICK_TCO42 - Network Performance
 358
 359   * OPNFV_YARDSTICK_TCO43 - Network Latency Between NFVI Nodes
 360
 361   * OPNFV_YARDSTICK_TCO44 - Memory Utilization
 362
 363   * OPNFV_YARDSTICK_TCO55 - Compute Capacity
 364
 365   * OPNFV_YARDSTICK_TCO61 - Network Utilization
 366
 367   * OPNFV_YARDSTICK_TCO63 - Storage Capacity
 368
 369   * OPNFV_YARDSTICK_TCO69 - Memory Bandwidth
 370
 371   * OPNFV_YARDSTICK_TCO70 - Latency, Memory Utilization, Throughput, Packet
 372     Loss
 373
 374   * OPNFV_YARDSTICK_TCO71 - Latency, Cache Utilization, Throughput, Packet Loss
 375
 376   * OPNFV_YARDSTICK_TCO72 - Latency, Network Utilization, Throughput, Packet
 377     Loss
 378
 379   * OPNFV_YARDSTICK_TC073 - Network Latency and Throughput Between Nodes
 380
 381   * OPNFV_YARDSTICK_TCO75 - Network Capacity and Scale
 382
 383   * OPNFV_YARDSTICK_TCO76 - Network frame error rate
 384
 385 * Test Cases for OPNFV HA Project:
 386
 387   * OPNFV_YARDSTICK_TCO19 - HA: Control node Openstack service down
 388
 389   * OPNFV_YARDSTICK_TC025 - HA: OpenStacK Controller Node abnormally down
 390
 391   * OPNFV_YARDSTICK_TCO45 - HA: Control node Openstack service down - neutron
 392     server
 393
 394   * OPNFV_YARDSTICK_TC046 - HA: Control node Openstack service down - keystone
 395
 396   * OPNFV_YARDSTICK_TCO47 - HA: Control node Openstack service down - glance
 397     api
 398
 399   * OPNFV_YARDSTICK_TC048 - HA: Control node Openstack service down - cinder
 400     api
 401
 402   * OPNFV_YARDSTICK_TCO49 - HA: Control node Openstack service down - swift
 403     proxy
 404
 405   * OPNFV_YARDSTICK_TC050 - HA: OpenStack Controller Node Network High
 406     Availability
 407
 408   * OPNFV_YARDSTICK_TCO51 - HA: OpenStack Controller Node CPU Overload High
 409     Availability
 410
 411   * OPNFV_YARDSTICK_TC052 - HA: OpenStack Controller Node Disk I/O Block High
 412     Availability
 413
 414   * OPNFV_YARDSTICK_TCO53 - HA: OpenStack Controller Load Balance Service High
 415     Availability
 416
 417   * OPNFV_YARDSTICK_TC054 - HA: OpenStack Virtual IP High Availability
 418
 419 * Test Case for OPNFV IPv6 Project:
 420
 421   * OPNFV_YARDSTICK_TCO27 - IPv6 connectivity
 422
 423 * Test Case for OPNFV KVM Project:
 424
 425   * OPNFV_YARDSTICK_TCO28 - KVM Latency measurements
 426
 427 * Test Case for OPNFV Parser Project:
 428
 429   * OPNFV_YARDSTICK_TCO40 - Verify Parser Yang-to-Tosca
 430
 431 * Test Case for OPNFV StorPerf Project:
 432
 433   * OPNFV_YARDSTICK_TCO74 - Storperf
 434
 435 * Test Cases for Virtual Traffic Classifier:
 436
 437   * OPNFV_YARDSTICK_TC006 - Virtual Traffic Classifier Data Plane Throughput
 438
 439 * Benchmarking Test:
 440
 441   * OPNFV_YARDSTICK_TC007 - Virtual Traffic Classifier Data Plane Throughput
 442
 443 * Benchmarking in presence of noisy neighbors Test:
 444
 445   * OPNFV_YARDSTICK_TC020 - Virtual Traffic Classifier Instantiation Test
 446
 447   * OPNFV_YARDSTICK_TC021 - Virtual Traffic Classifier Instantiation in
 448     presence of noisy neighbors Test
 449
 450
 451 Version Change
 452 ==============
 453
 454 Module Version Changes
 455 ----------------------
 456
 457 This is the fourth tracked release of Yardstick. It is based on following
 458 upstream versions:
 459
 460 - ONOS Ibis
 461
 462 - OpenStack Newton
 463
 464 - OpenDaylight Boron
 465
 466
 467 Document Version Changes
 468 ------------------------
 469
 470 This is the fourth tracked version of the Yardstick framework in OPNFV.
 471 It includes the following documentation updates:
 472
 473 - Yardstick User Guide: add "network service benchmarking(NSB)" chapter;
 474   add "Yardstick - NSB Testing -Installation" chapter; add "Yardstick API" chapter;
 475   add "Yardstick user interface" chapter; Update Yardstick installation chapter;
 476
 477 - Yardstick Developer Guide
 478
 479 - Yardstick Release Notes for Yardstick: this document
 480
 481
 482 Feature additions
 483 -----------------
 484
 485 - Yardstick RESTful API support
 486
 487 - Introduce Network service benchmarking
 488
 489 - Introduce stress testing with Bottlenecks team
 490
 491 - Yardstick framework improvement:
 492
 493   - Parellel test cases execution support
 494
 495   - yardstick report CLI
 496
 497   - Node context support openstack configuration via Ansible
 498
 499   - Https support
 500
 501 - Python 3 support
 502
 503
 504 Scenario Matrix
 505 ===============
 506
 507 For Danube 3.0, Yardstick was tested on the following scenarios:
 508
 509 +-------------------------+---------+---------+---------+---------+
 510 |         Scenario        |  Apex   | Compass |  Fuel   |   Joid  |
 511 +=========================+=========+=========+=========+=========+
 512 | os-nosdn-nofeature-noha |         |         |    X    |    X    |
 513 +-------------------------+---------+---------+---------+---------+
 514 | os-nosdn-nofeature-ha   |    X    |    X    |    X    |    X    |
 515 +-------------------------+---------+---------+---------+---------+
 516 | os-odl_l2-nofeature-ha  |         |    X    |    X    |    X    |
 517 +-------------------------+---------+---------+---------+---------+
 518 | os-odl_l2-nofeature-noha|         |         |    X    |         |
 519 +-------------------------+---------+---------+---------+---------+
 520 | os-odl_l3-nofeature-ha  |    X    |    X    |    X    |         |
 521 +-------------------------+---------+---------+---------+---------+
 522 | os-odl_l3-nofeature-noha|         |         |    X    |         |
 523 +-------------------------+---------+---------+---------+---------+
 524 | os-onos-sfc-ha          |         |         |         |         |
 525 +-------------------------+---------+---------+---------+---------+
 526 | os-onos-nofeature-ha    |         |    X    |         |    X    |
 527 +-------------------------+---------+---------+---------+---------+
 528 | os-onos-nofeature-noha  |         |         |         |         |
 529 +-------------------------+---------+---------+---------+---------+
 530 | os-odl_l2-sfc-ha        |         |         |    X    |         |
 531 +-------------------------+---------+---------+---------+---------+
 532 | os-odl_l2-sfc-noha      |         |         |    X    |         |
 533 +-------------------------+---------+---------+---------+---------+
 534 | os-odl_l2-bgpvpn-ha     |    X    |         |    X    |         |
 535 +-------------------------+---------+---------+---------+---------+
 536 | os-odl_l2-bgpvpn-noha   |         |         |    X    |         |
 537 +-------------------------+---------+---------+---------+---------+
 538 | os-nosdn-kvm-ha         |    X    |         |    X    |         |
 539 +-------------------------+---------+---------+---------+---------+
 540 | os-nosdn-kvm-noha       |         |         |    X    |         |
 541 +-------------------------+---------+---------+---------+---------+
 542 | os-nosdn-ovs-ha         |         |         |    X    |         |
 543 +-------------------------+---------+---------+---------+---------+
 544 | os-nosdn-ovs-noha       |         |         |    X    |         |
 545 +-------------------------+---------+---------+---------+---------+
 546 | os-ocl-nofeature-ha     |         |    X    |         |         |
 547 +-------------------------+---------+---------+---------+---------+
 548 | os-nosdn-lxd-ha         |         |         |         |    X    |
 549 +-------------------------+---------+---------+---------+---------+
 550 | os-nosdn-lxd-noha       |         |         |         |    X    |
 551 +-------------------------+---------+---------+---------+---------+
 552 | os-nosdn-fdio-ha        |    X    |         |         |         |
 553 +-------------------------+---------+---------+---------+---------+
 554 | os-odl_l2-fdio-noha     |    X    |         |         |         |
 555 +-------------------------+---------+---------+---------+---------+
 556 | os-odl-gluon-noha       |    X    |         |         |         |
 557 +-------------------------+---------+---------+---------+---------+
 558 | os-nosdn-openo-ha       |         |    X    |         |         |
 559 +-------------------------+---------+---------+---------+---------+
 560 | os-nosdn-kvm_ovs_dpdk   |         |         |    X    |         |
 561 | -noha                   |         |         |         |         |
 562 +-------------------------+---------+---------+---------+---------+
 563 | os-nosdn-kvm_ovs_dpdk-ha|         |         |    X    |         |
 564 +-------------------------+---------+---------+---------+---------+
 565 | os-nosdn-kvm_ovs_dpdk   |         |         |    X    |         |
 566 | _bar-ha                 |         |         |         |         |
 567 +-------------------------+---------+---------+---------+---------+
 568 | os-nosdn-kvm_ovs_dpdk   |         |         |    X    |         |
 569 | _bar-noha               |         |         |         |         |
 570 +-------------------------+---------+---------+---------+---------+
 571 | opnfv_os-ovn-nofeature- |    X    |         |         |         |
 572 | noha_daily              |         |         |         |         |
 573 +-------------------------+---------+---------+---------+---------+
 574
 575 Test results
 576 ============
 577
 578 Test results are available in:
 579
 580  - jenkins logs on CI: https://build.opnfv.org/ci/view/yardstick/
 581
 582 The reporting pages can be found at:
 583
 584  * apex: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-apex.html
 585  * compass: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-compass.html
 586  * fuel: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-fuel.html
 587  * joid: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-joid.html
 588
 589
 590 Known Issues/Faults
 591 ------------
 592
 593  - Floating IP not supported in bgpvpn scenario
 594
 595  - VM instance cannot get floating IP in compass-os-odl_l2-nofeature-ha scenario
 596
 597 .. note:: The faults not related to *Yardstick* framework, addressing scenarios
 598   which were not fully verified, are listed in the OPNFV installer's release
 599   notes.
 600
 601
 602 Corrected Faults
 603 ----------------
 604
 605 Danube.3.2:
 606
 607 +----------------------------+------------------------------------------------+
 608 | **JIRA REFERENCE**         | **DESCRIPTION**                                |
 609 |                            |                                                |
 610 +----------------------------+------------------------------------------------+
 611 | JIRA: YARDSTICK-776        | Bugfix: cannot run task if without             |
 612 |                            | yardstick.conf in danube                       |
 613 +----------------------------+------------------------------------------------+
 614
 615
 616 Danube.3.1:
 617
 618 +----------------------------+------------------------------------------------+
 619 | **JIRA REFERENCE**         | **DESCRIPTION**                                |
 620 |                            |                                                |
 621 +----------------------------+------------------------------------------------+
 622 | JIRA: YARDSTICK-714        | Add yardstick env influxdb/grafana command for |
 623 |                            | CentOS                                         |
 624 +----------------------------+------------------------------------------------+
 625 | JIRA: YARDSTICK-655        | Monitor command in tc019 may not show the      |
 626 |                            | real nova-api service status                   |
 627 +----------------------------+------------------------------------------------+
 628 | JIRA: YARDSTICK-397        | HA testing framework improvement               |
 629 |                            |                                                |
 630 +----------------------------+------------------------------------------------+
 631 | JIRA: YARDSTICK-660        | Improve monitor_process pass criteria          |
 632 |                            |                                                |
 633 +----------------------------+------------------------------------------------+
 634 | JIRA: YARDSTICK-657        | HA monitor_multi bug,                          |
 635 |                            | KeyError: 'max_outage_time'                    |
 636 +----------------------------+------------------------------------------------+
 637 | JIRA: YARDSTICK-647        | TC025 fault_type value is wrong when using     |
 638 |                            | baremetal pod scripts                          |
 639 +----------------------------+------------------------------------------------+
 640 | JIRA: YARDSTICK-659        | Terminate openstack service process using kill |
 641 |                            | command in HA test cases                       |
 642 +----------------------------+------------------------------------------------+
 643 | JIRA: ARMBAND-275          | Yardstick TC005 fails with                     |
 644 |                            | "Cannot map zero-fill pages" error             |
 645 +----------------------------+------------------------------------------------+
 646 | JIRA: YARDSTICK-561        | Bugfix: AttributeError: 'dict' object has no   |
 647 |                            | attribute 'split' if run sample/ping-hot.yaml  |
 648 +----------------------------+------------------------------------------------+
 649 | JIRA: ARMBAND-268          | ERROR No JSON object could be decoded from     |
 650 |                            | LMBENCH in TC010                               |
 651 +----------------------------+------------------------------------------------+
 652 | JIRA: YARDSTICK-680        | storperf test case tc074 do not get results    |
 653 |                            |                                                |
 654 +----------------------------+------------------------------------------------+
 655
 656 Danube.2.0:
 657
 658 +----------------------------+------------------------------------------------+
 659 | **JIRA REFERENCE**         | **DESCRIPTION**                                |
 660 |                            |                                                |
 661 +----------------------------+------------------------------------------------+
 662 | JIRA: YARDSTICK-608        | Set work directory in Yardstick container      |
 663 |                            |                                                |
 664 +----------------------------+------------------------------------------------+
 665 | JIRA: YARDSTICK-616        | Bugfix: https support should adapt insecure    |
 666 |                            | situation                                      |
 667 +----------------------------+------------------------------------------------+
 668 | JIRA: YARDSTICK-620        | Yardstick virtualenv support                   |
 669 |                            |                                                |
 670 +----------------------------+------------------------------------------------+
 671 | JIRA: YARDSTICK-621        | Bugfix: fix query job status in TC074          |
 672 |                            |                                                |
 673 +----------------------------+------------------------------------------------+
 674 | JIRA: YARDSTICK-622        | Bugfix: take test case modification into       |
 675 |                            | effect in load_images.sh                       |
 676 +----------------------------+------------------------------------------------+
 677 | JIRA: YARDSTICK-623        | change openrc file path to                     |
 678 |                            | /etc/yardstick/openstack.creds                 |
 679 +----------------------------+------------------------------------------------+
 680 | JIRA: YARDSTICK-624        | Add opnfv_os-ovn-nofeature-noha_daily test     |
 681 |                            | suite                                          |
 682 +----------------------------+------------------------------------------------+
 683 | JIRA: YARDSTICK-628        | Bugfix: Make tc019 and tc025 accept            |
 684 |                            | --task-args options                            |
 685 +----------------------------+------------------------------------------------+
 686 | JIRA: YARDSTICK-629        | Bugfix: yardstick env prepare cmd do not       |
 687 |                            | support other installer                        |
 688 +----------------------------+------------------------------------------------+
 689 | JIRA: YARDSTICK-632        | Bugfix: KeyError when using http dispatcher    |
 690 |                            |                                                |
 691 +----------------------------+------------------------------------------------+
 692 | JIRA: YARDSTICK-633        | Bugfix: Environment Compatibility Issues in HA |
 693 |                            | Test Cases                                     |
 694 +----------------------------+------------------------------------------------+
 695 | JIRA: YARDSTICK-634        | fix ha issue when run tc050~tc054 in ci        |
 696 |                            |                                                |
 697 +----------------------------+------------------------------------------------+
 698 | JIRA: YARDSTICK-635        | Bugfix: Local Openstack Operation in HA test   |
 699 |                            | frameworks                                     |
 700 +----------------------------+------------------------------------------------+
 701
 702 Danube.1.0:
 703
 704 +----------------------------+------------------------------------------------+
 705 | **JIRA REFERENCE**         | **DESCRIPTION**                                |
 706 |                            |                                                |
 707 +----------------------------+------------------------------------------------+
 708 | JIRA: YARDSTICK-599        | Could not load EntryPoint.parse when using     |
 709 |                            | 'openstack -h'                                 |
 710 +----------------------------+------------------------------------------------+
 711 | JIRA: YARDSTICK-602        | Don't rely on staic ip addresses as they       |
 712 |                            | are dynamic                                    |
 713 +----------------------------+------------------------------------------------+
 714
 715
 716 Danube 3.2 known restrictions/issues
 717 ====================================
 718 +-----------+-----------+----------------------------------------------+
 719 | Installer | Scenario  |  Issue                                       |
 720 +===========+===========+==============================================+
 721 | any       | *-bgpvpn  | Floating ips not supported. Some Test cases  |
 722 |           |           | related to floating ips are excluded.        |
 723 +-----------+-----------+----------------------------------------------+
 724 | any       | odl_l3-*  | Some test cases related to using floating IP |
 725 |           |           | addresses fail because of a known ODL bug.   |
 726 |           |           |                                              |
 727 +-----------+-----------+----------------------------------------------+
 728 | compass   | odl_l2-*  | In some test cases, VM instance will failed  |
 729 |           |           | raising network interfaces.                  |
 730 |           |           |                                              |
 731 +-----------+-----------+----------------------------------------------+
 732
 733
 734 Open JIRA tickets
 735 =================
 736
 737 +----------------------------+------------------------------------------------+
 738 | **JIRA REFERENCE**         | **DESCRIPTION**                                |
 739 |                            |                                                |
 740 +----------------------------+------------------------------------------------+
 741 | JIRA: YARDSTICK-626        | Fio and Lmbench don't work in Ubuntu-arm64     |
 742 |                            | image                                          |
 743 +----------------------------+------------------------------------------------+
 744 | JIRA: YARDSTICK-603        | Timeout waiting for floating ip                |
 745 |                            | (which actually pingable)                      |
 746 +----------------------------+------------------------------------------------+
 747 | JIRA: YARDSTICK-412        | IPv6 test case should add support for newton   |
 748 |                            |                                                |
 749 +----------------------------+------------------------------------------------+
 750
 751
 752 Useful links
 753 ============
 754
 755  - wiki project page: https://wiki.opnfv.org/display/yardstick/Yardstick
 756
 757  - wiki Yardstick Danube release planing page: https://wiki.opnfv.org/display/yardstick/Yardstick+Danube+Release+Planning
 758
 759  - Yardstick repo: https://git.opnfv.org/cgit/yardstick
 760
 761  - Yardstick CI dashboard: https://build.opnfv.org/ci/view/yardstick
 762
 763  - Yardstick grafana dashboard: http://testresults.opnfv.org/grafana/
 764
 765  - Yardstick IRC chanel: #opnfv-yardstick