X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=blobdiff_plain;f=docs%2Frelease%2Frelease-notes%2Frelease_notes.rst;h=b52830ec614329c433cc31dbca7e459920f9bef3;hb=66fbd5fcfafd5dcfde58a61aab33c1812405e097;hp=dd08a0e41f0caf5510234d44e52d06a254d5c644;hpb=ec58becb7fe95a1d6fbc651a7a2032c3948659d8;p=bottlenecks.git diff --git a/docs/release/release-notes/release_notes.rst b/docs/release/release-notes/release_notes.rst index dd08a0e4..b52830ec 100644 --- a/docs/release/release-notes/release_notes.rst +++ b/docs/release/release-notes/release_notes.rst @@ -31,19 +31,34 @@ Apache License, Version 2.0. Version History =============== -+----------------+--------------------+---------------------------------+ -| *Date* | *Version* | *Comment* | -| | | | -+----------------+--------------------+---------------------------------+ -| Sept 22nd, 2016| 1.0 | Bottlenecks Colorado release 1.0| -| | | | -+----------------+--------------------+---------------------------------+ -| Feb 17nd, 2017 | 1.1 | Bottlenecks Danube release 1.0 | -| | | | -+----------------+--------------------+---------------------------------+ -| Mar 24nd, 2017 | 1.2 | Bottlenecks Danube release 1.0 | -| | | | -+----------------+--------------------+---------------------------------+ ++----------------+--------------------+-----------------------------------+ +| *Date* | *Version* | *Comment* | +| | | | ++----------------+--------------------+-----------------------------------+ +| Sept 22nd, 2016| 1.0 | Bottlenecks Colorado release 1.0 | +| | | | ++----------------+--------------------+-----------------------------------+ +| Feb 17nd, 2017 | 1.1 | Bottlenecks Danube release 1.0 | +| | | | ++----------------+--------------------+-----------------------------------+ +| Mar 24nd, 2017 | 1.2 | Bottlenecks Danube release 1.0 | +| | | | ++----------------+--------------------+-----------------------------------+ +| Mar 24nd, 2017 | 1.3 | Bottlenecks Danube release 1.0 | +| | | | ++----------------+--------------------+-----------------------------------+ +| Apr 25th, 2017 | 1.4 | Bottlenecks Danube release 2.0 | +| | | | ++----------------+--------------------+-----------------------------------+ +| Jun 19th, 2017 | 1.5 | Bottlenecks Danube release 3.0 | +| | | | ++----------------+--------------------+-----------------------------------+ +| Sept 15th, 2017| 1.6 | Bottlenecks Euphrates release 5.0 | +| | | | ++----------------+--------------------+-----------------------------------+ +| Nov 15th, 2017 | 1.6.1 | Bottlenecks Euphrates release 5.1 | +| | | | ++----------------+--------------------+-----------------------------------+ Summary ======= @@ -58,7 +73,7 @@ Summary * User Guide - * Development Guide + * Developer Guide The *Bottlenecks software* is developed in the OPNFV community, by the Bottlenecks_ team. @@ -66,74 +81,129 @@ Bottlenecks_ team. Release Data ============ +Euphrates Release Data +----------------------- + ++--------------------------------------+--------------------------------+ +| **Project** | Bottlenecks | +| | | ++--------------------------------------+--------------------------------+ +| **Repo/tag** | * Bottlenecks/5.1.0 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Bottlenecks Docker image tag** | * 5.1.0 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Release designation** | * 5.1.0 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Release date** | * Nov 15th 2017 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Purpose of the delivery** | Euphrates stable release | +| | | ++--------------------------------------+--------------------------------+ + ++--------------------------------------+--------------------------------+ +| **Project** | Bottlenecks | +| | | ++--------------------------------------+--------------------------------+ +| **Repo/tag** | * Bottlenecks/5.0.0 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Bottlenecks Docker image tag** | * 5.0.0 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Release designation** | * 5.0.0 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Release date** | * Sept 15th 2017 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Purpose of the delivery** | Euphrates stable release | +| | | ++--------------------------------------+--------------------------------+ + Danube Release Data ----------------------- -+--------------------------------------+--------------------------------------+ -| **Project** | Bottlenecks | -| | | -+--------------------------------------+--------------------------------------+ -| **Repo/tag** | Bottlenecks/danube.1.0 | -| | | -+--------------------------------------+--------------------------------------+ -| **Bottlenecks Docker image tag** | danube.1.0 | -| | | -+--------------------------------------+--------------------------------------+ -| **Release designation** | Danube 1.0 | -| | | -+--------------------------------------+--------------------------------------+ -| **Release date** | March 31 2017 | -| | | -+--------------------------------------+--------------------------------------+ -| **Purpose of the delivery** | Danube base release | -| | | -+--------------------------------------+--------------------------------------+ ++--------------------------------------+--------------------------------+ +| **Project** | Bottlenecks | +| | | ++--------------------------------------+--------------------------------+ +| **Repo/tag** | * Bottlenecks/danube.1.0 | +| | * Bottlenecks/danube.2.0 | +| | | ++--------------------------------------+--------------------------------+ +| **Bottlenecks Docker image tag** | * danube.1.0 | +| | * danube.2.0 | +| | | ++--------------------------------------+--------------------------------+ +| **Release designation** | * Danube 1.0 | +| | * Danube 2.0 | +| | | ++--------------------------------------+--------------------------------+ +| **Release date** | * March 31st 2017 | +| | * May 1st 2017 | +| | | ++--------------------------------------+--------------------------------+ +| **Purpose of the delivery** | Danube stable release | +| | | ++--------------------------------------+--------------------------------+ Colorado Release Data ----------------------- -+--------------------------------------+--------------------------------------+ -| **Project** | Bottlenecks | -| | | -+--------------------------------------+--------------------------------------+ -| **Repo/tag** | Bottlenecks/colorado.1.0 | -| | | -+--------------------------------------+--------------------------------------+ -| **Bottlenecks Docker image tag** | colorado.1.0 | -| | | -+--------------------------------------+--------------------------------------+ -| **Release designation** | Colorado 1.0 | -| | | -+--------------------------------------+--------------------------------------+ -| **Release date** | September 22 2016 | -| | | -+--------------------------------------+--------------------------------------+ -| **Purpose of the delivery** | Colorado base release | -| | | -+--------------------------------------+--------------------------------------+ ++--------------------------------------+--------------------------------+ +| **Project** | Bottlenecks | +| | | ++--------------------------------------+--------------------------------+ +| **Repo/tag** | Bottlenecks/colorado.1.0 | +| | | ++--------------------------------------+--------------------------------+ +| **Bottlenecks Docker image tag** | * colorado.1.0 | +| | | ++--------------------------------------+--------------------------------+ +| **Release designation** | * Colorado 1.0 | +| | | ++--------------------------------------+--------------------------------+ +| **Release date** | * September 22 2016 | +| | | ++--------------------------------------+--------------------------------+ +| **Purpose of the delivery** | Colorado stable release | +| | | ++--------------------------------------+--------------------------------+ Bramaputra Release Data ----------------------- -+--------------------------------------+--------------------------------------+ -| **Project** | Bottlenecks | -| | | -+--------------------------------------+--------------------------------------+ -| **Repo/tag** | Bottlenecks/brahmaputra.1.0 | -| | | -+--------------------------------------+--------------------------------------+ -| **Bottlenecks Docker image tag** | brahmaputra.1.0 | -| | | -+--------------------------------------+--------------------------------------+ -| **Release designation** | Brahmaputra 1.0 | -| | | -+--------------------------------------+--------------------------------------+ -| **Release date** | February 25 2016 | -| | | -+--------------------------------------+--------------------------------------+ -| **Purpose of the delivery** | Brahmaputra base release | -| | | -+--------------------------------------+--------------------------------------+ ++--------------------------------------+--------------------------------+ +| **Project** | Bottlenecks | +| | | ++--------------------------------------+--------------------------------+ +| **Repo/tag** | Bottlenecks/brahmaputra.1.0 | +| | | ++--------------------------------------+--------------------------------+ +| **Bottlenecks Docker image tag** | * brahmaputra.1.0 | +| | | ++--------------------------------------+--------------------------------+ +| **Release designation** | * Brahmaputra 1.0 | +| | | ++--------------------------------------+--------------------------------+ +| **Release date** | * February 25 2016 | +| | | ++--------------------------------------+--------------------------------+ +| **Purpose of the delivery** | Brahmaputra stable release | +| | | ++--------------------------------------+--------------------------------+ Danube Deliverables @@ -142,7 +212,7 @@ Danube Deliverables Software Deliverables --------------------- -**Bottlenecks source code ** +**Bottlenecks source code** * https://gerrit.opnfv.org/gerrit/gitweb?p=bottlenecks.git;a=summary @@ -150,47 +220,52 @@ Software Deliverables Documentatiion Deliverables --------------------------- -**Bottlenecks documentation ** - -* Release Notes: TO-DO -* User Guide: TO-DO -* Development Guide: TO-DO - -Colorado Deliverables -===================== - -Software Deliverables ---------------------- - -**Bottlenecks source code ** +**Bottlenecks documentation ** -* https://gerrit.opnfv.org/gerrit/gitweb?p=bottlenecks.git;a=summary +* Release Notes: http://docs.opnfv.org/en/stable-euphrates/submodules/bottlenecks/docs/release/release-notes/release_notes.html +* User Guide: http://docs.opnfv.org/en/stable-euphrates/submodules/bottlenecks/docs/testing/user/userguide/index.html +* Developer Guide: http://docs.opnfv.org/en/stable-euphrates/submodules/bottlenecks/docs/testing/developer/devguide/index.html +**Bottlenecks documentation ** -Documentatiion Deliverables ---------------------------- +* Release Notes: http://docs.opnfv.org/en/stable-danube/submodules/bottlenecks/docs/release/release-notes/release_notes.html +* User Guide: http://docs.opnfv.org/en/stable-danube/submodules/bottlenecks/docs/testing/user/userguide/index.html +* Developer Guide: http://docs.opnfv.org/en/stable-danube/submodules/bottlenecks/docs/testing/developer/devguide/index.html **Bottlenecks documentation ** -* Platformation Overview: http://artifacts.opnfv.org/bottlenecks/colorado/1.0/docs/platformoverview/index.html +* Platform Overview: http://artifacts.opnfv.org/bottlenecks/colorado/1.0/docs/platformoverview/index.html * Configuration Guide: http://artifacts.opnfv.org/bottlenecks/colorado/1.0/configurationguide/index.html * Installation Guide: http://artifacts.opnfv.org/bottlenecks/colorado/docs/installationprocedure/index.html * http://artifacts.opnfv.org/bottlenecks/colorado/1.0/releasenotes/index.html * User Guide: http://artifacts.opnfv.org/bottlenecks/colorado/1.0/docs/userguide/index.html -Reason for Version -================== +Reason for Euphrates +==================== -* In Danube, POSCA testsuite is introduced and two stress tests are implemented. Rubbos and vstf are -not supported in this release. -* In Colorado, rubbos is refactored by using puppet, which makes it quite flexible to configure with different number of load generator(Client), worker(tomcat). -* In Colorado, vstf is refactored by extracting the test case's configuration information. +* Introduction of a simple monitoring module, i.e., Prometheus+Collectd+Node+Grafana to monitor the system behavior when executing stress tests. +* Support VNF scale up/out tests to verify NFVI capability to adapt the resource consuming. +* Extend Life-cycle test to data-plane to validate the system capability to handle concurrent networks usage. +* Testing framework is revised to support installer-agnostic testing. + +These enhancements and test cases help the end users to gain more comprehensive understanding of the SUT. +Graphic reports of the system behavior additional to test cases are provided to indicate the confidence level of SUT. +Installer-agnostic testing framework allow end user to do stress testing adaptively over either Open Source or commercial deployments. Known restrictions/issues ========================= -* TODO +* Bottlenecks uses Yardstick to do stress tests by iteratively calling yardstick running test cases and in the meantime increasing load. + + * Sometimes, Yardstick just waits for the test environment preparing. + This cause troubles since it seams to get Yardstick into a loop to keep waiting. + For OPNFV CI, this loop will be automatically stopped because of the default timeout setting. However, for local testing, user should stop this loop manually. + + * Sometimes, Yardstick will return empty test results with test flag indicating test is excuted succefully. + It maybe due to the environment issue or poor internet connection causing testing tools are not installed successfully. + +* Sometimes, a process will go to sleep state. In this case when a running tool go to sleep state, we try to call it twice. Normally, it will response. This applies to the traffic generator, i.e., netperf. Test results @@ -221,20 +296,23 @@ The test results are reported to MongoDB. An example is given below. } For more information, please refer to: https://wiki.opnfv.org/display/testing/Result+alignment+for+ELK+post-processing + - Results reported in MongoDB could find at http://testresults.opnfv.org/test/api/v1/results?project=bottlenecks + - Test Cases are defined in http://testresults.opnfv.org/test/api/v1/results?project=bottlenecks Open JIRA tickets ================= -+------------------+-----------------------------------------------+ -| JIRA | Description | -+==================+===============================================+ -| BOTTLENECK-103 | Refactoring the Bottlenecks tesing workflow | -+------------------+-----------------------------------------------+ -| BOTTLENECK-135 | Avoiding docker-compose building new image | -| | for Bottlenecks | -+------------------+-----------------------------------------------+ -| BOTTLENECK-147 | Investigting why calling remote docker client | -+------------------+-----------------------------------------------+ ++------------------+----------------------------------------------------+ +| JIRA | Description | ++==================+====================================================+ +| BOTTLENECK-147 | Investigting why calling remote docker client | ++------------------+----------------------------------------------------+ +| BOTTLENECK-207 | Yardstick loop when executing test cases | ++------------------+----------------------------------------------------+ +| BOTTLENECK-208 | Yardstick empty results when executing test cases | ++------------------+----------------------------------------------------+ +| BOTTLENECK-209 | Sleep process when executing test cases | ++------------------+----------------------------------------------------+ Useful links @@ -249,5 +327,3 @@ Useful links - Bottlenecks CI dashboard: https://build.opnfv.org/ci/view/bottlenecks - Bottlenecks IRC chanel: #opnfv-bottlenecks - -