-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Ericsson AB and others.
+=======
+License
+=======
+OPNFV Colorado release note for Yardstick Docs
+are licensed under a Creative Commons Attribution 4.0 International License.
+You should have received a copy of the license along with this.
+If not, see <http://creativecommons.org/licenses/by/4.0/>.
-============================================
+The *Yardstick framework*, the *Yardstick test cases* and the *ApexLake*
+experimental framework are opensource software, licensed under the terms of the
+Apache License, Version 2.0.
+
+=========================================
OPNFV Colorado Release Note for Yardstick
-============================================
+=========================================
.. toctree::
:maxdepth: 2
This document describes the release note of Yardstick project.
-License
-=======
-
-OPNFV Colorado release note for Yardstick Docs
-are licensed under a Creative Commons Attribution 4.0 International License.
-You should have received a copy of the license along with this.
-If not, see <http://creativecommons.org/licenses/by/4.0/>.
-
-The *Yardstick framework*, the *Yardstick test cases* and the *ApexLake*
-experimental framework are opensource software, licensed under the terms of the
-Apache License, Version 2.0.
-
-
Version History
===============
| *Date* | *Version* | *Comment* |
| | | |
+----------------+--------------------+---------------------------------+
+| Oct 27nd, 2016 | 2.0 | Yardstick for Colorado release |
+| | | |
++----------------+--------------------+---------------------------------+
| Aug 22nd, 2016 | 1.0 | Yardstick for Colorado release |
| | | |
+----------------+--------------------+---------------------------------+
independent.
-Summary
-=======
+OPNFV Colorado Release
+======================
This Colorado release provides *Yardstick* as a framework for NFVI testing
and OPNFV feature testing, automated in the OPNFV CI pipeline, including:
| **Project** | Yardstick |
| | |
+--------------------------------------+--------------------------------------+
-| **Repo/tag** | yardstick/colorado.1.0 |
+| **Repo/tag** | yardstick/colorado.2.0 |
| | |
+--------------------------------------+--------------------------------------+
-| **Yardstick Docker image tag** | colorado.1.0 |
+| **Yardstick Docker image tag** | colorado.2.0 |
| | |
+--------------------------------------+--------------------------------------+
-| **Release designation** | Colorado base release |
+| **Release designation** | Colorado |
| | |
+--------------------------------------+--------------------------------------+
-| **Release date** | September 22 2016 |
+| **Release date** | October 27 2016 |
| | |
+--------------------------------------+--------------------------------------+
-| **Purpose of the delivery** | Colorado base release |
+| **Purpose of the delivery** | OPNFV Colorado release 2.0 |
| | |
+--------------------------------------+--------------------------------------+
Deliverables
============
+Documents
+---------
+
+ - User Guide: http://artifacts.opnfv.org/yardstick/colorado/docs/userguide/index.html
+
+ - Test Results: http://artifacts.opnfv.org/yardstick/colorado/docs/results/overview.html
+
+
Software Deliverables
---------------------
-**Yardstick framework source code <colorado.1.0>**
+**Yardstick framework source code <colorado.2.0>**
+--------------------------------------+--------------------------------------+
| **Project** | Yardstick |
| | |
+--------------------------------------+--------------------------------------+
-| **Repo/tag** | yardstick/colorado.1.0 |
+| **Repo/tag** | yardstick/colorado.2.0 |
| | |
+--------------------------------------+--------------------------------------+
-| **Yardstick Docker image tag** | colorado.1.0 |
+| **Yardstick Docker image tag** | colorado.2.0 |
| | |
+--------------------------------------+--------------------------------------+
-| **Release designation** | colorado |
+| **Release designation** | Colorado |
| | |
+--------------------------------------+--------------------------------------+
-| **Release date** | September 22th, 2016 |
+| **Release date** | October 27th, 2016 |
| | |
+--------------------------------------+--------------------------------------+
| **Purpose of the delivery** | OPNFV Colorado release |
Module Version Changes
----------------------
-This is the third tracked release of Yardstick. It is based on following
+This is the second tracked release of Yardstick. It is based on following
upstream versions:
+- ONOS Goldeneye
+
- OpenStack Mitaka
- OpenDaylight Beryllium
Document Version Changes
------------------------
-This is the third tracked version of the Yardstick framework in OPNFV.
+This is the second tracked version of the Yardstick framework in OPNFV.
It includes the following documentation updates:
-- Yardstick User Guide: corrected faulty links
+- Yardstick User Guide: added yardstick plugin chapter; added Store Other
+Project's Test Results in InfluxDB chapter; Refine yardstick instantion chapter.
- Yardstick Code Documentation: no changes
- Test Results report for Colorado testing with Yardstick: updated listed of
verified scenarios and limitations
-Documentation updates on the second tracked version:
-- Yardstick User Guide: added software architecture chapter
+Feature additions
+-----------------
+ - Yardstick plugin
+ - Yardstick reporting
+ - StorPerf Integration
+
+
+Scenario Matrix
+===============
-- Yardstick Code Documentation: no changes
+For Colorado 2.0, Yardstick was tested on the following scenarios:
+
++-------------------------+---------+---------+---------+---------+
+| Scenario | Apex | Compass | Fuel | Joid |
++=========================+=========+=========+=========+=========+
+| os-nosdn-nofeature-noha | | | | X |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-nofeature-ha | X | | X | X |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-nofeature-ha | X | X | X | X |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-nofeature-noha| | X | | |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l3-nofeature-ha | X | | X | |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l3-nofeature-ha | | X | | |
++-------------------------+---------+---------+---------+---------+
+| os-onos-sfc-ha | X | | X | X |
++-------------------------+---------+---------+---------+---------+
+| os-onos-nofeature-ha | X | | X | X |
++-------------------------+---------+---------+---------+---------+
+| os-onos-nofeature-noha | | X | | |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-sfc-ha | | | X | |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-sfc-noha | X | X | | |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-bgpvpn-ha | X | | X | |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-bgpvpn-noha | | X | | |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-kvm-ha | | | X | |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-kvm-noha | | X | | |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-ovs-ha | | | X | |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-ovs-noha | X | | X | |
++-------------------------+---------+---------+---------+---------+
+| os-ocl-nofeature-ha | | | | |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-lxd-ha | | | | X |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-lxd-noha | | | | X |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-fdio-noha | X | | | |
++-------------------------+---------+---------+---------+---------+
-- Yardstick Release Notes for Yardstick: this document
-- Test Results report for Brahmaputra testing with Yardstick: added test cases
-and results for virtual Traffic Classifier
+Test results
+============
+Test results are available in:
-Reason for Version
-==================
-* TODO *
+ - jenkins logs on CI: https://build.opnfv.org/ci/view/yardstick/
-Feature additions
------------------
-* TODO *
+The reporting pages can be found at:
+
+ * apex: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-apex.html
+ * compass: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-compass.html
+ * fuel: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-fuel.html
+ * joid: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-joid.html
+
+You can get additional details through test logs on http://artifacts.opnfv.org/.
+As no search engine is available on the OPNFV artifact web site you must
+retrieve the pod identifier on which the tests have been executed (see
+field pod in any of the results) then click on the selected POD and look
+for the date of the test you are interested in.
-Corrected Faults
-----------------
-* TODO *
Known Issues/Faults
------------
-* TODO *
+ - Floating IP not supported in bgpvpn scenario
+ - Floating IP not supported in apex-os-odl_l3-nofeature-ha scenario
.. note:: The faults not related to *Yardstick* framework, addressing scenarios
which were not fully verified, are listed in the OPNFV installer's release
notes.
-Colorado known restrictions/issues
+Corrected Faults
+----------------
+
+Colorado.2.0:
+
++----------------------------+------------------------------------------------+
+| **JIRA REFERENCE** | **SLOGAN** |
+| | |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-325 | Provide raw format yardstick vm image for |
+| | nova-lxd scenario. |
+| | |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-358 | tc027 ipv6 test case to de-coupling to the |
+| | installers. |
+| | |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-359 | ipv6 testcase disable port-security on |
+| | vRouter. |
+| | |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-363 | ipv6 testcase to support fuel. |
+| | |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-367 | Add d3 graph presentation to yardstick |
+| | reporting. |
+| | |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-371 | Provide raw format yardstick vm image for |
+| | nova-lxd scenario. |
+| | |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-372 | cannot find yardstick-img-dpdk-modify and |
+| | yardstick-img-lxd-modify in environment |
+| | varibales. |
+| | |
++----------------------------+------------------------------------------------+
+
+
+Colorado 2.0 known restrictions/issues
==================================
+-----------+-----------+----------------------------------------------+
| Installer | Scenario | Issue |
+===========+===========+==============================================+
-| any | *-bgpvpn | floating ips not supported. Some Test cases |
+| any | *-bgpvpn | Floating ips not supported. Some Test cases |
| | | related to floating ips are excluded. |
+-----------+-----------+----------------------------------------------+
-* TODO *
-
-
-Test results
-============
-
-Test results are available in:
-
- - jenkins logs on CI: https://build.opnfv.org/ci/view/yardstick/
+| any | odl_l3-* | Some test cases related to using floating IP |
+| | | addresses fail because of a known ODL bug. |
+| | | https://jira.opnfv.org/browse/APEX-112 |
++-----------+-----------+----------------------------------------------+
Open JIRA tickets
=================
-+------------------+-----------------------------------------------+
-| JIRA | Description |
-+==================+===============================================+
-+------------------+-----------------------------------------------+
-+------------------+-----------------------------------------------+
-+------------------+-----------------------------------------------+
-+------------------+-----------------------------------------------+
-+------------------+-----------------------------------------------+
-
Useful links
============
- Yardstick IRC chanel: #opnfv-yardstick
+.. _`YARDSTICK-325` : https://jira.opnfv.org/browse/YARDSTICK-325
+
+.. _`YARDSTICK-358` : https://jira.opnfv.org/browse/YARDSTICK-358
+
+.. _`YARDSTICK-359` : https://jira.opnfv.org/browse/YARDSTICK-359
+
+.. _`YARDSTICK-363` : https://jira.opnfv.org/browse/YARDSTICK-363
+
+.. _`YARDSTICK-367` : https://jira.opnfv.org/browse/YARDSTICK-367
+
+.. _`YARDSTICK-371` : https://jira.opnfv.org/browse/YARDSTICK-371
+
+.. _`YARDSTICK-372` : https://jira.opnfv.org/browse/YARDSTICK-372
Code Review / yardstick.git / blobdiff