-- Bugfixes in refstack client
-
-- Fixed vIMS onboarding and ims deployment
-
-- Increased timeout of OpenBaton installation
-
-- Switched to mitaka-eol in openstack repositories, as stable/mitake is not longer existing
-
-- Minor fixes in logging and test cases
-
-- 3 scenarios have been successfully executed on aarch64 architectures on Fuel deployments (nosdn, odl_l2 and odl_l3)
-
-
-Scenario Matrix
-===============
-
-For Danube 3.0, Functest was tested on the following HA scenarios (new
-dabube scenarios in bold):
-
-+---------------------+---------+---------+---------+---------+
-| Scenario | Apex | Compass | Fuel | Joid |
-+=====================+=========+=========+=========+=========+
-| nosdn | X | X | X | X |
-+---------------------+---------+---------+---------+---------+
-| **fdio** | X | | | |
-+---------------------+---------+---------+---------+---------+
-| kvm | X | | X | |
-+---------------------+---------+---------+---------+---------+
-| **kvm_ovs_dpdk** | | | X | |
-+---------------------+---------+---------+---------+---------+
-| **kvm_ovs_dpdk-bar**| | | X | |
-+---------------------+---------+---------+---------+---------+
-| lxd | | | | X |
-+---------------------+---------+---------+---------+---------+
-| **ovs** | | | X | |
-+---------------------+---------+---------+---------+---------+
-| **openo** | | X | | |
-+---------------------+---------+---------+---------+---------+
-| odl_l2 | | X | X | X |
-+---------------------+---------+---------+---------+---------+
-| odl-bgpvpn | X | | | |
-+---------------------+---------+---------+---------+---------+
-| odl_l2-bgpvpn | | | X | |
-+---------------------+---------+---------+---------+---------+
-| **odl_l2-fdio** | X | | | |
-+---------------------+---------+---------+---------+---------+
-| odl_l2-sfc | | | X | |
-+---------------------+---------+---------+---------+---------+
-| odl_l3 | X | X | X | |
-+---------------------+---------+---------+---------+---------+
-| **ocl** | | X | | |
-+---------------------+---------+---------+---------+---------+
-| onos | | X | | |
-+---------------------+---------+---------+---------+---------+
-| multisite | | | X | |
-+---------------------+---------+---------+---------+---------+
-
-Non HA scenarios:
-
-+---------------------+---------+---------+---------+---------+
-| Scenario | Apex | Compass | Fuel | Joid |
-+=====================+=========+=========+=========+=========+
-| nosdn | | | X | X |
-+---------------------+---------+---------+---------+---------+
-| kvm | | | X | |
-+---------------------+---------+---------+---------+---------+
-| **kvm_ovs_dpdk** | | | X | |
-+---------------------+---------+---------+---------+---------+
-| **kvm_ovs_dpdk-bar**| | | X | |
-+---------------------+---------+---------+---------+---------+
-| lxd | | | | X |
-+---------------------+---------+---------+---------+---------+
-| ovs | | | X | |
-+---------------------+---------+---------+---------+---------+
-| odl_l2 | | | X | |
-+---------------------+---------+---------+---------+---------+
-| odl_l2-bgpvpn | | | X | |
-+---------------------+---------+---------+---------+---------+
-| odl_l2-fdio | X | | | |
-+---------------------+---------+---------+---------+---------+
-| **odl_l3-fdio** | X | | | |
-+---------------------+---------+---------+---------+---------+
-| odl_l2-sfc | | | X | |
-+---------------------+---------+---------+---------+---------+
-| odl_l3 | | | X | |
-+---------------------+---------+---------+---------+---------+
-| **odl_gluon** | X | | | |
-+---------------------+---------+---------+---------+---------+
-
-Colorado deprecated scenarios:
-
- * odl_l2-moon
- * onos-sfc
- * onos-noha
- * onos-sfc-noha
-
-For the scenario validation, we consider only the healthcheck, smoke and
-features categories. These tests are run systematically in the CI daily loops.
-
-Success criterias have been defined for these test cases, they can be
-PASS/FAIL or a success rate may be declared (100%, > 90%).
-All the criteria, as well as the test dependencies are declared in the
-testcases.yaml file located in the ci directory of the repository.
-
-The scoring for the Danube release per installer can be described as
-follows.
-
-The scoring is an indicator showing how many feature project test suites
-have been integrated in the scenario.
-
-The scoring equals the number of tests * successful iteration of each
-test [0-3]. The scoring method is described in https://wiki.opnfv.org/pages/viewpage.action?pageId=6828617
-
- e.g.
- apex/odl_l2-nofeature-ha
- tests = vping_ssh+vping_userdata+tempest+rally+odl+doctor+copper
- Scoring = 21/21 = 7 * 3
-
-By default, if not specified, the scenarios are HA.
-HA means OpenStack High Availability (main services). Note that not
-all VIM (e.g. OpenStack) services are deployed in HA mode, as that
-depends upon support of the specific service for HA deployment.
-For example, in the Danube release, the Congress service
-is deployed in non-HA mode even for HA OPNFV scenarios, as explicit
-support for HA operation has not yet been verified.
-See the release notes of the installers for more details.
-
-
-Apex
-----
-
-+------------------+---------+---------+-----------------+
-| Scenario | Scoring | Success | Results |
-| | | rate | |
-+==================+=========+=========+=================+
-| nosdn | 20/30 | 67% | `apex-res-1`_ |
-+------------------+---------+---------+-----------------+
-| odl_l3 | 16/30 | 53% | `apex-res-2`_ |
-+------------------+---------+---------+-----------------+
-| odl-bgpvpn | 18/33 | 56% | `apex-res-3`_ |
-+------------------+---------+---------+-----------------+
-| odl-gluon | 20/33 | 61% | `apex-res-4`_ |
-+------------------+---------+---------+-----------------+
-| kvm | 20/30 | 67% | `apex-res-5`_ |
-+------------------+---------+---------+-----------------+
-| odl_l2-fdio | 25/33 | 76% | `apex-res-6`_ |
-+------------------+---------+---------+-----------------+
-| odl_l2-fdio-noha | 24/33 | 73% | local push |
-+------------------+---------+---------+-----------------+
-| odl_l3-fdio-noha | 18/30 | 60% | local push |
-+------------------+---------+---------+-----------------+
-| fdio | 25/27 | 93% | local push |
-+------------------+---------+---------+-----------------+
-
-Compass
--------
-
-+------------------+---------+---------+------------------+
-| Scenario | Scoring | Success | Results |
-| | | rate | |
-+==================+=========+=========+==================+
-| nosdn | 29/30 | 97% | `compass-res-1`_ |
-+------------------+---------+---------+------------------+
-| odl_l2 | 29/33 | 88% | `compass-res-2`_ |
-+------------------+---------+---------+------------------+
-| odl_l3 | 23/30 | 77% | `compass-res-3`_ |
-+------------------+---------+---------+------------------+
-| onos | 29/30 | 97% | `compass-res-4`_ |
-+------------------+---------+---------+------------------+
-| onos-sfc | 20/36 | 56% | `compass-res-5`_ |
-+------------------+---------+---------+------------------+
-
-Note: all the Compass tests for Danube have been executed on virtual
-environment. Bare metal resources were used for Master branch.
-
-
-Fuel
-----
-
-+----------------------+---------+---------+----------------+
-| Scenario | Scoring | Success | Results |
-| | | rate | |
-+======================+=========+=========+================+
-| nosdn | 39/39 | 100% | `fuel-res-1`_ |
-+----------------------+---------+---------+----------------+
-| nosdn-noha | 36/36 | 100% | `fuel-res-2`_ |
-+----------------------+---------+---------+----------------+
-| nosdn-kvm | 39/39 | 100% | `fuel-res-3`_ |
-+----------------------+---------+---------+----------------+
-| nosdn-kvm-noha | 36/36 | 100% | `fuel-res-4`_ |
-+----------------------+---------+---------+----------------+
-| nosdn-ovs | 39/39 | 100% | `fuel-res-5`_ |
-+----------------------+---------+---------+----------------+
-| nosdn-ovs-noha | 36/36 | 100% | `fuel-res-6`_ |
-+----------------------+---------+---------+----------------+
-| odl_l2 | 42/42 | 100% | `fuel-res-7`_ |
-+----------------------+---------+---------+----------------+
-| odl_l2-noha | 39/39 | 100% | `fuel-res-8`_ |
-+----------------------+---------+---------+----------------+
-| odl_l2-sfc | 45/45 | 100% | `fuel-res-9`_ |
-+----------------------+---------+---------+----------------+
-| odl_l2-sfc-noha | 35/42 | 83% | `fuel-res-10`_ |
-+----------------------+---------+---------+----------------+
-| odl_l3 | 36/39 | 92% | `fuel-res-11`_ |
-+----------------------+---------+---------+----------------+
-| odl_l3-noha | 36/36 | 100% | `fuel-res-12`_ |
-+----------------------+---------+---------+----------------+
-| kvm_ovs_dpdk | 39/39 | 100% | `fuel-res-13`_ |
-+----------------------+---------+---------+----------------+
-| kvm_ovs_dpdk_noha | 35/36 | 97% | `fuel-res-14`_ |
-+----------------------+---------+---------+----------------+
-| kvm_ovs_dpdk_bar | 42/42 | 100% | `fuel-res-15`_ |
-+----------------------+---------+---------+----------------+
-| kvm_ovs_dpdk_bar_noha| 36/39 | 92% | `fuel-res-16`_ |
-+----------------------+---------+---------+----------------+
-
-
-
-
-Joid
-----
-
-+---------------------+---------+---------+---------------+
-| Scenario | Scoring | Success | Results |
-| | | rate | |
-+=====================+=========+=========+===============+
-| nosdn | 30/30 | 100% | `joid-res-1`_ |
-+---------------------+---------+---------+---------------+
-| nosdn-noha | 10/30 | 33% | `joid-res-2`_ |
-+---------------------+---------+---------+---------------+
-| nosdn-lxd | 19/21 | 90% | `joid-res-3`_ |
-+---------------------+---------+---------+---------------+
-| nosdn-lxd-noha | 15/21 | 71% | `joid-res-4`_ |
-+---------------------+---------+---------+---------------+
-| odl_l2 | 6/33 | 18% | `joid-res-5`_ |
-+---------------------+---------+---------+---------------+
-
-It is highly recommended to install a json viewer in your browser
-(e.g. https://addons.mozilla.org/fr/firefox/addon/jsonview/)
-
-You can get additional details through test logs on http://artifacts.opnfv.org/.
-As no search engine is available on the OPNFV artifact web site you must
-retrieve the pod identifier on which the tests have been executed (see
-field pod in any of the results) then click on the selected POD and look
-for the date of the test you are interested in.
-
-The reporting pages can be found at:
-
- * apex: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-apex.html
- * compass: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-compass.html
- * fuel: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-fuel.html
- * joid: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-joid.html
-
-Danube known restrictions/issues
-==================================
-
-************* TODO *****************************************
-
-+-----------+-----------+----------------------------------------------+
-| Installer | Scenario | Issue |
-+===========+===========+==============================================+
-| fuel | odl_-* | Tempest test case "TestServerBasicOps" |
-| | | disabled due to bug `tempest-bug`_ |
-+-----------+-----------+----------------------------------------------+
-| apex/fuel | *-bgpvpn | Due to some instabilities in the bgpvpn |
-| | | test case, the scenario has been postponed |
-| | | to Danube 2.0 |
-+-----------+-----------+----------------------------------------------+
-| apex | *-gluon | vPing_ssh disabled due to floating ips |
-| | | not working 100% of the times. |
-| | | Tempest test "test_reboot_server_hard" |
-| | | disabled due to bug `gluon-bug`_ |
-+-----------+-----------+----------------------------------------------+
-| joid | any | Tempest cases related to object storage |
-| | | excluded |
-+-----------+-----------+----------------------------------------------+
-| any | any | The VNF tier has not been fully tested |
-| | | since it has not been run in daily loops |
-| | | in CI. Weekly jobs have been activated |
-| | | a bit late in the process and have not been |
-| | | used to validate the scenarios. |
-+-----------+-----------+----------------------------------------------+
Code Review / functest.git / blobdiff