docs/release/release-notes/functest-release.rst

   1 .. SPDX-License-Identifier: CC-BY-4.0
   2
   3 ======================================
   4 OPNFV master release note for Functest
   5 ======================================
   6
   7 Abstract
   8 ========
   9
  10 This document contains the release notes of the Functest project.
  11
  12 OPNFV master Release
  13 ====================
  14
  15 Functest deals with functional testing of the OPNFV solution.
  16 It includes test cases developed within the project, test cases developed in
  17 other OPNFV projects and it also integrates test cases from other upstream
  18 communities.
  19
  20 OpenStack
  21 ---------
  22
  23 The internal test cases are:
  24
  25  * connection_check
  26  * api_check
  27  * snaps_health_check
  28  * vping_ssh
  29  * vping_userdata
  30  * tempest_smoke_serial
  31  * rally_sanity
  32  * refstack_defcore
  33  * patrole
  34  * odl
  35  * snaps_smoke
  36  * neutron_trunk
  37  * tempest_full_parallel
  38  * rally_full
  39  * cloudify_ims
  40  * vyos_vrouter
  41  * juju_epc
  42
  43 The OPNFV projects integrated into Functest framework for automation are:
  44
  45  * doctor
  46  * bgpvpn
  47  * odl-sfc
  48  * barometer
  49  * fds
  50  * parser
  51
  52 Kubernetes
  53 ----------
  54
  55 The internal test cases are:
  56
  57  * k8s_smoke
  58  * k8s_conformance
  59
  60 The OPNFV projects integrated into Functest framework for automation are:
  61
  62  * stor4nfv
  63  * clover
  64
  65 Release Data
  66 ============
  67
  68 +--------------------------------------+--------------------------------------+
  69 | **Project**                          | functest                             |
  70 +--------------------------------------+--------------------------------------+
  71 | **Repository branch**                | master                               |
  72 +--------------------------------------+--------------------------------------+
  73
  74 Deliverables
  75 ============
  76
  77 Software
  78 --------
  79
  80  Functest Docker images (OpenStack):
  81
  82  * https://hub.docker.com/r/opnfv/functest-healthcheck
  83  * https://hub.docker.com/r/opnfv/functest-smoke
  84  * https://hub.docker.com/r/opnfv/functest-features
  85  * https://hub.docker.com/r/opnfv/functest-components
  86  * https://hub.docker.com/r/opnfv/functest-vnf
  87  * https://hub.docker.com/r/opnfv/functest-parser
  88
  89  Functest Docker images (Kubernetes):
  90
  91  * https://hub.docker.com/r/opnfv/functest-kubernetes-healthcheck
  92  * https://hub.docker.com/r/opnfv/functest-kubernetes-smoke
  93  * https://hub.docker.com/r/opnfv/functest-kubernetes-features
  94
  95 Docker tag for master: latest
  96
  97 Documents
  98 ---------
  99
 100  * Config Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
 101  * User Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html
 102  * Developer Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/developer/devguide/index.html
 103  * API Docs: http://functest.readthedocs.io/en/latest/
 104
 105 Version change
 106 ==============
 107
 108 New test cases
 109 --------------
 110
 111  * neutron_trunk
 112  * patrole
 113  * juju_epc
 114  * k8s_smoke
 115  * k8s_conformance
 116  * stor4nfv
 117  * clover
 118
 119 Key changes
 120 -----------
 121
 122  * update test cases and containers to `OpenStack Pike`_
 123  * move the framework into a separate project: Xtesting_
 124  * ease running all containers thanks to default values
 125  * clean interfaces with OPNFV Installers and Features
 126  * rewrite all vnfs to allow multiple tests in parallel
 127  * fully support non-default region names and Keystone v3 domains
 128  * refactor all tempest-based test cases (e.g. refstack_defcore)
 129  * remove obsolete OpenStack and Functest utils
 130  * verify all changes via doc8 and yamllint too
 131  * generate reports for all tempest-based test cases
 132
 133 .. _`OpenStack Pike`: https://raw.githubusercontent.com/openstack/requirements/stable/pike/upper-constraints.txt
 134 .. _Xtesting: http://xtesting.readthedocs.io/en/latest/
 135
 136 Key benefits
 137 ------------
 138
 139  * the enduser can run all tests by setting only one input (EXTERNAL_NETWORK)
 140  * the developer can only work on the test suites without diving into CI/CD
 141    integration
 142  * both OpenStack and Kubernetes deployments can be verified
 143  * Functest test cases are trustable as they meet the best coding rules (unit
 144    tests, coverage, linters, etc.)
 145  * Functest can be reused in other projects (e.g.
 146    `ONS: Re-using OPNFV framework tests for LFN projects`_)
 147
 148 .. _`ONS: Re-using OPNFV framework tests for LFN projects`: https://wiki.lfnetworking.org/display/LN/LFN+Developer+Forum+Schedule?preview=/328197/328329/ONS-OPNFV%20framework%20tests%20for%20LFN%20projects.pdf
 149
 150 Code quality
 151 ------------
 152
 153  * pylint: ~9.5/10
 154  * code coverage: ~70%
 155
 156 Useful links
 157 ============
 158
 159  * wiki project page: https://wiki.opnfv.org/opnfv_functional_testing
 160  * Functest git repository: https://git.opnfv.org/cgit/functest
 161  * Functest CI dashboard: https://build.opnfv.org/ci/view/functest/
 162  * JIRA dashboard: https://jira.opnfv.org/secure/Dashboard.jspa?selectPageId=10611
 163  * Functest IRC channel: #opnfv-functest
 164  * Reporting page: http://testresults.opnfv.org/reporting/master/functest/functest.html