docs/release/release-notes/functest-release.rst

   1 .. SPDX-License-Identifier: CC-BY-4.0
   2
   3 ======================================
   4 OPNFV master release note for Functest
   5 ======================================
   6
   7 Abstract
   8 ========
   9
  10 This document contains the release notes of the Functest project.
  11
  12 OPNFV master Release
  13 ====================
  14
  15 Functest deals with functional testing of the OPNFV solution.
  16 It includes test cases developed within the project, test cases developed in
  17 other OPNFV projects and it also integrates test cases from other upstream
  18 communities.
  19
  20 OpenStack
  21 ---------
  22
  23 The internal test cases are:
  24
  25  * connection_check
  26  * api_check
  27  * snaps_health_check
  28  * vping_ssh
  29  * vping_userdata
  30  * tempest_smoke_serial
  31  * rally_sanity
  32  * refstack_defcore
  33  * patrole
  34  * odl
  35  * snaps_smoke
  36  * neutron_trunk
  37  * tempest_full_parallel
  38  * rally_full
  39  * cloudify_ims
  40  * vyos_vrouter
  41  * juju_epc
  42
  43 The OPNFV projects integrated into Functest framework for automation are:
  44
  45  * doctor
  46  * bgpvpn
  47  * odl-sfc
  48  * barometer
  49  * fds
  50
  51 Kubernetes
  52 ----------
  53
  54 The internal test cases are:
  55
  56  * k8s_smoke
  57  * k8s_conformance
  58
  59 The OPNFV projects integrated into Functest framework for automation are:
  60
  61  * stor4nfv
  62  * clover
  63
  64 Release Data
  65 ============
  66
  67 +--------------------------------------+--------------------------------------+
  68 | **Project**                          | functest                             |
  69 +--------------------------------------+--------------------------------------+
  70 | **Repository branch**                | master                               |
  71 +--------------------------------------+--------------------------------------+
  72
  73 Deliverables
  74 ============
  75
  76 Software
  77 --------
  78
  79  Functest Docker images (OpenStack):
  80
  81  * https://hub.docker.com/r/opnfv/functest-healthcheck
  82  * https://hub.docker.com/r/opnfv/functest-smoke
  83  * https://hub.docker.com/r/opnfv/functest-features
  84  * https://hub.docker.com/r/opnfv/functest-components
  85  * https://hub.docker.com/r/opnfv/functest-vnf
  86
  87  Functest Docker images (Kubernetes):
  88
  89  * https://hub.docker.com/r/opnfv/functest-kubernetes-healthcheck
  90  * https://hub.docker.com/r/opnfv/functest-kubernetes-smoke
  91  * https://hub.docker.com/r/opnfv/functest-kubernetes-features
  92
  93 Docker tag for master: latest
  94
  95 Documents
  96 ---------
  97
  98  * Config Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
  99  * User Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html
 100  * Developer Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/developer/devguide/index.html
 101  * API Docs: http://functest.readthedocs.io/en/latest/
 102
 103 Version change
 104 ==============
 105
 106 New test cases
 107 --------------
 108
 109  * neutron_trunk
 110  * patrole
 111  * juju_epc
 112  * k8s_smoke
 113  * k8s_conformance
 114  * stor4nfv
 115  * clover
 116
 117 Key changes
 118 -----------
 119
 120  * update test cases and containers to `OpenStack Pike`_
 121  * move the framework into a separate project: Xtesting_
 122  * ease running all containers thanks to default values
 123  * clean interfaces with OPNFV Installers and Features
 124  * rewrite all vnfs to allow multiple tests in parallel
 125  * fully support non-default region names and Keystone v3 domains
 126  * refactor all tempest-based test cases (e.g. refstack_defcore)
 127  * remove obsolete OpenStack and Functest utils
 128  * verify all changes via doc8 and yamllint too
 129  * generate reports for all tempest-based test cases
 130
 131 .. _`OpenStack Pike`: https://raw.githubusercontent.com/openstack/requirements/stable/pike/upper-constraints.txt
 132 .. _Xtesting: http://xtesting.readthedocs.io/en/latest/
 133
 134 Key benefits
 135 ------------
 136
 137  * the enduser can run all tests by setting only one input (EXTERNAL_NETWORK)
 138  * the developer can only work on the test suites without diving into CI/CD
 139    integration
 140  * both OpenStack and Kubernetes deployments can be verified
 141  * Functest test cases are trustable as they meet the best coding rules (unit
 142    tests, coverage, linters, etc.)
 143  * Functest can be reused in other projects (e.g.
 144    `ONS: Re-using OPNFV framework tests for LFN projects`_)
 145
 146 .. _`ONS: Re-using OPNFV framework tests for LFN projects`: https://wiki.lfnetworking.org/display/LN/LFN+Developer+Forum+Schedule?preview=/328197/328329/ONS-OPNFV%20framework%20tests%20for%20LFN%20projects.pdf
 147
 148 Code quality
 149 ------------
 150
 151  * pylint: ~9.5/10
 152  * code coverage: ~70%
 153
 154 Useful links
 155 ============
 156
 157  * wiki project page: https://wiki.opnfv.org/opnfv_functional_testing
 158  * Functest git repository: https://git.opnfv.org/cgit/functest
 159  * Functest CI dashboard: https://build.opnfv.org/ci/view/functest/
 160  * JIRA dashboard: https://jira.opnfv.org/secure/Dashboard.jspa?selectPageId=10611
 161  * Functest IRC channel: #opnfv-functest
 162  * Reporting page: http://testresults.opnfv.org/reporting/master/functest/functest.html