docs/release/release-notes/functest-release.rst

   1 .. SPDX-License-Identifier: CC-BY-4.0
   2
   3 ======================================
   4 OPNFV master release note for Functest
   5 ======================================
   6
   7 Abstract
   8 ========
   9
  10 This document contains the release notes of the Functest project.
  11
  12 OPNFV master Release
  13 ====================
  14
  15 Functest deals with functional testing of the OPNFV solution.
  16 It includes test cases developed within the project, test cases developed in
  17 other OPNFV projects and it also integrates test cases from other upstream
  18 communities.
  19
  20 OpenStack
  21 ---------
  22
  23 The internal test cases are:
  24
  25  * connection_check
  26  * api_check
  27  * snaps_health_check
  28  * vping_ssh
  29  * vping_userdata
  30  * tempest_smoke_serial
  31  * rally_sanity
  32  * refstack_defcore
  33  * patrole
  34  * odl
  35  * odl-netvirt
  36  * snaps_smoke
  37  * neutron_trunk
  38  * tempest_full_parallel
  39  * rally_full
  40  * cloudify_ims
  41  * vyos_vrouter
  42  * juju_epc
  43
  44 The OPNFV projects integrated into Functest framework for automation are:
  45
  46  * doctor
  47  * bgpvpn
  48  * odl-sfc
  49  * barometer
  50  * fds
  51  * parser
  52
  53 Kubernetes
  54 ----------
  55
  56 The internal test cases are:
  57
  58  * k8s_smoke
  59  * k8s_conformance
  60
  61 The OPNFV projects integrated into Functest framework for automation are:
  62
  63  * stor4nfv
  64  * clover
  65
  66 Release Data
  67 ============
  68
  69 +--------------------------------------+--------------------------------------+
  70 | **Project**                          | functest                             |
  71 +--------------------------------------+--------------------------------------+
  72 | **Repository branch**                | master                               |
  73 +--------------------------------------+--------------------------------------+
  74
  75 Deliverables
  76 ============
  77
  78 Software
  79 --------
  80
  81  Functest Docker images (OpenStack):
  82
  83  * https://hub.docker.com/r/opnfv/functest-healthcheck
  84  * https://hub.docker.com/r/opnfv/functest-smoke
  85  * https://hub.docker.com/r/opnfv/functest-features
  86  * https://hub.docker.com/r/opnfv/functest-components
  87  * https://hub.docker.com/r/opnfv/functest-vnf
  88  * https://hub.docker.com/r/opnfv/functest-parser
  89
  90  Functest Docker images (Kubernetes):
  91
  92  * https://hub.docker.com/r/opnfv/functest-kubernetes-healthcheck
  93  * https://hub.docker.com/r/opnfv/functest-kubernetes-smoke
  94  * https://hub.docker.com/r/opnfv/functest-kubernetes-features
  95
  96 Docker tag for master: latest
  97
  98 Documents
  99 ---------
 100
 101  * Config Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
 102  * User Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html
 103  * Developer Guide: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/developer/devguide/index.html
 104  * API Docs: http://functest.readthedocs.io/en/latest/
 105
 106 Version change
 107 ==============
 108
 109 New test cases
 110 --------------
 111
 112  * neutron_trunk
 113  * patrole
 114  * juju_epc
 115  * k8s_smoke
 116  * k8s_conformance
 117  * stor4nfv
 118  * clover
 119
 120 Key changes
 121 -----------
 122
 123  * update test cases and containers to `OpenStack Pike`_
 124  * move the framework into a separate project: Xtesting_
 125  * ease running all containers thanks to default values
 126  * clean interfaces with OPNFV Installers and Features
 127  * rewrite all vnfs to allow multiple tests in parallel
 128  * fully support non-default region names and Keystone v3 domains
 129  * refactor all tempest-based test cases (e.g. refstack_defcore)
 130  * remove obsolete OpenStack and Functest utils
 131  * verify all changes via doc8 and yamllint too
 132  * generate reports for all tempest-based test cases
 133
 134 .. _`OpenStack Pike`: https://raw.githubusercontent.com/openstack/requirements/stable/pike/upper-constraints.txt
 135 .. _Xtesting: http://xtesting.readthedocs.io/en/latest/
 136
 137 Key benefits
 138 ------------
 139
 140  * the enduser can run all tests by setting only one input (EXTERNAL_NETWORK)
 141  * the developer can only work on the test suites without diving into CI/CD
 142    integration
 143  * both OpenStack and Kubernetes deployments can be verified
 144  * Functest test cases are trustable as they meet the best coding rules (unit
 145    tests, coverage, linters, etc.)
 146  * Functest can be reused in other projects (e.g.
 147    `ONS: Re-using OPNFV framework tests for LFN projects`_)
 148
 149 .. _`ONS: Re-using OPNFV framework tests for LFN projects`: https://wiki.lfnetworking.org/display/LN/LFN+Developer+Forum+Schedule?preview=/328197/328329/ONS-OPNFV%20framework%20tests%20for%20LFN%20projects.pdf
 150
 151 Code quality
 152 ------------
 153
 154  * pylint: ~9.5/10
 155  * code coverage: ~70%
 156
 157 Useful links
 158 ============
 159
 160  * wiki project page: https://wiki.opnfv.org/opnfv_functional_testing
 161  * Functest git repository: https://git.opnfv.org/cgit/functest
 162  * Functest CI dashboard: https://build.opnfv.org/ci/view/functest/
 163  * JIRA dashboard: https://jira.opnfv.org/secure/Dashboard.jspa?selectPageId=10611
 164  * Functest IRC channel: #opnfv-functest
 165  * Reporting page: http://testresults.opnfv.org/reporting/master/functest/functest.html