docs/release/release-notes/functest-release.rst

   1 .. SPDX-License-Identifier: CC-BY-4.0
   2
   3 ======================================
   4 OPNFV Fraser release note for Functest
   5 ======================================
   6
   7 Abstract
   8 ========
   9
  10 This document contains the release notes of the Functest project.
  11
  12 OPNFV Fraser Release
  13 ====================
  14
  15 Functest deals with functional testing of the OPNFV solution.
  16 It includes test cases developed within the project, test cases developed in
  17 other OPNFV projects and it also integrates test cases from other upstream
  18 communities.
  19
  20 OpenStack
  21 ---------
  22
  23 The internal test cases are:
  24
  25  * connection_check
  26  * api_check
  27  * snaps_health_check
  28  * vping_ssh
  29  * vping_userdata
  30  * tempest_smoke_serial
  31  * rally_sanity
  32  * refstack_defcore
  33  * patrole
  34  * odl
  35  * snaps_smoke
  36  * neutron_trunk
  37  * tempest_full_parallel
  38  * rally_full
  39  * cloudify_ims
  40  * vyos_vrouter
  41  * juju_epc
  42
  43 The OPNFV projects integrated into Functest framework for automation are:
  44
  45  * doctor
  46  * bgpvpn
  47  * odl-sfc
  48  * barometer
  49  * fds
  50  * parser
  51
  52 Kubernetes
  53 ----------
  54
  55 The internal test cases are:
  56
  57  * k8s_smoke
  58  * k8s_conformance
  59
  60 The OPNFV projects integrated into Functest framework for automation are:
  61
  62  * stor4nfv
  63  * clover
  64
  65 Release Data
  66 ============
  67
  68 +--------------------------------------+--------------------------------------+
  69 | **Project**                          | functest                             |
  70 +--------------------------------------+--------------------------------------+
  71 | **Release designation**              | Fraser                               |
  72 +--------------------------------------+--------------------------------------+
  73 | **Repository branch**                | stable/fraser                        |
  74 +--------------------------------------+--------------------------------------+
  75 | **Repository tag**                   | opnfv-6.1.0                          |
  76 +--------------------------------------+--------------------------------------+
  77 | **Release date**                     | May 25th 2018                        |
  78 +--------------------------------------+--------------------------------------+
  79
  80 Deliverables
  81 ============
  82
  83 Software
  84 --------
  85
  86  Functest Docker images (OpenStack):
  87
  88  * https://hub.docker.com/r/opnfv/functest-healthcheck
  89  * https://hub.docker.com/r/opnfv/functest-smoke
  90  * https://hub.docker.com/r/opnfv/functest-features
  91  * https://hub.docker.com/r/opnfv/functest-components
  92  * https://hub.docker.com/r/opnfv/functest-vnf
  93  * https://hub.docker.com/r/opnfv/functest-parser
  94
  95  Functest Docker images (Kubernetes):
  96
  97  * https://hub.docker.com/r/opnfv/functest-kubernetes-healthcheck
  98  * https://hub.docker.com/r/opnfv/functest-kubernetes-smoke
  99  * https://hub.docker.com/r/opnfv/functest-kubernetes-features
 100
 101 Docker tag for Fraser release: opnfv-6.1.0
 102 Docker tag for Fraser series: fraser
 103
 104 Documents
 105 ---------
 106
 107  * Config Guide: http://docs.opnfv.org/en/stable-fraser/submodules/functest/docs/testing/user/configguide/index.html
 108  * User Guide: http://docs.opnfv.org/en/stable-fraser/submodules/functest/docs/testing/user/userguide/index.html
 109  * Developer Guide: http://docs.opnfv.org/en/stable-fraser/submodules/functest/docs/testing/developer/devguide/index.html
 110  * API Docs: http://functest.readthedocs.io/en/latest/
 111
 112 Version change
 113 ==============
 114
 115 New test cases
 116 --------------
 117
 118  * neutron_trunk
 119  * patrole
 120  * juju_epc
 121  * k8s_smoke
 122  * k8s_conformance
 123  * stor4nfv
 124  * clover
 125
 126 Key changes
 127 -----------
 128
 129  * update test cases and containers to `OpenStack Pike`_
 130  * move the framework into a separate project: Xtesting_
 131  * ease running all containers thanks to default values
 132  * clean interfaces with OPNFV Installers and Features
 133  * rewrite all vnfs to allow multiple tests in parallel
 134  * fully support non-default region names and Keystone v3 domains
 135  * refactor all tempest-based test cases (e.g. refstack_defcore)
 136  * remove obsolete OpenStack and Functest utils
 137  * verify all changes via doc8 and yamllint too
 138  * generate reports for all tempest-based test cases
 139
 140 .. _`OpenStack Pike`: https://raw.githubusercontent.com/openstack/requirements/stable/pike/upper-constraints.txt
 141 .. _Xtesting: http://xtesting.readthedocs.io/en/latest/
 142
 143 Key benefits
 144 ------------
 145
 146  * the enduser can run all tests by setting only one input (EXTERNAL_NETWORK)
 147  * the developer can only work on the test suites without diving into CI/CD
 148    integration
 149  * both OpenStack and Kubernetes deployments can be verified
 150  * Functest test cases are trustable as they meet the best coding rules (unit
 151    tests, coverage, linters, etc.)
 152  * Functest can be reused in other projects (e.g.
 153    `ONS: Re-using OPNFV framework tests for LFN projects`_)
 154
 155 .. _`ONS: Re-using OPNFV framework tests for LFN projects`: https://wiki.lfnetworking.org/display/LN/LFN+Developer+Forum+Schedule?preview=/328197/328329/ONS-OPNFV%20framework%20tests%20for%20LFN%20projects.pdf
 156
 157 Code quality
 158 ------------
 159
 160  * pylint: ~9.5/10
 161  * code coverage: ~70%
 162
 163 Useful links
 164 ============
 165
 166  * wiki project page: https://wiki.opnfv.org/opnfv_functional_testing
 167  * Functest git repository: https://git.opnfv.org/cgit/functest
 168  * Functest CI dashboard: https://build.opnfv.org/ci/view/functest/
 169  * JIRA dashboard: https://jira.opnfv.org/secure/Dashboard.jspa?selectPageId=10611
 170  * Functest IRC channel: #opnfv-functest
 171  * Reporting page: http://testresults.opnfv.org/reporting/fraser/functest/functest.html