Update userguide

Change-Id: I2725ffcb4101496c2019d22215b1585fa3444775
Signed-off-by: Linda Wang <wangwulin@huawei.com>

diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst
index aa0661e..fe7e6a9 100644 (file)
--- a/docs/testing/user/userguide/test_details.rst
+++ b/docs/testing/user/userguide/test_details.rst
@@ -164,6 +164,8 @@ Tiers:
   * Smoke Tier - Test Case 'tempest_smoke_serial'
   * Components Tier - Test case 'tempest_full_parallel'
   * Neutron Trunk Port - Test case 'neutron_trunk'
   * Smoke Tier - Test Case 'tempest_smoke_serial'
   * Components Tier - Test case 'tempest_full_parallel'
   * Neutron Trunk Port - Test case 'neutron_trunk'

+  * OpenStack interop testcases - Test case 'refstack_defcore'
+  * Testing and verifying RBAC policy enforcement - Test case 'patrole'

 
 NOTE: Test case 'tempest_smoke_serial' executes a defined set of tempest smoke
 tests with a single thread (i.e. serial mode). Test case
 
 NOTE: Test case 'tempest_smoke_serial' executes a defined set of tempest smoke
 tests with a single thread (i.e. serial mode). Test case


@@ -176,6 +178,16 @@ separated networks using a single NIC. The feature neutron trunk ports have
 been supported by Apex, Fuel and Compass, so the tempest testcases have been
 integrated normally.
 
 been supported by Apex, Fuel and Compass, so the tempest testcases have been
 integrated normally.
 

+NOTE: Rally is also used to run Openstack Interop testcases `[9]`_, which focus
+on testing interoperability between OpenStack clouds.
+
+NOTE: Patrole is a tempest plugin for testing and verifying RBAC policy
+enforcement. It runs Tempest-based API tests using specified RBAC roles, thus
+allowing deployments to verify that only intended roles have access to those
+APIs. Patrole currently offers testing for the following OpenStack services:
+Nova, Neutron, Glance, Cinder and Keystone. Currently in functest, only neutron
+and glance are tested.
+

 The goal of the Tempest test suite is to check the basic functionalities of the
 different OpenStack components on an OPNFV fresh installation, using the
 corresponding REST API interfaces.
 The goal of the Tempest test suite is to check the basic functionalities of the
 different OpenStack components on an OPNFV fresh installation, using the
 corresponding REST API interfaces.


@@ -214,102 +226,15 @@ NOTE: Test case 'rally_sanity' executes a limited number of Rally smoke test
 cases. Test case 'rally_full' executes the full defined set of Rally tests.
 
 
 cases. Test case 'rally_full' executes the full defined set of Rally tests.
 
 

-Refstack-client to run OpenStack interop testcases
---------------------------------------------------
-
-Refstack-client `[8]`_ is a command line utility that allows you to
-execute Tempest test runs based on configurations you specify.
-It is the official tool to run Openstack Interop (previously known as Defcore)
-testcases `[9]`_, which focus on testing interoperability between OpenStack
-clouds.
-
-Refstack-client is integrated in Functest, consumed by Dovetail, which
-intends to define and provide a set of OPNFV related validation criteria
-that will provide input for the evaluation of the use of OPNFV trademarks.
-This progress is under the guideline of Compliance Verification Program(CVP).
-
-Running methods
-^^^^^^^^^^^^^^^
-
-Two running methods are provided after refstack-client integrated into
-Functest, Functest command line and manually, respectively.
-
-By default, for Defcore test cases run by Functest command line,
-are run followed with automatically generated
-configuration file, i.e., refstack_tempest.conf. In some circumstances,
-the automatic configuration file may not quite satisfied with the SUT,
-Functest also inherits the refstack-client command line and provides a way
-for users to set its configuration file according to its own SUT manually.
-
-*command line*
-
-Inside the Functest container, first to prepare Functest environment:
-
-::
-
-  functest env prepare
-
-then to run default defcore testcases by using refstack-client:
-
-::
-
-  functest testcase run refstack_defcore
-
-In OPNFV Continuous Integration(CI) system, the command line method is used.
-
-*manually*
-
-Prepare the tempest configuration file and the testcases want to run with the
-SUT, run the testcases with:
-
-::
-
-  ./refstack-client test -c <Path of the tempest configuration file to use> -v --test-list <Path or URL of test list>
-
-using help for more information:
-
-::
-
-  ./refstack-client --help
-  ./refstack-client test --help
-
-Reference tempest configuration
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-*command line method*
-
-When command line method is used, the default tempest configuration file
-is generated by Rally.
-
-*manually*
-
-When running manually is used, recommended way to generate tempest
-configuration file is:
-
-::
-
-  cd /usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/refstack_client
-  python tempest_conf.py
-
-a file called tempest.conf is stored in the current path by default, users can
-do some adjustment according to the SUT:
-
-::
-
-  vim refstack_tempest.conf
-
-a reference article can be used `[15]`_.
-
-

 snaps_smoke
 ------------
 
 This test case contains tests that setup and destroy environments with VMs with
 and without Floating IPs with a newly created user and project. Set the config
 value snaps.use_floating_ips (True|False) to toggle this functionality.
 snaps_smoke
 ------------
 
 This test case contains tests that setup and destroy environments with VMs with
 and without Floating IPs with a newly created user and project. Set the config
 value snaps.use_floating_ips (True|False) to toggle this functionality.

-Please note that when the configuration value of snaps.use_keystone is True,
+Please note that When the configuration value of snaps.use_keystone is True,

 Functest must have access the cloud's private network.
 Functest must have access the cloud's private network.

-This suite consists in 38 tests (test duration < 10 minutes)
+This suite consists in 120 tests (test duration ~= 50 minutes)

 
 
 SDN Controllers
 
 
 SDN Controllers


@@ -472,16 +397,6 @@ following section in
           module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf'
           class: 'CloudifyImsPerf'
 
           module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf'
           class: 'CloudifyImsPerf'
 

-orchestra_openims
-^^^^^^^^^^^^^^^^^
-Orchestra test case deals with the deployment of OpenIMS with OpenBaton
-orchestrator.
-
-orchestra_clearwaterims
-^^^^^^^^^^^^^^^^^^^^^^^
-Orchestra test case deals with the deployment of Clearwater vIMS with OpenBaton
-orchestrator.
-

 vyos-vrouter
 ^^^^^^^^^^^^
 This test case deals with the deployment and the test of vyos vrouter with
 vyos-vrouter
 ^^^^^^^^^^^^
 This test case deals with the deployment and the test of vyos vrouter with


@@ -501,6 +416,8 @@ The Workflow is as follows:
 
 The vyos-vrouter architecture is described in `[14]`_
 
 
 The vyos-vrouter architecture is described in `[14]`_
 

+juju_epc
+^^^^^^^^

 
 .. _`[2]`: http://docs.openstack.org/developer/tempest/overview.html
 .. _`[3]`: https://rally.readthedocs.org/en/latest/index.html
 
 .. _`[2]`: http://docs.openstack.org/developer/tempest/overview.html
 .. _`[3]`: https://rally.readthedocs.org/en/latest/index.html


@@ -512,4 +429,3 @@ The vyos-vrouter architecture is described in `[14]`_
 .. _`[12]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html
 .. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO
 .. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter
 .. _`[12]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html
 .. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO
 .. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter

-.. _`[15]`: https://aptira.com/testing-openstack-tempest-part-1/

diff --git a/docs/testing/user/userguide/test_overview.rst b/docs/testing/user/userguide/test_overview.rst
index 250a0f0..91eb343 100644 (file)
--- a/docs/testing/user/userguide/test_overview.rst
+++ b/docs/testing/user/userguide/test_overview.rst
@@ -67,6 +67,13 @@ validate the scenario for the release.
 |             |               | refstack   | Reference RefStack suite         |
 |             |               | \_defcore  | tempest selection for NFV        |
 |             |               +------------+----------------------------------+
 |             |               | refstack   | Reference RefStack suite         |
 |             |               | \_defcore  | tempest selection for NFV        |
 |             |               +------------+----------------------------------+

+|             |               | patrole    | Patrole is a tempest plugin for  |
+|             |               |            | testing and verifying RBAC policy|
+|             |               |            | enforcement, which offers testing|
+|             |               |            | for the following OpenStack      |
+|             |               |            | services: Nova, Neutron, Glance, |
+|             |               |            | Cinder and Keystone              |
+|             +---------------+------------+----------------------------------+

 |             |               | neutron    | The neutron trunk port testcases |
 |             |               | \_trunk    | have been introduced and they are|
 |             |               |            | supported by installers :        |
 |             |               | neutron    | The neutron trunk port testcases |
 |             |               | \_trunk    | have been introduced and they are|
 |             |               |            | supported by installers :        |


@@ -146,6 +153,9 @@ validate the scenario for the release.
 |             |               | vyos       | vRouter testing                  |
 |             |               | \_vrouter  |                                  |
 |             |               +------------+----------------------------------+
 |             |               | vyos       | vRouter testing                  |
 |             |               | \_vrouter  |                                  |
 |             |               +------------+----------------------------------+

+|             |               | juju_epc   | vEPC validation with Juju as VNF |
+|             |               |            | manager and ABoT as test executor|
+|             |               +------------+----------------------------------+

 |             |               | cloudify   | Based on cloudify_ims test case  |
 |             |               | \_ims_perf | cloudify_ims_perf substitutes    |
 |             |               |            | the signaling test suite by an   |
 |             |               | cloudify   | Based on cloudify_ims test case  |
 |             |               | \_ims_perf | cloudify_ims_perf substitutes    |
 |             |               |            | the signaling test suite by an   |