Add user guide for using ansible-playbook to run QTIP benchmarks

Change-Id: Ic04f83d91447944ed0fa507d6fffeed0a18b65fe
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

diff --git a/docs/testing/user/userguide/ansible.rst b/docs/testing/user/userguide/ansible.rst
new file mode 100644 (file)
index 0000000..ff2cb0a
--- /dev/null
+++ b/docs/testing/user/userguide/ansible.rst
@@ -0,0 +1,151 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+****************
+Run with Ansible
+****************
+
+QTIP benchmarking tasks are built upon `Ansible`_ playbooks and roles. If you are familiar with Ansible, it is possible
+to run it with ``ansible-playbook`` command.
+
+.. _Ansible: https://www.ansible.com/
+
+Create workspace
+================
+
+There is a playbook in ``tests/integration`` used for creating a new workspace for QTIP benchmarking::
+
+    cd qtip/tests/integration
+    ansible-playbook workspace-create.yml
+
+You will be prompted for required information for the Pod under test::
+
+    (qtip) ➜  integration git:(master) ✗ ansible-playbook workspace-create.yml
+    name of the pod under test (used in reporting) [qtip-pod]:
+    scenario deployed in the pod: [default]:
+    installer type of the pod (apex|fuel|other) [fuel]:
+    master host/vm of the installer (accessible by `ssh <hostname>`): f5
+    workspace name (new directory will be created) [workspace]:
+
+    PLAY [localhost] ***************************************************************
+
+    TASK [qtip-workspace : generating workspace] ***********************************
+
+    PLAY RECAP *********************************************************************
+    localhost                  : ok=0    changed=0    unreachable=0    failed=0
+
+
+You may hit ``Enter`` to accept default value
+
+NOTE: if this playbook is moved to other directory, configuration in ``ansible.cfg`` needs to be updated accordingly.
+The ansible roles from QTIP, i.e. ``<path_of_qtip>/resources/ansible_roles`` must be added to ``roles_path`` in
+`Ansible configuration file`_. For example::
+
+    roles_path = ../../resources/ansible_roles
+
+.. _Ansible configuration file:
+
+Executing benchmark
+===================
+
+Before executing the setup playbook, make sure `~/.ssh/config` has been configured properly so that you can login the
+**master node** "directly". Skip next section, if you can login with ``ssh <master-host>`` from localhost,
+
+SSH access to master node
+-------------------------
+
+It is common that the master node is behind some jump host. In this case, ssh option ``ProxyCommand`` and ``ssh-agent``
+shall be required.
+
+Assume that you need to login to deploy server, then login to the master node from there. An example configuration is
+as following::
+
+    Host fuel-deploy
+      HostName 172.50.0.250
+      User root
+
+    Host fuel-master
+      HostName 192.168.122.63
+      User root
+      ProxyCommand ssh -o 'ForwardAgent yes' apex-deploy 'ssh-add && nc %h %p'
+
+If several jumps are required to reach the master node, we may chain the jump hosts like below::
+
+    Host jumphost
+      HostName 10.62.105.31
+      User zte
+      Port 22
+
+    Host fuel-deploy
+      HostName 172.50.0.250
+      User root
+      ProxyJump jumphost
+
+    Host fuel-master
+      HostName 192.168.122.63
+      User root
+      ProxyCommand ssh -o 'ForwardAgent yes' apex-deploy 'ssh-add && nc %h %p'
+
+NOTE: ``ProxyJump`` is equivalent to the long ``ProxyCommand`` option, but it is only available since OpenSSH 7.3
+
+Setup testing environment
+-------------------------
+
+Run the setup playbook to generate ansible inventory of system under test by querying the slave nodes from the installer
+master::
+
+    cd workspace
+    ansible-playbook setup.yml
+
+Currently, QTIP supports automatic discovery from `apex`_ and `fuel`_
+
+.. _apex: https://wiki.opnfv.org/display/apex
+.. _fuel: https://wiki.opnfv.org/display/fuel
+
+It will update the ``hosts`` and ``ssh.cfg``
+
+Run the tests
+-------------
+
+Run the benchmarks with the following command::
+
+    ansible-playbook run.yml
+
+CAVEAT: QTIP will install required packages in system under test.
+
+Inspect the results
+-------------------
+
+The test results and calculated output are stored in ``results``::
+
+    current/
+        node-2/
+            arithmetic/
+                metric.json
+                report
+                unixbench.log
+            dpi/
+            ...
+        node-4/
+        ...
+        qtip-pod-qpi.json
+    qtip-pod-20170425-1710/
+    qtip-pod-20170425-1914/
+    ...
+
+The folders are named as ``<pod_name>-<start_time>/`` and the results are organized by *hosts* under test. Inside each
+host, the test data are organized by metrics as defined in QPI specification.
+
+For each metrics, it usually includes the following content
+
+* log file generated by the performance testing tool
+* metrics collected from the log files
+* reported rendered with the metrics collected
+
+Teardown the test environment
+-----------------------------
+
+QTIP will create temporary files for testing in system under test. Execute the teardown playbook to clean it up::
+
+    ansible-playbook teardown.yml

diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst
index d0d555f..dd54e57 100644 (file)
--- a/docs/testing/user/userguide/index.rst
+++ b/docs/testing/user/userguide/index.rst
@@ -12,6 +12,7 @@ QTIP User Guide
    :maxdepth: 2
 
    overview.rst
+   ansible.rst
    cli.rst
    api.rst
    compute.rst

diff --git a/tests/integration/workspace-create.yml b/tests/integration/workspace-create.yml
index 194747f..741d1df 100644 (file)
--- a/tests/integration/workspace-create.yml
+++ b/tests/integration/workspace-create.yml
@@ -15,28 +15,28 @@
   vars_prompt:
 
     - name: pod_name
-      prompt: 'name of the pod under test:'
+      prompt: 'name of the pod under test (used in reporting)'
       default: qtip-pod
       private: no
 
     - name: scenario
       prompt: 'scenario deployed in the pod:'
-      default: generic
+      default: default
       private: no
 
     - name: installer
-      prompt: 'installer type of the pod (apex|fuel):'
+      prompt: 'installer type of the pod (apex|fuel|other)'
       default: fuel
       private: no
 
     - name: installer_master_host
-      prompt: 'master host of the installer:'
-      default: master
+      prompt: 'master host/vm of the installer (accessible by `ssh <hostname>`)'
       private: no
 
     - name: workspace
-      prompt: 'workspace name:'
+      prompt: 'workspace name (new directory will be created)'
       default: workspace
+      private: no
 
   roles: