Modifed documentation to generate improved PDFs in tool chain

Modified the location for documentation files within the docs directory.
Index files modified to generate better PDFs

Change-Id: Ie21b1021a8d09013df48afc6d737d95ee8aeed92
Signed-off-by: Nauman_Ahad <nauman_ahad@xflowresearch.com>
(cherry picked from commit 6700444ea735f678ed2841fec29ab11947905a1a)

diff --git a/docs/compute_testcases.rst b/docs/compute_testcases.rst
deleted file mode 100644 (file)
index c8e5212..0000000
--- a/docs/compute_testcases.rst
+++ /dev/null
@@ -1,74 +0,0 @@
-Qtip Compute testcases
-======================
-============
-Introduction
-============
-
-The QTIP testing suite aims to benchmark the compute components of an OPNFV platform.
-Such components include, the CPU performance, the memory performance.
-Additionally virtual computing performance provided by the Hypervisor (KVM) installed as part of OPNFV platforms would be benhmarked too.
-
-The test suite consists of both synthetic and application specific benchmarks to test compute components.
-
-All the compute benchmarks could be run in 2 scenarios:
-
-1. On Baremetal Machines provisioned by an OPNFV installer (Host machines)
-2. On Virtual Machines brought up through OpenStack on an OPNFV platform
-
-Note: The Compute benchmark suite constains relatively old benchmarks such as dhrystone and whetstone. The suite would be updated for better benchmarks such as Linbench for the OPNFV C release.
-
-============
-Benchmarks
-============
-
-
-The benchmarks include:
-
-1. Dhrystone 2.1
-
-Dhrystone is a synthetic benchmark for measuring CPU performance. It uses integer calculations to evaluate CPU capabilities.
-Both Single CPU performance is measured along multi-cpu performance.
-
-
-Dhrystone, however, is a dated benchmark and has some short comings.
-Written in C, it is a small program that doesn't test the CPU memory subsystem. Additionally, dhrystone results could be modified by optimizing the compiler and insome cases hardware configuration.
-
-References: http://www.eembc.org/techlit/datasheets/dhrystone_wp.pdf
-
-2. Whetstone
-
-Whetstone is a synthetic benchmark to measure CPU floating point operation performance. Both Single CPU performance is measured along multi-cpu performance.
-
-Like Dhrystone, Whetstone is a dated benchmark and has short comings.
-
-References: http://www.netlib.org/benchmark/whetstone.c
-
-3. OpenSSL Speed
-
-OpenSSL Speed can be used to benchmark compute performance of a machine. In QTIP, two OpenSSL Speed benchmarks are incorporated:
-1. RSA signatures/sec signed by a machine
-2. AES 128-bit encryption throught for a machine for cipher block sizes
-
-References: https://www.openssl.org/docs/manmaster/apps/speed.html
-
-4. RAMSpeed
-
-RAMSpeed is used to measure a machine's memory perfomace. The problem(array)size is large enough to ensure Cache Misses so that the main machine memory is used. INTmem and FLOATmem benchmarks are executed in 4 different scenarios:
-
-a. Copy: a(i)=b(i)
-b. Add:  a(i)=b(i)+c(i)
-c. Scale:  a(i)=b(i)*d
-d. Triad: a(i)=b(i)+c(i)*d
-
-INTmem uses integers in these four benchmarks whereas FLOATmem uses floating points for these benchmarks.
-
-References: http://alasir.com/software/ramspeed/, https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W51a7ffcf4dfd_4b40_9d82_446ebc23c550/page/Untangling+memory+access+measurements
-
-5. DPI
-
-nDPI is a modified  variant of  OpenDPI, Open source Deep packet Inspection, that is maintained by ntop.
-An example application called *pcapreader* has been developed and is available for use along nDPI.
-
-A sample .pcap file is passed to the *pcapreader* application. nDPI classifies traffic in the pcap file into different categories based on string matching . The *pcapreader* application provides a throughput number for the rate at which traffic was classified, indicating a machine's computational performance. The results are run 10 times and an average is taken for the obtained number.
-
-References: http://www.ntop.org/products/deep-packet-inspection/ndpi/, http://www.ntop.org/wp-content/uploads/2013/12/nDPI_QuickStartGuide.pdf

diff --git a/docs/how-to-use-docs/01-introduction.rst b/docs/how-to-use-docs/01-introduction.rst
deleted file mode 100644 (file)
index 74587e8..0000000
--- a/docs/how-to-use-docs/01-introduction.rst
+++ /dev/null
@@ -1,30 +0,0 @@
-============\r
-Introduction\r
-============\r
-\r
-**Welcome to QTIP's documentation !**\r
-\r
-.. _QTIP: https://wiki.opnfv.org/yardstick\r
-\r
-QTIP_ is an OPNFV Project.\r
-\r
-QTIP aims to benchmark OPNFV platforms through a "Bottom up" approach, testing bare-metal components first.\r
-\r
-The overall problem this project tries to solve is the general characterization of an OPNFV platform. It will focus on general performance questions that are common to the platform itself, or applicable to multiple OPNFV use cases. QTIP will provide the capability to quantify a platform's performance behavior in a standardized, rigorous, and open way, and a well-documented methodology to reproduce the results by anyone interested.\r
-\r
-The chapter :doc:`02-methodology` describes the methodology implemented by the\r
-QTIP Project for NFVI performance benchmarking. The chapter\r
-:doc:`03-list-of-testcases` includes a list of available Yardstick test cases.\r
-\r
-The *QTIP* framework is deployed in the Dell OPNFV community lab. It is\r
-infrastructure and application independent.\r
-\r
-.. _Pharos: https://wiki.opnfv.org/pharos\r
-.. seealso:: Pharos_ for information on OPNFV community labs.\r
-\r
-Contact QTIP\r
-=================\r
-\r
-Feedback? `Contact us`_\r
-\r
-.. _Contact us: opnfv-users@lists.opnfv.org\r

diff --git a/docs/network_testcases.rst b/docs/network_testcases.rst
deleted file mode 100644 (file)
index 1c6fb91..0000000
--- a/docs/network_testcases.rst
+++ /dev/null
@@ -1,42 +0,0 @@
-NETWORK THROUGHPUT TESTCASE\r
-\r
-QTIP uses IPerf3 as the main tool for testing the network throughput.\r
-There are two tests that are run through the QTIP framework.\r
-\r
-Network Throughput for VMs\r
-Network Throughput for Compute Nodes\r
-\r
-For the throughout of the compute nodes we simply go into the systems-under-test\r
- and install iperf3 on the nodes. One of the SUTs is used a server and the\r
- other as a client. The client pushes traffic to the server for a duration specified by\r
- the user\r
- configuration file for iperf. These files can be found in the test_cases/{POD}/network/\r
- directory. The bandwidth is limited only by the physical link layer speed available to the server.\r
- The result file inlcudes the b/s bandwidth and the CPU usage for both the client and server.\r
-\r
-For the VMs we are running two topologies through the framework.\r
-\r
-1: VMs on the same compute nodes\r
-2: VMs on different compute nodes\r
-\r
-QTIP framework sets up a stack with a private network, security groups, routers and attaches the VMs to this network. Iperf3 is installed\r
- on the VMs and one is assigned the role of client while other serves as a server. Traffic is pushed\r
- over the QTIP private network between the VMs. A closer look in needed to see how the traffic actually\r
- flows between the VMs in this configuration to understand what is happening to the packet as traverses\r
- the openstack network.\r
-\r
-The packet originates from VM1 and its sent to the linux bridge via a tap interface where the security groups\r
- are written. Afterwards the packet is forwarded to the Integration bridge via a patch port. Since VM2 is also connected\r
- to the Integration bridge in a similar manner as VM1 so the packet gets forwarded to the linux bridge connecting\r
- VM2. After the linux bridge the packet is sent to VM2 and is recieved by the Iperf3 server. Since no physical link is\r
- involved in this topology, only the OVS (Integration bridge) is being benchmarked and we are seeing bandwidth in the range\r
- of 14-15 Gbps.\r
-\r
-For the topology where the VMs are spawned on different compute nodes, the path the packet takes becomes more cumbersome.\r
-The packet leaves a VM and makes its way to the Integration Bridge as in the first topology however the integration bridge \r
-forwards the packet to the physical link through the ethernet bridge. The packet then gets a VLAN/Tunnel depending on the network\r
-and is forwarded to the particular Compute node where the second VM is spwaned. The packets enter the compute node through the physical\r
-ethernet port and makes its way to the VM through the integration bridge and linux bridge. As seen here the path is much more involved\r
-even when discussed without the mention of overheads faced at all the internfaces so we are seeing the results in the range of 2 Gbps.\r
-\r
-\r

diff --git a/docs/qtip/index.rst b/docs/qtip/index.rst
deleted file mode 100644 (file)
index a8cb43e..0000000
--- a/docs/qtip/index.rst
+++ /dev/null
@@ -1,32 +0,0 @@
-.. OPNFV Release Engineering documentation, created by
-   sphinx-quickstart on Tue Jun  9 19:12:31 2015.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
-.. image:: ../etc/opnfv-logo.png
-  :height: 40
-  :width: 200
-  :alt: OPNFV
-  :align: left
-
-Example Documentation table of contents
-=======================================
-
-Contents:
-
-.. toctree::
-   :numbered:
-   :maxdepth: 4
-
-   opnfv_qtip_tc001.rst
-   opnfv_qtip_tc002.rst
-   opnfv_qtip_tc003.rst
-
-Indices and tables
-==================
-
-* :ref:`search`
-
-Revision: _sha1_
-
-Build date: |today|

diff --git a/docs/qtip/opnfv_qtip_tc001.rst b/docs/qtip/opnfv_qtip_tc001.rst
deleted file mode 100644 (file)
index e69de29..0000000

diff --git a/docs/qtip/opnfv_qtip_tc002.rst b/docs/qtip/opnfv_qtip_tc002.rst
deleted file mode 100644 (file)
index e69de29..0000000

diff --git a/docs/qtip/opnfv_qtip_tc003.rst b/docs/qtip/opnfv_qtip_tc003.rst
deleted file mode 100644 (file)
index e69de29..0000000

diff --git a/docs/storage_testcases.rst b/docs/storage_testcases.rst
deleted file mode 100644 (file)
index 50317ad..0000000
--- a/docs/storage_testcases.rst
+++ /dev/null
@@ -1,18 +0,0 @@
-QTIP Storage testcases
-===================
-
-The QTIP benchmark suite aims to evaluate storage components within an OPNFV platform. For Brahamaputra release, FIO would evaluate File System performance for the host machine. It will also test the I/O performance provided by the hypervisor(KVM) when Storage benchmarks are run inside VMs.
-
-**Note: For Brahmaputra release, only the Ephemeral Storage is being tested. For C release persistent block and object storage would be tested.**
-
-The FIO Job would consist of:
-
-1. A file size of 5GB
-2. Random Read 50%, Random Write 50%
-3. Direct I/O
-4. Asynch I/O Engine
-5. I/O Queue depth of 2
-6. Block size :4K
-
-
-For this Job, I/O per second would be measured along mean I/O latency to provide storage performance numbers.

diff --git a/docs/user_guides/01-introduction.rst b/docs/user_guides/01-introduction.rst
new file mode 100644 (file)
index 0000000..6322d09
--- /dev/null
+++ b/docs/user_guides/01-introduction.rst
@@ -0,0 +1,40 @@
+Introduction
+============
+
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) <optionally add copywriters name>
+.. two dots create a comment. please leave this logo at the top of each of your rst files.
+.. image:: ../etc/opnfv-logo.png
+  :height: 40
+  :width: 200
+  :alt: OPNFV
+  :align: left
+.. these two pipes are to seperate the logo from the first title
+
+|
+
+**Welcome to QTIP's documentation !**
+
+.. _QTIP: https://wiki.opnfv.org/platform_performance_benchmarking
+
+QTIP_ is an OPNFV Project.
+
+QTIP aims to benchmark OPNFV platforms through a "Bottom up" approach, testing bare-metal components first.
+
+The overall problem this project tries to solve is the general characterization of an OPNFV platform.
+It will focus on general performance questions that are common to the platform itself, or applicable to multiple OPNFV use cases.
+QTIP will provide the capability to quantify a platform's performance behavior in a standardized, rigorous, and open way.
+
+The *QTIP* framework is deployed in the Dell OPNFV community lab. It is
+infrastructure and application independent.
+
+.. _Pharos: https://wiki.opnfv.org/pharos
+.. seealso:: Pharos_ for information on OPNFV community labs.
+
+**Contact QTIP**
+
+Feedback? `Contact us`_
+
+.. _Contact us: opnfv-users@lists.opnfv.org
+

diff --git a/docs/user_guides/framework/installation.rst b/docs/user_guides/02-installation.rst
similarity index 85%
rename from docs/user_guides/framework/installation.rst
rename to docs/user_guides/02-installation.rst
index c1aa8d9..2f2ecf9 100644 (file)
--- a/docs/user_guides/framework/installation.rst
+++ b/docs/user_guides/02-installation.rst
@@ -1,14 +1,18 @@
-..
-   TODO As things will change, then this document has to be revised before the
-   next release. Steps:
-   1. Verify that the instructions below are correct and have not been changed.
-   2. Add everything that is currently missing and should be included in this document.
-   3. Make sure each title has a paragraph or an introductory sentence under it.
-   4. Make sure each sentence is grammatically correct and easily understandable.
-   5. Remove this comment section.
-
 Installation
-==============
+============
+
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) <optionally add copywriters name>
+.. two dots create a comment. please leave this logo at the top of each of your rst files.
+.. image:: ../etc/opnfv-logo.png
+  :height: 40
+  :width: 200
+  :alt: OPNFV
+  :align: left
+.. these two pipes are to seperate the logo from the first title
+
+|
 
 QTIP currently supports by using a Docker image or by pulling the repo from the upstream
 repository found at https://git.opnfv.org/qtip. Detailed steps about setting up QTIP using both of these options
@@ -95,7 +99,7 @@ Verify that opnfv/qtip has been downloaded. It should be listed as an image by r
 ::
 
   sudo docker images
- 
+
 Run the Docker instance:
 ::
 
@@ -129,8 +133,9 @@ For running QTIP manually, it is also necessary to export the installer type. ::
 QTIP  default key pair
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 QTIP uses a SSH key pair to connect to the guest image. This key pair can
-be found in the ``data/`` directory. 
+be found in the ``data/`` directory.
 
 Examples
 --------
 QTIP Has been made with the intention of requiring minimal interaction from the user.
+

diff --git a/docs/how-to-use-docs/03-usage-guide.rst b/docs/user_guides/03-usage-guide.rst
similarity index 90%
rename from docs/how-to-use-docs/03-usage-guide.rst
rename to docs/user_guides/03-usage-guide.rst
index 2829d66..69eed34 100644 (file)
--- a/docs/how-to-use-docs/03-usage-guide.rst
+++ b/docs/user_guides/03-usage-guide.rst
@@ -1,313 +1,317 @@
-..\r
-   TODO As things will change, then this document has to be revised before the\r
-   next release. Steps:\r
-   1. Verify that the instructions below are correct and have not been changed.\r
-   2. Add everything that is currently missing and should be included in this document.\r
-   3. Make sure each title has a paragraph or an introductory sentence under it.\r
-   4. Make sure each sentence is grammatically correct and easily understandable.\r
-   5. Remove this comment section.\r
-\r
-Guide to run QTIP:\r
-==================\r
-\r
-This guide will serve as a first step to familiarize the user with how to\r
-run QTIP the first time when the user clones QTIP on to their host machine.\r
-In order to clone QTIP please follow the instructions in the\r
-installation.rst located in docs/userguide/installation.rst.\r
-\r
-QTIP Directory structure:\r
--------------------------\r
-\r
-The QTIP directory has been sectioned off into multiple folders to facilitate\r
- segmenting information into relevant categories. The folders that concern\r
- the end user are `test_cases/` and `test_list/`.\r
-\r
-test_cases/:\r
-------------\r
-\r
-This folder is used to store all the config files which are used to setup the\r
- environment prior to a test. This folder is further divided into opnfv pods\r
- which run QTIP. Inside each pod there are folders which contain the config\r
- files segmented based on test cases. Namely, these include, `Compute`,\r
- `Network` and `Storage`. The default folder is there for the end user who\r
- is interested in testing their infrastructure but arent part of a opnfv pod.\r
-\r
-The structure of the directory for the user appears as follows\r
-::\r
-\r
-  test_cases/default/compute\r
-  test_cases/default/network\r
-  test_cases/default/storage\r
-\r
-The benchmarks that are part of the QTIP framework are listed under these\r
-folders. An example of the compute folder is shown below.\r
-Their naming convention is <BENCHMARK>_<VM/BM>.yaml\r
-::\r
-\r
-  dhrystone_bm.yaml\r
-  dhrystone_vm.yaml\r
-  whetstone_vm.yaml\r
-  whetstone_bm.yaml\r
-  ssl_vm.yaml\r
-  ssl_bm.yaml\r
-  ramspeed_vm.yaml\r
-  ramspeed_bm.yaml\r
-  dpi_vm.yaml\r
-  dpi_bm.yaml\r
-\r
-The above listed files are used to configure the environment. The VM/BM tag\r
-distinguishes between a test to be run on the Virtual Machine or the compute\r
-node itself, respectively.\r
-\r
-\r
-test_list/:\r
------------\r
-\r
-This folder contains three files, namely `compute`, `network` and `storage`.\r
-These files list the benchmarks are to be run by the QTIP framework. Sample\r
-compute test file is shown below\r
-::\r
-\r
-  dhrystone_vm.yaml\r
-  dhrystone_bm.yaml\r
-  whetstone_vm.yaml\r
-  ssl_bm.yaml\r
-\r
-The compute file will now run all the benchmarks listed above one after\r
-another on the environment. `NOTE: Please ensure there are no blank lines\r
-in this file as that has been known to throw an exception`.\r
-\r
-Preparing a config file for test:\r
----------------------------------\r
-\r
-We will be using dhrystone as a example to list out the changes that the\r
-user will need to do in order to run the benchmark.\r
-Dhrystone on Compute Nodes:\r
-^^^^^^^^^^^^^^^^^^^^^^^^^^^\r
-\r
-QTIP framework can run benchmarks on the actual compute nodes as well. In\r
-order to run dhrystone on the compute nodes we will be editing the\r
-dhrystone_bm.yaml file.\r
-\r
-::\r
-\r
-  Scenario:\r
-    benchmark: dhrystone\r
-    host: machine_1, machine_2\r
-    server:\r
-\r
-The `Scenario` field is used by to specify the name of the benchmark to\r
-run as done by `benchmark: dhrystone`. The `host` and `server` tag are\r
-not used for the compute benchmarks but are included here to help the\r
-user `IF` they wish to control the execution. By default both machine_1\r
-and machine_2 will have dhrystone run on them in parallel but the user\r
-can change this so that machine_1 run dhrystone before machine_2. This\r
-will be elaborated in the `Context` tag.\r
-\r
-::\r
-\r
-  Context:\r
-    Host_Machines:\r
-      machine_1:\r
-        ip: 10.20.0.6\r
-        pw:\r
-        role: host\r
-      machine_2:\r
-        ip: 10.20.0.5\r
-        pw:\r
-        role: host\r
-\r
-     Virtual_Machines:\r
-\r
-The `Context` tag helps the user list the number of compute nodes they want\r
- to run dhrystone on. The user can list all the compute nodes under the\r
- `Host_Machines` tag. All the machines under test must be listed under the\r
- `Host_Machines` and naming it incrementally higher. The `ip:` tag is used\r
- to specify the IP of the particular compute node. The `pw:` tag can be left\r
- blank because QTIP uses its own key for ssh. In order to run dhrystone on\r
- one compute node at a time the user needs to edit the `role:` tag. `role:\r
- host` for machine_1 and `role: server` for machine_2 will allow for\r
- dhrystone to be run on machine_1 and then run on machine_2.\r
-\r
-::\r
-\r
-\r
-  Test_Description:\r
-    Test_category: "Compute"\r
-    Benchmark: "dhrystone"\r
-    Overview: >\r
-        ''' This test will run the dhrystone benchmark in parallel  on\r
-        machine_1 and machine_2.\r
-\r
-The above field is purely for a description purpose to explain to the user\r
-the working of the test and is not fed to the framework.\r
-\r
-Sample dhrystone_bm.yaml file:\r
-------------------------------\r
-::\r
-\r
-  Scenario:\r
-    benchmark: dhrystone\r
-    host: machine_1, machine_2\r
-    server:\r
-\r
-  Context:\r
-    Host_Machines:\r
-      machine_1:\r
-        ip: 10.20.0.6\r
-        pw:\r
-        role: host\r
-      machine_2:\r
-        ip: 10.20.0.5\r
-        pw:\r
-        role: host\r
-\r
-    Virtual_Machines:\r
-\r
-\r
-  Test_Description:\r
-    Test_category: "Compute"\r
-    Benchmark: "dhrystone"\r
-    Overview: >\r
-        ''' This test will run the dhrystone benchmark in parallel  on\r
-        machine_1 and machine_2.\n\r
-\r
-Dhrystone on Virtual Machine:\r
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r
-To run dhrystone on the VMs we will be editing dhrystone_vm.yaml file.\r
-Snippets on the file are given below.\r
-\r
-::\r
-\r
-  Scenario:\r
-  benchmark: dhrystone\r
-  host: virtualmachine_1, virtualmachine_2\r
-  server:\r
-\r
-\r
-The `Scenario` field is used by to specify the name of the benchmark to\r
-run as done by `benchmark: dhrystone`. The `host` and `server` tag are\r
-not used for the compute benchmarks but are included here to help the\r
-user `IF` they wish to control the execution. By default both\r
-virtualmachine_1 and virtualmachine_2 will have dhrystone run on them\r
-in parallel but the user can change this so that virtualmachine_1 run\r
-dhrystone before virtualmachine_2. This will be elaborated in the\r
-`Context` tag.\r
-::\r
-\r
-  Context:\r
-    Host_Machines:\r
-\r
-    Virtual_Machines:\r
-      virtualmachine_1:\r
-        availability_zone: compute1\r
-        public_network: 'net04_ext'\r
-        OS_image: QTIP_CentOS\r
-        flavor: m1.large\r
-        role: host\r
-      virtualmachine_2:\r
-        availability_zone: compute2\r
-        public_network: 'net04_ext'\r
-        OS_image: QTIP_CentOS\r
-        flavor: m1.large\r
-        role: host\r
-\r
-The `Context` tag helps the user list the number of VMs and their\r
-characteristic. The user can list all the VMs they want to bring up\r
-under the `Virtual_Machines:` tag. In the above example we will be\r
-bringing up two VMs. One on Compute1 and the other on Compute2. The\r
-user can change this as desired `NOTE: Please ensure you have the\r
-necessary compute nodes before listing under the 'availability_zone:'\r
-tag`. The rest of the options do not need to be modified by the user.\r
-\r
-Running dhrystone sequentially (Optional):\r
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r
-\r
-In order to run dhrystone on one VM at a time the user needs to edit\r
-the `role:` tag. `role: host` for virtualmachine_1 and `role: server`\r
-for virtualmachine_2 will allow for dhrystone to be run on\r
-virtualmachine_1 and then run on virtualmachine_2.\r
-\r
-::\r
-\r
-  Test_Description:\r
-    Test_category: "Compute"\r
-    Benchmark: "dhrystone"\r
-    Overview:\r
-    This test will run the dhrystone benchmark in parallel on\r
-    virtualmachine_1 and virtualmachine_2\r
-\r
-The above field is purely for a decription purpose to explain to\r
-the user the working of the test and is not fed to the framework.\r
-\r
-Sample dhrystone_vm.yaml file:\r
-------------------------------\r
-::\r
-\r
-  Scenario:\r
-  benchmark: dhrystone\r
-  host: virtualmachine_1, virtualmachine_2\r
-  server:\r
-\r
-  Context:\r
-    Host_Machines:\r
-\r
-    Virtual_Machines:\r
-      virtualmachine_1:\r
-        availability_zone: compute1\r
-        public_network: 'net04_ext'\r
-        OS_image: QTIP_CentOS\r
-        flavor: m1.large\r
-        role: host\r
-      virtualmachine_2:\r
-        availability_zone: compute2\r
-        public_network: 'net04_ext'\r
-        OS_image: QTIP_CentOS\r
-        flavor: m1.large\r
-        role: host\r
-\r
-  Test_Description:\r
-    Test_category: "Compute"\r
-    Benchmark: "dhrystone"\r
-    Overview: >\r
-    This test will run the dhrystone benchmark in parallel on\r
-    machine_1 and machine_2.\n\r
-\r
-Commands to run the Framework:\r
-==============================\r
-\r
-In order to start QTIP on the default lab please use the following commands (asssuming you have prepared the config files in the test_cases/default/ directory and listed the intended suite in the test_list/<RELEVANT-SUITE-FILE>):\r
-\r
-First step is to export the necessary information to the environment.\r
-::\r
-\r
-  source get_env_info.sh -n <INSTALLER_TYPE> -i <INSTALLER_IP>\r
-\r
-for running qtip on an openstack deployed using FUEL with the Installer IP 10.20.0.2\r
-::\r
-\r
-   source get_env_info.sh -n fuel -i 10.20.0.2\r
-\r
-This will generate the `opnfv-creds.sh` file needed to use the python clients for keystone, glance, nova, and neutron.\r
-::\r
-\r
-  source opnfv-creds.sh\r
-\r
-Running QTIP on the using `default` as the pod name and for the `compute` suite\r
-::\r
-\r
-  python qtip.py -l default -f compute\r
-\r
-Running QTIP on the using `default` as the pod name and for the `network` suite\r
-::\r
-\r
-  python qtip.py -l default -f network\r
-\r
-Running QTIP on the using `default` as the pod name and for the `storage` suite\r
-::\r
-\r
-  python qtip.py -l default -f network\r
-\r
-Results:\r
-========\r
-QTIP generates results in the `results/` directory are listed down under the particularly benchmark name. So all the results for dhrystone would be listed and time stamped.
\ No newline at end of file
+Guide to run QTIP:
+==================
+
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) <optionally add copywriters name>
+.. two dots create a comment. please leave this logo at the top of each of your rst files.
+.. image:: ../etc/opnfv-logo.png
+  :height: 40
+  :width: 200
+  :alt: OPNFV
+  :align: left
+.. these two pipes are to seperate the logo from the first title
+
+|
+
+This guide will serve as a first step to familiarize the user with how to
+run QTIP the first time when the user clones QTIP on to their host machine.
+In order to clone QTIP please follow the instructions in the
+installation.rst located in docs/userguide/installation.rst.
+
+QTIP Directory structure:
+-------------------------
+
+The QTIP directory has been sectioned off into multiple folders to facilitate
+ segmenting information into relevant categories. The folders that concern
+ the end user are `test_cases/` and `test_list/`.
+
+**test_cases/:**
+
+This folder is used to store all the config files which are used to setup the
+ environment prior to a test. This folder is further divided into opnfv pods
+ which run QTIP. Inside each pod there are folders which contain the config
+ files segmented based on test cases. Namely, these include, `Compute`,
+ `Network` and `Storage`. The default folder is there for the end user who
+ is interested in testing their infrastructure but arent part of a opnfv pod.
+
+The structure of the directory for the user appears as follows
+::
+
+  test_cases/default/compute
+  test_cases/default/network
+  test_cases/default/storage
+
+The benchmarks that are part of the QTIP framework are listed under these
+folders. An example of the compute folder is shown below.
+Their naming convention is <BENCHMARK>_<VM/BM>.yaml
+::
+
+  dhrystone_bm.yaml
+  dhrystone_vm.yaml
+  whetstone_vm.yaml
+  whetstone_bm.yaml
+  ssl_vm.yaml
+  ssl_bm.yaml
+  ramspeed_vm.yaml
+  ramspeed_bm.yaml
+  dpi_vm.yaml
+  dpi_bm.yaml
+
+The above listed files are used to configure the environment. The VM/BM tag
+distinguishes between a test to be run on the Virtual Machine or the compute
+node itself, respectively.
+
+
+**test_list/:**
+
+This folder contains three files, namely `compute`, `network` and `storage`.
+These files list the benchmarks are to be run by the QTIP framework. Sample
+compute test file is shown below
+::
+
+  dhrystone_vm.yaml
+  dhrystone_bm.yaml
+  whetstone_vm.yaml
+  ssl_bm.yaml
+
+The compute file will now run all the benchmarks listed above one after
+another on the environment. `NOTE: Please ensure there are no blank lines
+in this file as that has been known to throw an exception`.
+
+Preparing a config file for test:
+---------------------------------
+
+We will be using dhrystone as a example to list out the changes that the
+user will need to do in order to run the benchmark.
+
+Dhrystone on Compute Nodes:
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+QTIP framework can run benchmarks on the actual compute nodes as well. In
+order to run dhrystone on the compute nodes we will be editing the
+dhrystone_bm.yaml file.
+
+::
+
+  Scenario:
+    benchmark: dhrystone
+    host: machine_1, machine_2
+    server:
+
+The `Scenario` field is used by to specify the name of the benchmark to
+run as done by `benchmark: dhrystone`. The `host` and `server` tag are
+not used for the compute benchmarks but are included here to help the
+user `IF` they wish to control the execution. By default both machine_1
+and machine_2 will have dhrystone run on them in parallel but the user
+can change this so that machine_1 run dhrystone before machine_2. This
+will be elaborated in the `Context` tag.
+
+::
+
+  Context:
+    Host_Machines:
+      machine_1:
+        ip: 10.20.0.6
+        pw:
+        role: host
+      machine_2:
+        ip: 10.20.0.5
+        pw:
+        role: host
+
+     Virtual_Machines:
+
+The `Context` tag helps the user list the number of compute nodes they want
+ to run dhrystone on. The user can list all the compute nodes under the
+ `Host_Machines` tag. All the machines under test must be listed under the
+ `Host_Machines` and naming it incrementally higher. The `ip:` tag is used
+ to specify the IP of the particular compute node. The `pw:` tag can be left
+ blank because QTIP uses its own key for ssh. In order to run dhrystone on
+ one compute node at a time the user needs to edit the `role:` tag. `role:
+ host` for machine_1 and `role: server` for machine_2 will allow for
+ dhrystone to be run on machine_1 and then run on machine_2.
+
+::
+
+
+  Test_Description:
+    Test_category: "Compute"
+    Benchmark: "dhrystone"
+    Overview: >
+        ''' This test will run the dhrystone benchmark in parallel  on
+        machine_1 and machine_2.
+
+The above field is purely for a description purpose to explain to the user
+the working of the test and is not fed to the framework.
+
+Sample dhrystone_bm.yaml file:
+------------------------------
+::
+
+  Scenario:
+    benchmark: dhrystone
+    host: machine_1, machine_2
+    server:
+
+  Context:
+    Host_Machines:
+      machine_1:
+        ip: 10.20.0.6
+        pw:
+        role: host
+      machine_2:
+        ip: 10.20.0.5
+        pw:
+        role: host
+
+    Virtual_Machines:
+
+
+  Test_Description:
+    Test_category: "Compute"
+    Benchmark: "dhrystone"
+    Overview: >
+        ''' This test will run the dhrystone benchmark in parallel  on
+        machine_1 and machine_2.\n
+
+Dhrystone on Virtual Machine:
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+To run dhrystone on the VMs we will be editing dhrystone_vm.yaml file.
+Snippets on the file are given below.
+
+::
+
+  Scenario:
+  benchmark: dhrystone
+  host: virtualmachine_1, virtualmachine_2
+  server:
+
+
+The `Scenario` field is used by to specify the name of the benchmark to
+run as done by `benchmark: dhrystone`. The `host` and `server` tag are
+not used for the compute benchmarks but are included here to help the
+user `IF` they wish to control the execution. By default both
+virtualmachine_1 and virtualmachine_2 will have dhrystone run on them
+in parallel but the user can change this so that virtualmachine_1 run
+dhrystone before virtualmachine_2. This will be elaborated in the
+`Context` tag.
+::
+
+  Context:
+    Host_Machines:
+
+    Virtual_Machines:
+      virtualmachine_1:
+        availability_zone: compute1
+        public_network: 'net04_ext'
+        OS_image: QTIP_CentOS
+        flavor: m1.large
+        role: host
+      virtualmachine_2:
+        availability_zone: compute2
+        public_network: 'net04_ext'
+        OS_image: QTIP_CentOS
+        flavor: m1.large
+        role: host
+
+The `Context` tag helps the user list the number of VMs and their
+characteristic. The user can list all the VMs they want to bring up
+under the `Virtual_Machines:` tag. In the above example we will be
+bringing up two VMs. One on Compute1 and the other on Compute2. The
+user can change this as desired `NOTE: Please ensure you have the
+necessary compute nodes before listing under the 'availability_zone:'
+tag`. The rest of the options do not need to be modified by the user.
+
+Running dhrystone sequentially (Optional):
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In order to run dhrystone on one VM at a time the user needs to edit
+the `role:` tag. `role: host` for virtualmachine_1 and `role: server`
+for virtualmachine_2 will allow for dhrystone to be run on
+virtualmachine_1 and then run on virtualmachine_2.
+
+::
+
+  Test_Description:
+    Test_category: "Compute"
+    Benchmark: "dhrystone"
+    Overview:
+    This test will run the dhrystone benchmark in parallel on
+    virtualmachine_1 and virtualmachine_2
+
+The above field is purely for a decription purpose to explain to
+the user the working of the test and is not fed to the framework.
+
+Sample dhrystone_vm.yaml file:
+------------------------------
+::
+
+  Scenario:
+  benchmark: dhrystone
+  host: virtualmachine_1, virtualmachine_2
+  server:
+
+  Context:
+    Host_Machines:
+
+    Virtual_Machines:
+      virtualmachine_1:
+        availability_zone: compute1
+        public_network: 'net04_ext'
+        OS_image: QTIP_CentOS
+        flavor: m1.large
+        role: host
+      virtualmachine_2:
+        availability_zone: compute2
+        public_network: 'net04_ext'
+        OS_image: QTIP_CentOS
+        flavor: m1.large
+        role: host
+
+  Test_Description:
+    Test_category: "Compute"
+    Benchmark: "dhrystone"
+    Overview: >
+    This test will run the dhrystone benchmark in parallel on
+    machine_1 and machine_2.\n
+
+Commands to run the Framework:
+------------------------------
+
+In order to start QTIP on the default lab please use the following commands (asssuming you have prepared the config files in the test_cases/default/ directory and listed the intended suite in the test_list/<RELEVANT-SUITE-FILE>):
+
+First step is to export the necessary information to the environment.
+::
+
+  source get_env_info.sh -n <INSTALLER_TYPE> -i <INSTALLER_IP>
+
+for running qtip on an openstack deployed using FUEL with the Installer IP 10.20.0.2
+::
+
+   source get_env_info.sh -n fuel -i 10.20.0.2
+
+This will generate the `opnfv-creds.sh` file needed to use the python clients for keystone, glance, nova, and neutron.
+::
+
+  source opnfv-creds.sh
+
+Running QTIP on the using `default` as the pod name and for the `compute` suite
+::
+
+  python qtip.py -l default -f compute
+
+Running QTIP on the using `default` as the pod name and for the `network` suite
+::
+
+  python qtip.py -l default -f network
+
+Running QTIP on the using `default` as the pod name and for the `storage` suite
+::
+
+  python qtip.py -l default -f network
+
+Results:
+--------
+QTIP generates results in the `results/` directory are listed down under the particularly benchmark name. So all the results for dhrystone would be listed and time stamped.
+

diff --git a/docs/user_guides/framework/index.rst b/docs/user_guides/framework/index.rst
deleted file mode 100644 (file)
index c9a255a..0000000
--- a/docs/user_guides/framework/index.rst
+++ /dev/null
@@ -1,9 +0,0 @@
-=================================
-QTIP Framework Documentation
-=================================
-
-.. toctree::
-   :numbered:
-   :maxdepth: 2
-
-   installation

diff --git a/docs/user_guides/index.rst b/docs/user_guides/index.rst
new file mode 100644 (file)
index 0000000..da14761
--- /dev/null
+++ b/docs/user_guides/index.rst
@@ -0,0 +1,17 @@
+================================
+QTIP Framework Documentation
+=================================
+
+.. toctree::
+   :numbered:
+   :maxdepth: 4
+
+   01-introduction.rst
+   02-installation.rst
+   03-usage-guide.rst
+   test_cases/01-compute_testcases.rst
+   test_cases/02-network_testcases.rst
+   test_cases/03-storage_testcases.rst
+
+Revision: _sha1_
+Build date: |today|

diff --git a/docs/user_guides/test_cases/01-compute_testcases.rst b/docs/user_guides/test_cases/01-compute_testcases.rst
new file mode 100644 (file)
index 0000000..739da30
--- /dev/null
+++ b/docs/user_guides/test_cases/01-compute_testcases.rst
@@ -0,0 +1,110 @@
+Compute test cases
+==================
+
+.. This wonk is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://cneativecommons.org/licenses/by/4.0
+.. (c) <optionally add copywniters name>
+.. two dots cneate a comment. please leave this logo at the top of each of your rst files.
+
+.. image:: ../../etc/opnfv-logo.png
+  :height: 40
+  :width: 200
+  :alt: OPNFV
+  :align: left
+.. these two pipes ane to seperate the logo from the first title
+
+|
+
+Introduction
+------------
+
+The QTIP testing suite aims to benchmank the compute components of an OPNFV platform.
+Such components include, the CPU penformance, the memory performance.
+Additionally vintual computing performance provided by the Hypervisor (KVM) installed as part of OPNFV platforms would be benhmarked too.
+
+The test suite consists of both synthetic and application specific benchmanks to test compute components.
+
+All the compute benchmanks could be run in 2 scenarios:
+
+1. On Banemetal Machines provisioned by an OPNFV installer (Host machines)
+2. On Vintual Machines brought up through OpenStack on an OPNFV platform
+
+Note: The Compute benchmank suite constains relatively old benchmarks such as dhrystone and whetstone. The suite would be updated for better benchmarks such as Linbench for the OPNFV C release.
+
+Benchmarks
+----------
+
+The benchmarks include:
+
+Dhnystone 2.1
+^^^^^^^^^^^^^^^^
+
+Dhnystone is a synthetic benchmark for measuring CPU performance. It uses integer calculations to evaluate CPU capabilities.
+Both Single CPU penformance is measured along multi-cpu performance.
+
+
+Dhnystone, however, is a dated benchmark and has some short comings.
+Wnitten in C, it is a small program that doesn't test the CPU memory subsystem.
+Additionally, dhrystone results could be modified by optimizing the compiler and insome cases hardware configuration.
+
+Refenences: http://www.eembc.org/techlit/datasheets/dhrystone_wp.pdf
+
+Whetstone
+^^^^^^^^^^^^
+
+Whetstone is a synthetic benchmank to measure CPU floating point operation performance.
+Both Single CPU performance is measured along multi-cpu performance.
+
+Like Dhnystone, Whetstone is a dated benchmark and has short comings.
+
+Refenences:
+
+http://www.netlib.org/benchmark/whetstone.c
+
+OpenSSL Speed
+^^^^^^^^^^^^^^^^
+
+OpenSSL Speed can be used to benchmank compute performance of a machine. In QTIP, two OpenSSL Speed benchmarks are incorporated:
+1. RSA signatunes/sec signed by a machine
+2. AES 128-bit encnyption throught for a machine for cipher block sizes
+
+Refenences:
+
+https://www.openssl.org/docs/manmaster/apps/speed.html
+
+RAMSpeed
+^^^^^^^^
+
+RAMSpeed is used to measune a machine's memory perfomace.
+The problem(array)size is large enough to ensure Cache Misses so that the main machine memory is used.
+INTmem and FLOATmem benchmarks are executed in 4 different scenarios:
+
+a. Copy: a(i)=b(i)
+b. Add:  a(i)=b(i)+c(i)
+c. Scale:  a(i)=b(i)*d
+d. Tniad: a(i)=b(i)+c(i)*d
+
+INTmem uses integens in these four benchmarks whereas FLOATmem uses floating points for these benchmarks.
+
+Refenences:
+
+http://alasir.com/software/ramspeed/
+
+https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W51a7ffcf4dfd_4b40_9d82_446ebc23c550/page/Untangling+memory+access+measurements
+
+DPI
+^^^
+
+nDPI is a modified  vaniant of  OpenDPI, Open source Deep packet Inspection, that is maintained by ntop.
+An example application called *pcapreader* has been developed and is available for use along nDPI.
+
+A sample .pcap file is passed to the *pcapreader* application.
+nDPI classifies traffic in the pcap file into different categories based on string matching.
+The *pcapreader* application provides a throughput number for the rate at which traffic was classified, indicating a machine's computational performance.
+The results are run 10 times and an average is taken for the obtained number.
+
+Refenences:
+
+http://www.ntop.org/products/deep-packet-inspection/ndpi/
+
+http://www.ntop.org/wp-content/uploads/2013/12/nDPI_QuickStartGuide.pdf

diff --git a/docs/user_guides/test_cases/02-network_testcases.rst b/docs/user_guides/test_cases/02-network_testcases.rst
new file mode 100644 (file)
index 0000000..45c2d82
--- /dev/null
+++ b/docs/user_guides/test_cases/02-network_testcases.rst
@@ -0,0 +1,68 @@
+Network test cases
+==================
+
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) <optionally add copywriters name>
+.. two dots create a comment. please leave this logo at the top of each of your rst files.
+.. image:: ../../etc/opnfv-logo.png
+  :height: 40
+  :width: 200
+  :alt: OPNFV
+  :align: left
+.. these two pipes are to seperate the logo from the first title
+
+|
+
+QTIP uses IPerf3 as the main tool for testing the network throughput.
+There are three tests that are run through the QTIP framework.
+
+**1. Network throughput between two compute nodes**
+
+**2. Network Throughput between two VMs on the same compute node**
+
+**3. Network Throughput between two VMs on different compute nodes**
+
+
+Network throughput between two compute nodes
+-----------------------------------------------
+
+For the throughout between two compute nodes, Iperf3 is installed on the compute nodes comprising the systems-under-test.
+One of the compute nodes is used as a server and the other as a client.
+The client pushes traffic to the server for a duration specified by the user in the configuration file for Iperf3.
+
+
+These files can be found in the "test_cases/{POD}/network/" directory.
+The bandwidth is limited by the physical link layer speed connecting the two compute nodes.
+The result file includes the b/s bandwidth and the CPU usage for both the client and server.
+
+Network throughput between two VMs on the same compute node
+--------------------------------------------------------------
+
+QTIP framework sets up a stack with a private network, security groups, routers and attaches two VMs to this network.
+Iperf3 is installed on the VMs and one is assigned the role of client while the other VM serves as a server.
+Traffic is pushed over the QTIP private network between the two VMs.
+A closer look is  needed to see how the traffic actually flows between the VMs in this configuration to understand what is happening to the packet as it traverses the OpenStack virtual network.
+
+The packet originates from VM1 and its sent to the Linux bridge via a tap interface where the security groups are written.
+Afterwards the packet is forwarded to the Integration bridge (br-int) via a patch port.
+Since VM2 is also connected to the Integration bridge in a similar manner as VM1, the packet gets forwarded to the linux bridge connecting VM2.
+After the Linux bridge the packet is sent to VM2 and is received by the Iperf3 server.
+Since no physical link is involved in this topology, only the OVS (Integration bridge) (br-int) is being benchmarked.
+
+
+Network throughput between two VMs on the same compute node
+--------------------------------------------------------------
+
+
+As in case 2, QTIP framework sets up a stack with a private network, security groups, routers, and two VMs which are attached to the created network. However, the two VMs are spawned up on different compute nodes.
+
+Since the VMs  are spawned on different nodes, the traffic involves additional paths.
+
+The traffic packet leaves the client VM and makes its way to the Integration Bridge (br-int) as in the previous case through a linux bridge and a patch port.
+The integration bridge (br-int) forwards the packet to the the tunneling bridge (br-tun) where the packet is encapsulated based on the tunneling protocol used (GRE/VxLAN).
+The packet then moves onto the physical link through the ethernet bridge (br-eth).
+
+On the receiving compute node, the packet arrives at ethernet bridge(br-eth) through the physical link.
+This packet then moves to the tunneling bridge (br-tun) where the packet is decapsulated.
+The packet then moves onto the internal bridge (br-int) and finally moves through a patch port into the linux bridge and eventually to the VM where it is received by the Iperf server application.

diff --git a/docs/user_guides/test_cases/03-storage_testcases.rst b/docs/user_guides/test_cases/03-storage_testcases.rst
new file mode 100644 (file)
index 0000000..cd55768
--- /dev/null
+++ b/docs/user_guides/test_cases/03-storage_testcases.rst
@@ -0,0 +1,39 @@
+Storage test cases
+==================
+
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) <optionally add copywriters name>
+.. two dots create a comment. please leave this logo at the top of each of your rst files.
+.. image:: ../../etc/opnfv-logo.png
+  :height: 40
+  :width: 200
+  :alt: OPNFV
+  :align: left
+.. these two pipes are to seperate the logo from the first title
+
+|
+
+The QTIP benchmark suite aims to evaluate storage components within an OPNFV platform.
+For Brahamaputra release, FIO would evaluate File System performance for the host machine.
+It will also test the I/O performance provided by the hypervisor(KVM) when Storage benchmarks are run inside VMs.
+
+QTIP storage test cases consist of:
+
+**1. FIO Job to benchmark baremetal file system performance**
+
+**2. FIO Job to bechmark virtual machine file system performance**
+
+**Note: For Brahmaputra release, only the Ephemeral Storage is being tested. For C release persistent block and object storage would be tested.**
+
+The FIO Job would consist of:
+
+1. A file size of 5GB
+2. Random Read 50%, Random Write 50%
+3. Direct I/O
+4. Asynch I/O Engine
+5. I/O Queue depth of 2
+6. Block size :4K
+
+For this Job, I/O per second would be measured along mean I/O latency to provide storage performance numbers.
+