Merge "Add overview in developer guide" into stable/danube

Add overview in developer guide

Change-Id: I491f921d87cf0cec3bbef5a78e9f3065b1e012a1
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
(cherry picked from commit cb8ec13d8981cea1dc5556d86ea01b45c29f1046)

Add API docs

  - Add developer guide for API
  - Add user guide for API
  - Add api to user guide doctree

JIRA: QTIP-234

Change-Id: Id388f82c52e31b3c75599ab846e64431be8083b7
Signed-off-by: akhilbatra898 <akhil.batra@research.iiit.ac.in>
(cherry picked from commit 6cd177d400b3462cd6c6b251f9135b45c33dfbb8)

Merge "remove replicated dependencies" into stable/danube

Merge "cite danube.rst in index.rst" into stable/danube

Merge "update user docs" into stable/danube

Merge "Align folder structure `docs/testing/developer` to opnfvdocs template" into stable/danube

Merge "Rename docs/testing/developer/design to docs/testing/developer/devguide" into stable/danube

Merge "Add project introduction" into stable/danube

Add project introduction

Change-Id: I5d0553f0ccec571f9db90b2fd8f25f0941f00b03
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Rename docs/testing/developer/design to docs/testing/developer/devguide

See https://git.opnfv.org/opnfvdocs/tree/docs/testing/testing-dev.rst

Change-Id: Iac14adf4a2876fd74972dcf6279f7022e92cb9b5
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Align folder structure `docs/testing/developer` to opnfvdocs template

Change-Id: I9ce069cf4b5384b5788ca7cfc70e4188519667d5
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
(cherry picked from commit 9639c68d1ed674f74ff1b187346241f5d7c82910)

update user docs

Change-Id: I7a7a1343c457fe1c3063dfeafa079154d495eac0
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 16f2362065636b833c31722c778e485f62acb981)

cite danube.rst in index.rst

Change-Id: I00aa92a22e4e6aabd69463f3dd97d5d4a58f1e3a
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
(cherry picked from commit 7bae1022dcb5fd89016254b70566721e8b59eae3)

remove replicated dependencies

Change-Id: Ie82153d201a06e7008465a5ea116b389dc1311c6
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
(cherry picked from commit 50ab8b586a0564da1e1d029bff65d36464142fce)

Add info about selective running

Change-Id: Ic5f7969c3d3969b7790d17beec10d73991ca8d08
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit ee667225c20ded7d6dbfc38e8631fc96cc0dea2a)

Merge "Update CLI docs." into stable/danube

Merge "Add unit tests for List and get in API." into stable/danube

Merge "add qtip installation in Dockerfile" into stable/danube

Merge "bugfix: regex for dpi" into stable/danube

Update CLI docs.

- Update command to execute runner
- Include reporter

JIRA: QTIP-231

Change-Id: I834ac94821cea7221078fbb6160ef7467b2bd4cf
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>

add qtip installation in Dockerfile

Docker build successfully on local. And QTIP cli works well.

Change-Id: I0916c73e67808ae170ff7b290b8e03b605be8e69
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 8c325e64ada6fb2cb4377e592fa204170c7d8513)

bugfix: regex for dpi

see: https://build.opnfv.org/ci/view/qtip/job/
qtip-os-nosdn-kvm-ha-zte-pod3-daily-danube/3/console

Previous regex cann't match with log like:
"nDPI throughput:       943.34 K pps / 8.86 Gb/sec".
For pps, there are two type of unit: K and M.
Same with bps.

Change-Id: I247c8b6049cc8b264fe894538cd4f89ba544c0b8
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 6ee214ad9487dfaa121c72d8487464f63992c5ff)

Integrate cli, runner and reporter.

- Execute runner via a shell command
- Change format of qtip result directory to qtip-timestamp
- Add path option in reporter to match with runner

JIRA: QTIP-229

Change-Id: I7d8562fd7100b1f40cdc8d53b0daa6a06a55b495
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit c2bb13c460566a18e61a3c840bf12f7f717940c2)

Merge "Cleanup DEVELOP.md" into stable/danube

Add unit tests for List and get in API.

 - refactor controllers
 - remove abspath and other irrelvant data in response
 - move fixtures
 - refactor decorators

JIRA: QTIP-226

Change-Id: I5fac5b1bc998da198098992e7ddb47ba49685f31
Signed-off-by: akhilbatra898 <akhil.batra@research.iiit.ac.in>
(cherry picked from commit bef693f40ad87170b7233b9fef62f2fd8abfc8d8)

Add missing license headers

Change-Id: Idd76277c84a6afc9a545d6e2bba39aae6e7f5d71
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
(cherry picked from commit 29df737b48ccf31f555d0e4c08a65e14ecb66331)

Cleanup DEVELOP.md

- remove unecessary content
- move CLI develop guide to rst document

Change-Id: I20b6f458aa13f1feb1764e6c4d096ec8a9dddf49
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

correct the path of .ansible.cfg

In qtip container, /home/opnfv/ is HOME path. So the customized
.ansible.cfg should be put under this directory.

Change-Id: Ie0cddab9dac72fc45c14d7d3652b6863c5748260
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 8ecff917366390b8a8ddf268aea98c6f82a7376c)

Merge "provide get_one_exceptions to unify get_xx process" into stable/danube

Merge "Remove installation and scenario guide" into stable/danube

Merge "Create archtecture design specification" into stable/danube

Merge "White space control in report." into stable/danube

Merge "correct a mistake for os.getenv()" into stable/danube

White space control in report.

- Flaw: System Info and metric results at same line
  Packets per Second: 1.45System Information:

Change-Id: I22790e4b2c65fd2ef555e1c20e602a51ea97eb3b
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit 06638bb68811dda1a3599c7ef0176c5a58d056c5)

correct a mistake for os.getenv()

os.getenv returns a string not a boolean.
Although you set environment variable 'CI_DEBUG=false',
the cleanup will not be executed.

Change-Id: I077b602069f026a4b9ef6019869ac6c29c401c21
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit b07a423d0fb1b6bfbdb851195097d6fda38bc20e)

provide get_one_exceptions to unify get_xx process

add refactor get_metric()

Change-Id: I972c77e63a654eeb286c573d75efe842b2887c1d
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
(cherry picked from commit 9f87878becc36b19110e151e802180aa5f7fb305)

Refactor reporter module

- Create templates for report format specific to each metric.
- Outputs the details of the specified metric across all the nodes
  in cluster.
- Timeline to be covered in a separate patch

JIRA: QTIP-199

Change-Id: Ic83749725b0c9cc5bd9a7f24f21b2cd113abe0e1
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit 04e50e57819bdf40a2fd7abdd3fc7be798771e0b)

Remove installation and scenario guide

Not applicable for non-installer projects

Change-Id: Id83898cd0da14b6a7339e6c2cd9fd5fb607bf443
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

mistaken delete usefuel public key

Previously, cleanup_creds.sh will delete public keys which of
comment info is "root@$hostname". It could delete useful keys
by accident. In this patch, cleanup_creds.sh will only delete
the key which matches the key's content.

JIRA: QTIP-228

Change-Id: I1095de656570b816b7e170688acd01c27b98e74c
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit eaa225cf0c9b8b7bc63b35acdeef75dbacb8b905)

Create archtecture design specification

JIRA: QTIP-148
Change-Id: I1d5d83e0bd59b1f504bbf3ad3bd4a5cc9a6ff022
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Create release note draft

release content to be retrieved from JIRA

Change-Id: Id15cb437019ac162b6737e9d9c49d79194c3f28d
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Merge "Create outline for documents to be released in Danube" into stable/danube

Create outline for documents to be released in Danube

- deprecated legacy document
- created content placeholders

Change-Id: I12de6467d1fed83a9670750e23ec070901deee63
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Implement 'show' command.

- Render the description via templates

JIRA: QTIP-205

Change-Id: I10523f85f80350e901a4a701bb65ca4833f8ff7c
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit bb5af4b9be1325b61c7f80e71c7d50892ae22956)

Add aarch64 patch

This commit adds a patch that can be applied to the Dockerfile to adapt
it for aarch64. When applied from releng, a new Dockerfile.aarch64 is
created and an aarch64 qtip image is created.

The aarch64 qtip image will be build with the releng fix
https://gerrit.opnfv.org/gerrit/30181

JIRA: ARMBAND-219

Change-Id: I551ae7fa1d7fc23c6e5f6381d605b0b22dbec695
Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
(cherry picked from commit 2935006a650f92255c8046d236e691360148d9ff)

Merge "optimize console info" into stable/danube

Merge "add the cleanup step in runner.py" into stable/danube

Merge "Miss \ in Dockerfile" into stable/danube

optimize console info

- use QtipLogger instead of print
- delete useless print

Change-Id: I1e08382a5d78ce53cf455496363bce3762b81dd2
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 3391a3c6ab4f6b33e2f6d74e858a7bad573695b1)

add the cleanup step in runner.py

Add the cleanup step in runner.py

Change-Id: I0016986485eab5d7ab45d4a7a393bea5fb6f96b5
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 57ec2e7a9757cbb1da8a1f73f0c55e8a5cb67bec)

Merge "Implement the workflow of compute qpi" into stable/danube

Merge "parse benchmark result from logfile" into stable/danube

Miss \ in Dockerfile

https://build.opnfv.org/ci/view/qtip/job/qtip-docker-build-push-master/250/console

Change-Id: If6d76c6a27cc61a1b3d468751932e21d32dff277
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit b14071b6bfb19920028dd45a8c78d57af872922c)

Merge "Remove legacy code from stable branch" into stable/danube

Merge "env cleanup" into stable/danube

Implement the workflow of compute qpi

Local test is ok.
The result will be written to report.json.

usage: runner.py [-h] -d DEST -b BENCHMARK

optional arguments:
  -d DEST, --dest DEST  the destination where results will be stored.
  -b BENCHMARK, --benchmark BENCHMARK
                        the benchmark you want to execute.

Change-Id: Ic3a70c65a5aa045bf9df34ce4d14957a7a1b3dcf
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 478cd02a9219f7c8b49d8529d1f809a04399ad0d)

parse benchmark result from logfile

- Put all the regex rules in regex.yaml.
- According to benchmark name, we can find related regexes.

Change-Id: Ic15bd1c77b525be3751011fa94d582da077b0345
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit dc92e5ae19dd026bf6a14c1e0d2b9c50497845d5)

Some modifications for ansible playbook

- typo in ramspeed/run.yaml
- fix result logfile name for dhrystone and whetstone

Change-Id: Ic8456174a14d04c2207990eccb16601762c5a880
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 764c2d3916f1390e2ee2ca3b3efbcc33050fd525)

Remove legacy code from stable branch

Change-Id: I283f84dfcb3cd029739caaefb33ed5431114fae9
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Merge "Implement 'list' command." into stable/danube

env cleanup

If IF_DEBUG is True, there is not cleanup step.
If IF_DEBUG is False, host file and keypair will be
deleted on local  and public key will be removed from
remote nodes in the end of test.

Change-Id: I4a88acc2d428e41e4abaedacb011e27468b2cb57
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit d37dad746e504e833abcf2ae058f2b81e2a9d365)

Implement Retrieval of plans, QPIs, and metrics

- All specs can be listed
- All specs can be individually retrieved
- API can be installed alongside qtip
- API can be hosted using command `qtip-api`

JIRA: QTIP-221
JIRA: QTIP-222

Change-Id: I1e80e6609cae1252cb2fcdc38c71b4bf6d02e8c9
Signed-off-by: akhilbatra898 <akhil.batra@research.iiit.ac.in>
(cherry picked from commit 15fceada86ad0d5dabe26365451aae9c95b36be1)

Implement 'list' command.

- Loads Qtip components from the default path.

JIRA: QTIP-205

Change-Id: Id87993d65c5cd6a23c199cf1049d75a0971b6354
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit 0bd051505c2851d61356e7c448d7802913ab785f)

Merge "fill benchmarks"

Merge "bugfix benchmarks cannot be found"

fill benchmarks

Change-Id: I7ad5f504058b20db72c0878cf9618e6bff186b33
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>

bugfix benchmarks cannot be found

OSError: [Errno 2] No such file or directory:
'/Users/fengxiaowei/opnfv/qtip/.venv/lib/python2.7/
site-packages/qtip/cli/../../tests/data/benchmarks/plan'

Change-Id: I510a2beb215b3e5c93cfa5870fd892b0c7720f01
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>

Merge "Add missing softwares"

Merge "modify logfile directory structure"

Add missing softwares

Install iputils-ping for ping and rsync for synchronizing
files and directories.

Change-Id: I93ba44bcb04948bbe846fb179dc673aa86a6a92a
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Merge "Add basic swagger specifications and corresponding controllers"

Merge "Adding Labels"

modify logfile directory structure

After one qtip test execution, the logfiles will be stored like this:

`-- 2017-03-09-17-40
    `-- node-41
        |-- dhrystone
        |   |-- inxi.log
        |   |-- node-41.zte.com.cn-2017-03-09-01
        |   |-- node-41.zte.com.cn-2017-03-09-01.html
        |   |-- node-41.zte.com.cn-2017-03-09-01.log
        |   `-- top.log
        |-- dpi
        |   |-- dpi_dump.txt
        |   |-- inxi.log
        |   `-- top.log
        |-- ramspeed
        |   |-- Floatmem
        |   |-- Intmem
        |   |-- inxi.log
        |   `-- top.log
        |-- ssl
        |   |-- AES-128-CBC_dump
        |   |-- RSA_dump
        |   |-- inxi.log
        |   `-- top.log
        `-- whetstone
            |-- inxi.log
            |-- node-41.zte.com.cn-2017-03-09-01
            |-- node-41.zte.com.cn-2017-03-09-01.html
            |-- node-41.zte.com.cn-2017-03-09-01.log
            `-- top.log

Change-Id: I5889a0e5dbe99f28212d6e0cdf69b9c03d89a9fe
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Add basic swagger specifications and corresponding controllers

- Update specifications as per requirements
- map specifications with controllers
- make api installable

JIRA: QTIP-220

Change-Id: Id149fdcf68e869e31a00cf16d7e725e368d2b25f
Signed-off-by: akhilbatra898 <akhil.batra@research.iiit.ac.in>

Merge "Implement ansible driver"

Merge "Playbooks for unixbench"

regexes for dpi/ramspeed/ssl

- Add dpi/ramspeed/ssl logfiles
- update regexes for dpi/ramspeed/ssl

result sample:
dpi: {
    "bps": "4.57",
    "pps": "1.55"
}
ramspeed:{
    "float_add": "9967.13",
    "float_copy": "7908.64",
    "float_scale": "7870.16",
    "float_triad": "10073.43",
    "float_average": "8954.84",
    "integer_add": "11649.55",
    "integer_copy": "11562.63",
    "integer_scale": "11563.77",
    "integer_triad": "11671.22",
    "integer_average": "11611.79"
}
ssl:{
    "aes_128_cbc_1024_bytes": "584568.83k",
    "aes_128_cbc_16_bytes": "533103.05k",
    "aes_128_cbc_256_bytes": "580021.25k",
    "aes_128_cbc_64_bytes": "570042.22k",
    "aes_128_cbc_8192_bytes": "599470.83k"
    "rsa_sign_1024": "3.9",
    "rsa_sign_2048": "1.4",
    "rsa_sign_4096": "0.8",
    "rsa_sign_512": "8.4",
    "rsa_verify_1024": "88397.9",
    "rsa_verify_2048": "26951.3",
    "rsa_verify_4096": "7633.7",
    "rsa_verify_512": "222903.5"
}

Change-Id: I6db22a699111dd55d9dc48c6e8c2cf148b345563
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Merge "playbook for ramspeed"

Merge "Refactor openssl playbook"

Merge "Remove hardcode from some playbooks"

Merge "Refactor dpi playbook"

Implement ansible driver

- According to inputs parameters, setup test environment. If failed
during setupping, qtip will exited. If successful, execute metric
tests.
- Parameters(optional):
* keypair: the keypair to login/execute commands to the remote
  hosts. They can be automatically generated.
* hostfile: a inventory file. If not give, it can be
  automatically generated via installer.
* args: the parameters passed to playbook
- Use ansible python api to trigger ansible-playbook.

Unit test will be in a new patch.

Change-Id: I7470d348308f7cb6cb669bcc49063cf0f4da2111
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Refactor openssl playbook

Local test is ok.

Change-Id: I2800e364320697d436ed7b9202cc1baa3a893503
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Playbooks for unixbench

Actually, both dhrystone and whether use unixbench tool
to get metric. Their playbook are too smiliar.
The playbooks under /unixbench can be used to measure
both of them.

Change-Id: I26e31c2b4ac4de35ff8b7ec6dea26b676d5078de
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Adding Labels

Labels serve as anchors/ refs when referenced from other documents
This patch adds labels to docs/*/*/index.rst

Change-Id: Id23477c82231a1d02463802d60ee9d88a7fabc41
Signed-off-by: ShubhamRathi <shubhamiiitbackup@gmail.com>

playbook for ramspeed

These playbooks are successfully executed on Local.

Change-Id: Ie22ecb9b66364e6edf6ae0f59d0524603c4d46a9
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Refactor dpi playbook

Local test is ok.

Change-Id: I22a333eae940433a1ca228e6ee7f570b8d0c0fc9
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Remove hardcode from some playbooks

Remove hardcode ($HOME/qtip_result) from these three
playbooks.

Change-Id: Iefcf0d4c6caf6be6b6888ff6abe4c36621666129
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Merge "rework env.py"

Merge "Add `subrepo` basic usage in developer guide."

Merge "Qtip command completion."

Merge "Implement debug option"

Merge "Implement reporter module."

Merge "Create draft of report template in console"

Add `subrepo` basic usage in developer guide.

Change-Id: I4e066d91c9d0775ba698584f561cce997d60ad6b
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Merge "Switch License tool to develop branch"

rework env.py

Create a new class AnsibleEnvSetup for setuping
test environment for ansible driver.

Change-Id: I5d2385b33e5666acb7bbac1a6d960a805d846a32
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Merge "bugfix on bash scripts"

Merge "Refactor dhrystone playbook"