Merge "Update CLI docs." into stable/danube

Merge "Add unit tests for List and get in API." into stable/danube

Merge "add qtip installation in Dockerfile" into stable/danube

Merge "bugfix: regex for dpi" into stable/danube

Update CLI docs.

- Update command to execute runner
- Include reporter

JIRA: QTIP-231

Change-Id: I834ac94821cea7221078fbb6160ef7467b2bd4cf
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>

add qtip installation in Dockerfile

Docker build successfully on local. And QTIP cli works well.

Change-Id: I0916c73e67808ae170ff7b290b8e03b605be8e69
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 8c325e64ada6fb2cb4377e592fa204170c7d8513)

bugfix: regex for dpi

see: https://build.opnfv.org/ci/view/qtip/job/
qtip-os-nosdn-kvm-ha-zte-pod3-daily-danube/3/console

Previous regex cann't match with log like:
"nDPI throughput:       943.34 K pps / 8.86 Gb/sec".
For pps, there are two type of unit: K and M.
Same with bps.

Change-Id: I247c8b6049cc8b264fe894538cd4f89ba544c0b8
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 6ee214ad9487dfaa121c72d8487464f63992c5ff)

Integrate cli, runner and reporter.

- Execute runner via a shell command
- Change format of qtip result directory to qtip-timestamp
- Add path option in reporter to match with runner

JIRA: QTIP-229

Change-Id: I7d8562fd7100b1f40cdc8d53b0daa6a06a55b495
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit c2bb13c460566a18e61a3c840bf12f7f717940c2)

Merge "Cleanup DEVELOP.md" into stable/danube

Add unit tests for List and get in API.

 - refactor controllers
 - remove abspath and other irrelvant data in response
 - move fixtures
 - refactor decorators

JIRA: QTIP-226

Change-Id: I5fac5b1bc998da198098992e7ddb47ba49685f31
Signed-off-by: akhilbatra898 <akhil.batra@research.iiit.ac.in>
(cherry picked from commit bef693f40ad87170b7233b9fef62f2fd8abfc8d8)

Add missing license headers

Change-Id: Idd76277c84a6afc9a545d6e2bba39aae6e7f5d71
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
(cherry picked from commit 29df737b48ccf31f555d0e4c08a65e14ecb66331)

Cleanup DEVELOP.md

- remove unecessary content
- move CLI develop guide to rst document

Change-Id: I20b6f458aa13f1feb1764e6c4d096ec8a9dddf49
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

correct the path of .ansible.cfg

In qtip container, /home/opnfv/ is HOME path. So the customized
.ansible.cfg should be put under this directory.

Change-Id: Ie0cddab9dac72fc45c14d7d3652b6863c5748260
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 8ecff917366390b8a8ddf268aea98c6f82a7376c)

Merge "provide get_one_exceptions to unify get_xx process" into stable/danube

Merge "Remove installation and scenario guide" into stable/danube

Merge "Create archtecture design specification" into stable/danube

Merge "White space control in report." into stable/danube

Merge "correct a mistake for os.getenv()" into stable/danube

White space control in report.

- Flaw: System Info and metric results at same line
  Packets per Second: 1.45System Information:

Change-Id: I22790e4b2c65fd2ef555e1c20e602a51ea97eb3b
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit 06638bb68811dda1a3599c7ef0176c5a58d056c5)

correct a mistake for os.getenv()

os.getenv returns a string not a boolean.
Although you set environment variable 'CI_DEBUG=false',
the cleanup will not be executed.

Change-Id: I077b602069f026a4b9ef6019869ac6c29c401c21
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit b07a423d0fb1b6bfbdb851195097d6fda38bc20e)

provide get_one_exceptions to unify get_xx process

add refactor get_metric()

Change-Id: I972c77e63a654eeb286c573d75efe842b2887c1d
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
(cherry picked from commit 9f87878becc36b19110e151e802180aa5f7fb305)

Refactor reporter module

- Create templates for report format specific to each metric.
- Outputs the details of the specified metric across all the nodes
  in cluster.
- Timeline to be covered in a separate patch

JIRA: QTIP-199

Change-Id: Ic83749725b0c9cc5bd9a7f24f21b2cd113abe0e1
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit 04e50e57819bdf40a2fd7abdd3fc7be798771e0b)

Remove installation and scenario guide

Not applicable for non-installer projects

Change-Id: Id83898cd0da14b6a7339e6c2cd9fd5fb607bf443
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

mistaken delete usefuel public key

Previously, cleanup_creds.sh will delete public keys which of
comment info is "root@$hostname". It could delete useful keys
by accident. In this patch, cleanup_creds.sh will only delete
the key which matches the key's content.

JIRA: QTIP-228

Change-Id: I1095de656570b816b7e170688acd01c27b98e74c
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit eaa225cf0c9b8b7bc63b35acdeef75dbacb8b905)

Create archtecture design specification

JIRA: QTIP-148
Change-Id: I1d5d83e0bd59b1f504bbf3ad3bd4a5cc9a6ff022
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Create release note draft

release content to be retrieved from JIRA

Change-Id: Id15cb437019ac162b6737e9d9c49d79194c3f28d
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Merge "Create outline for documents to be released in Danube" into stable/danube

Create outline for documents to be released in Danube

- deprecated legacy document
- created content placeholders

Change-Id: I12de6467d1fed83a9670750e23ec070901deee63
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Implement 'show' command.

- Render the description via templates

JIRA: QTIP-205

Change-Id: I10523f85f80350e901a4a701bb65ca4833f8ff7c
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit bb5af4b9be1325b61c7f80e71c7d50892ae22956)

Add aarch64 patch

This commit adds a patch that can be applied to the Dockerfile to adapt
it for aarch64. When applied from releng, a new Dockerfile.aarch64 is
created and an aarch64 qtip image is created.

The aarch64 qtip image will be build with the releng fix
https://gerrit.opnfv.org/gerrit/30181

JIRA: ARMBAND-219

Change-Id: I551ae7fa1d7fc23c6e5f6381d605b0b22dbec695
Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
(cherry picked from commit 2935006a650f92255c8046d236e691360148d9ff)

Merge "optimize console info" into stable/danube

Merge "add the cleanup step in runner.py" into stable/danube

Merge "Miss \ in Dockerfile" into stable/danube

optimize console info

- use QtipLogger instead of print
- delete useless print

Change-Id: I1e08382a5d78ce53cf455496363bce3762b81dd2
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 3391a3c6ab4f6b33e2f6d74e858a7bad573695b1)

add the cleanup step in runner.py

Add the cleanup step in runner.py

Change-Id: I0016986485eab5d7ab45d4a7a393bea5fb6f96b5
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 57ec2e7a9757cbb1da8a1f73f0c55e8a5cb67bec)

Merge "Implement the workflow of compute qpi" into stable/danube

Merge "parse benchmark result from logfile" into stable/danube

Miss \ in Dockerfile

https://build.opnfv.org/ci/view/qtip/job/qtip-docker-build-push-master/250/console

Change-Id: If6d76c6a27cc61a1b3d468751932e21d32dff277
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit b14071b6bfb19920028dd45a8c78d57af872922c)

Merge "Remove legacy code from stable branch" into stable/danube

Merge "env cleanup" into stable/danube

Implement the workflow of compute qpi

Local test is ok.
The result will be written to report.json.

usage: runner.py [-h] -d DEST -b BENCHMARK

optional arguments:
  -d DEST, --dest DEST  the destination where results will be stored.
  -b BENCHMARK, --benchmark BENCHMARK
                        the benchmark you want to execute.

Change-Id: Ic3a70c65a5aa045bf9df34ce4d14957a7a1b3dcf
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 478cd02a9219f7c8b49d8529d1f809a04399ad0d)

parse benchmark result from logfile

- Put all the regex rules in regex.yaml.
- According to benchmark name, we can find related regexes.

Change-Id: Ic15bd1c77b525be3751011fa94d582da077b0345
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit dc92e5ae19dd026bf6a14c1e0d2b9c50497845d5)

Some modifications for ansible playbook

- typo in ramspeed/run.yaml
- fix result logfile name for dhrystone and whetstone

Change-Id: Ic8456174a14d04c2207990eccb16601762c5a880
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit 764c2d3916f1390e2ee2ca3b3efbcc33050fd525)

Remove legacy code from stable branch

Change-Id: I283f84dfcb3cd029739caaefb33ed5431114fae9
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Merge "Implement 'list' command." into stable/danube

env cleanup

If IF_DEBUG is True, there is not cleanup step.
If IF_DEBUG is False, host file and keypair will be
deleted on local  and public key will be removed from
remote nodes in the end of test.

Change-Id: I4a88acc2d428e41e4abaedacb011e27468b2cb57
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
(cherry picked from commit d37dad746e504e833abcf2ae058f2b81e2a9d365)

Implement Retrieval of plans, QPIs, and metrics

- All specs can be listed
- All specs can be individually retrieved
- API can be installed alongside qtip
- API can be hosted using command `qtip-api`

JIRA: QTIP-221
JIRA: QTIP-222

Change-Id: I1e80e6609cae1252cb2fcdc38c71b4bf6d02e8c9
Signed-off-by: akhilbatra898 <akhil.batra@research.iiit.ac.in>
(cherry picked from commit 15fceada86ad0d5dabe26365451aae9c95b36be1)

Implement 'list' command.

- Loads Qtip components from the default path.

JIRA: QTIP-205

Change-Id: Id87993d65c5cd6a23c199cf1049d75a0971b6354
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
(cherry picked from commit 0bd051505c2851d61356e7c448d7802913ab785f)

Merge "fill benchmarks"

Merge "bugfix benchmarks cannot be found"

fill benchmarks

Change-Id: I7ad5f504058b20db72c0878cf9618e6bff186b33
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>

bugfix benchmarks cannot be found

OSError: [Errno 2] No such file or directory:
'/Users/fengxiaowei/opnfv/qtip/.venv/lib/python2.7/
site-packages/qtip/cli/../../tests/data/benchmarks/plan'

Change-Id: I510a2beb215b3e5c93cfa5870fd892b0c7720f01
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>

Merge "Add missing softwares"

Merge "modify logfile directory structure"

Add missing softwares

Install iputils-ping for ping and rsync for synchronizing
files and directories.

Change-Id: I93ba44bcb04948bbe846fb179dc673aa86a6a92a
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Merge "Add basic swagger specifications and corresponding controllers"

Merge "Adding Labels"

modify logfile directory structure

After one qtip test execution, the logfiles will be stored like this:

`-- 2017-03-09-17-40
    `-- node-41
        |-- dhrystone
        |   |-- inxi.log
        |   |-- node-41.zte.com.cn-2017-03-09-01
        |   |-- node-41.zte.com.cn-2017-03-09-01.html
        |   |-- node-41.zte.com.cn-2017-03-09-01.log
        |   `-- top.log
        |-- dpi
        |   |-- dpi_dump.txt
        |   |-- inxi.log
        |   `-- top.log
        |-- ramspeed
        |   |-- Floatmem
        |   |-- Intmem
        |   |-- inxi.log
        |   `-- top.log
        |-- ssl
        |   |-- AES-128-CBC_dump
        |   |-- RSA_dump
        |   |-- inxi.log
        |   `-- top.log
        `-- whetstone
            |-- inxi.log
            |-- node-41.zte.com.cn-2017-03-09-01
            |-- node-41.zte.com.cn-2017-03-09-01.html
            |-- node-41.zte.com.cn-2017-03-09-01.log
            `-- top.log

Change-Id: I5889a0e5dbe99f28212d6e0cdf69b9c03d89a9fe
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Add basic swagger specifications and corresponding controllers

- Update specifications as per requirements
- map specifications with controllers
- make api installable

JIRA: QTIP-220

Change-Id: Id149fdcf68e869e31a00cf16d7e725e368d2b25f
Signed-off-by: akhilbatra898 <akhil.batra@research.iiit.ac.in>

Merge "Implement ansible driver"

Merge "Playbooks for unixbench"

regexes for dpi/ramspeed/ssl

- Add dpi/ramspeed/ssl logfiles
- update regexes for dpi/ramspeed/ssl

result sample:
dpi: {
    "bps": "4.57",
    "pps": "1.55"
}
ramspeed:{
    "float_add": "9967.13",
    "float_copy": "7908.64",
    "float_scale": "7870.16",
    "float_triad": "10073.43",
    "float_average": "8954.84",
    "integer_add": "11649.55",
    "integer_copy": "11562.63",
    "integer_scale": "11563.77",
    "integer_triad": "11671.22",
    "integer_average": "11611.79"
}
ssl:{
    "aes_128_cbc_1024_bytes": "584568.83k",
    "aes_128_cbc_16_bytes": "533103.05k",
    "aes_128_cbc_256_bytes": "580021.25k",
    "aes_128_cbc_64_bytes": "570042.22k",
    "aes_128_cbc_8192_bytes": "599470.83k"
    "rsa_sign_1024": "3.9",
    "rsa_sign_2048": "1.4",
    "rsa_sign_4096": "0.8",
    "rsa_sign_512": "8.4",
    "rsa_verify_1024": "88397.9",
    "rsa_verify_2048": "26951.3",
    "rsa_verify_4096": "7633.7",
    "rsa_verify_512": "222903.5"
}

Change-Id: I6db22a699111dd55d9dc48c6e8c2cf148b345563
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Merge "playbook for ramspeed"

Merge "Refactor openssl playbook"

Merge "Remove hardcode from some playbooks"

Merge "Refactor dpi playbook"

Implement ansible driver

- According to inputs parameters, setup test environment. If failed
during setupping, qtip will exited. If successful, execute metric
tests.
- Parameters(optional):
* keypair: the keypair to login/execute commands to the remote
  hosts. They can be automatically generated.
* hostfile: a inventory file. If not give, it can be
  automatically generated via installer.
* args: the parameters passed to playbook
- Use ansible python api to trigger ansible-playbook.

Unit test will be in a new patch.

Change-Id: I7470d348308f7cb6cb669bcc49063cf0f4da2111
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Refactor openssl playbook

Local test is ok.

Change-Id: I2800e364320697d436ed7b9202cc1baa3a893503
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Playbooks for unixbench

Actually, both dhrystone and whether use unixbench tool
to get metric. Their playbook are too smiliar.
The playbooks under /unixbench can be used to measure
both of them.

Change-Id: I26e31c2b4ac4de35ff8b7ec6dea26b676d5078de
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Adding Labels

Labels serve as anchors/ refs when referenced from other documents
This patch adds labels to docs/*/*/index.rst

Change-Id: Id23477c82231a1d02463802d60ee9d88a7fabc41
Signed-off-by: ShubhamRathi <shubhamiiitbackup@gmail.com>

playbook for ramspeed

These playbooks are successfully executed on Local.

Change-Id: Ie22ecb9b66364e6edf6ae0f59d0524603c4d46a9
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Refactor dpi playbook

Local test is ok.

Change-Id: I22a333eae940433a1ca228e6ee7f570b8d0c0fc9
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Remove hardcode from some playbooks

Remove hardcode ($HOME/qtip_result) from these three
playbooks.

Change-Id: Iefcf0d4c6caf6be6b6888ff6abe4c36621666129
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Merge "rework env.py"

Merge "Add `subrepo` basic usage in developer guide."

Merge "Qtip command completion."

Merge "Implement debug option"

Merge "Implement reporter module."

Merge "Create draft of report template in console"

Add `subrepo` basic usage in developer guide.

Change-Id: I4e066d91c9d0775ba698584f561cce997d60ad6b
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Merge "Switch License tool to develop branch"

rework env.py

Create a new class AnsibleEnvSetup for setuping
test environment for ansible driver.

Change-Id: I5d2385b33e5666acb7bbac1a6d960a805d846a32
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Merge "bugfix on bash scripts"

Merge "Refactor dhrystone playbook"

Implement debug option

Show traceback only when debug flag specified.

JIRA: QTIP-201

Change-Id: I9cc4b4ed5a2cb2d2efabd9c0eb3aac216321ebac
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>

Merge "Add missing license headers"

Implement reporter module.

- Add cli module
- Implement timeline outline

JIRA: QTIP-199

Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
Change-Id: I487b76123a785731c356143d1ae7f3123c11f398

Add missing license headers

Change-Id: I33cdd3b1304c695b81d2e23434a70acfcc9a21f8
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Switch License tool to develop branch

The default branch in the remote repository has been changed to develop.

-----------auto-generated by git-subrepo-------------
git subrepo pull --branch=develop third-party/License

subrepo:
  subdir:   "third-party/License"
  merged:   "88b1440"
upstream:
  origin:   "git@github.com:openzero-zte/License.git"
  branch:   "develop"
  commit:   "88b1440"
git-subrepo:
  version:  "0.3.0"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "988f8c8"
-------------------------------------------------------

Change-Id: I6efbee7217e1f04febeaf5b5079ee3e7eda797cd
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

bugfix on bash scripts

When debugging on real environment, I found these two bugs.
Local test is ok.

Change-Id: I799afe6a4c24fcdf370d761c83376ca3f2ce43d2
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Restructure docs

- Following the changes as in functest[1] since the guide[2] in wiki is a bit
  out of date
- Some folders are empty for now as a placeholder

Now:
 - release
    - release notes

 - testing
    - developer
       - design
       - overview
       - requirement

    - user
       - config guide
       - user guide
       - installation
       - scenarios

[1]: https://gerrit.opnfv.org/gerrit/#/c/29151/
[2]: https://wiki.opnfv.org/display/DOC/Documentation+Guide

Change-Id: I81b4597536c8bf02a925641d71d0969f6f5537f4
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Create draft of report template in console

Change-Id: Ie6e626753c6562c361542c45e8504d177d624d2a
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Refactor dhrystone playbook

- Break previous one playbook into three playbooks:
    setup.yaml: setup test environment.
    run.yaml: execute test steps
    clean.yaml: cleanup test environment to restore
    environment
  The advantage is cleaning part will never be blocked by another.
- Just do testing on remote hosts, parse logfile on localhost.
- Use ansible module instead of shell command.
- Move up all files under /ansible and remove ansible directory.
  Under /ansible, ansible package cann't be imported.

Change-Id: I43c65be23869b7d42ca769994d4decb9a012d196
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Install latest requirment packages

When I debug code in qtip container, I found
the version of paramiko(1.16.0) is outdated.
There is a bug based on this version.[1]

Solution:
1. Pip install qtip requirment packages with -U.
2. Update dependent packages.[2][3]

And docker image is built successfully on local.

[1] http://stackoverflow.com/questions/42029415/
paramiko-futurewarning-ctr-mode-needs-counter-parameter
[2] http://www.paramiko.org/installing.html
[3] https://cryptography.io/en/latest/installation/

Change-Id: I9d90c49f7cf3048f4717a76ffcb19dab68a687d4
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Qtip command completion.

Change-Id: I164150acd05eacf5ccdb182280b747495d34347d
Signed-off-by: Taseer Ahmed <taseer94@gmail.com>

Merge "refactor code related with setup env"

refactor code related with setup env

- modify env.py and scripts
- add unit test

Change-Id: I416cd517bdddfeb907675705009d83ecada7f87c
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>

Rename test case to avoid possible name conflict

pytest does not allow two test file have same name despite they are in
different folder.

Change-Id: I568fe042f6dbe99df336d0ce4d6ed9c78305fe74
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Deprecate unused drivers sample and yardstick

- yardstick driver will not be implemented in Danube release
- sample driver is no longer continued

Change-Id: I22a86fe22b08c85b2a18ef70443d725bc4a25246
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>

Merge "Fix bug when no paths is given for Plan constructor"