Revert ssh_timeout to 196 in Functest

The race condition has been fixed [1] and we can try the timeout used
in OpenStack gates.

[1] https://gerrit.opnfv.org/gerrit/c/functest/+/70606

Change-Id: Iaf8b260c743c1fd0088d31060a43b1788545743c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Increase check_console_loop to 12

It failed once when deeply testing in parallel [1].

https://build.opnfv.org/ci/job/functest-opnfv-functest-vnf-jerma-vyos_vrouter-run/80/console

Change-Id: Ia70c43d603302d1bed46d984541b0e71461be4c4
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update tempest_cinder_cntt test_counts to 11

http://artifacts.opnfv.org/functest/81YZ7LMI0PUW/functest-opnfv-functest-smoke-cntt-latest-tempest_cinder_cntt-run-149/tempest_cinder_cntt/tempest-report.html

Change-Id: Ibe7c5ce70262c6fb604bab05038fbd6c408d23f5
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Apply new Create-new-server-test_reboot_server_hard.patch

test_reboot_server_hard sometimes fail in all gates [1].
This hack could highlight if they are side effects between
test_create_backup and test_reboot_server_hard.

[1] http://artifacts.opnfv.org/functest/E5AZMH89OOK6/functest-opnfv-functest-smoke-cntt-hunter-tempest_full_cntt-run-142/tempest_full_cntt/tempest-report.html

Change-Id: I230a6faa4b619e71646a312500ee38a3a8b1093c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Skip ImportImagesTest.test_image_import in CNTT

ImportImagesTest skipped as image import is not available [1]

[1] http://artifacts.opnfv.org/functest/S9ABQ628T5IG/functest-opnfv-functest-smoke-latest-tempest_full-run-850/tempest_full/tempest-report.html

Change-Id: I1999ed96f546ac7bf2115d205382b3b7f4dfe9af
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Add nics net-id when booting dd load test

http://artifacts.opnfv.org/functest/EP1852KPIZEE/functest-opnfv-functest-smoke-latest-rally_sanity-run-920/rally_sanity/rally_sanity.html#/VMTasks.dd_load_test/failures

Change-Id: I959d414692dd52e76175a7341f7d16ef28ab8472
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Print data to debug ServerActionsTestJSON.test_reboot_server_hard

Change-Id: Ib3142be2d6f04315c51831d06e35cc086223603e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Create new server test_reboot_server_hard

It sometimes fail in all gates [1]
This hack could highlight if it's a timeout issue or race conditions
between all tempest tests.

[1] http://artifacts.opnfv.org/functest/E5AZMH89OOK6/functest-opnfv-functest-smoke-cntt-hunter-tempest_full_cntt-run-142/tempest_full_cntt/tempest-report.html

Change-Id: I8d03aa10c3d822dacb983e5ca019f79e1c582c9e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Rename ext_network to floating_network in Rally tasks

http://artifacts.opnfv.org/functest/TCEP6YYTDWIG/functest-opnfv-functest-benchmarking-latest-rally_full-run-181/rally_full/rally_full.html#/NovaServers.boot_and_associate_floating_ip

Change-Id: I45d666e80a55802ab6c672dfb5f3cf93dc5381bb
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Increate neutron_tempest count to 517

http://artifacts.opnfv.org/functest/UM6QD3YXHAAC/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-104/tempest_neutron_cntt/tempest-report.html

Change-Id: If7df0bcd08629ffca60fcb20eb45847199b28e0c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Add VMTasks.dd_load_test in Rally test cases

It allows running dd in VM (simple disk benchmarking).

https://github.com/cntt-n/CNTT/issues/1477

Change-Id: I4f1150637460f6e0b661fc80fec2cc292efcc511
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Rename ext_network to floating_network in Rally tasks

It now fails [1]:
Traceback (most recent call last):
  File "/src/rally/rally/task/runner.py", line 70, in _run_scenario_once
    getattr(scenario_inst, method_name)(**scenario_kwargs)
  File "/usr/lib/python3.8/site-packages/rally_openstack/task/scenarios/nova/servers.py", line 1115, in run
    floatingip = self.neutron.create_floatingip(**create_floating_ip_args)
  File "/src/rally/rally/task/service.py", line 114, in wrapper
    return func(instance, *args, **kwargs)
  File "/src/rally/rally/task/atomic.py", line 91, in func_atomic_actions
    f = func(self, *args, **kwargs)
TypeError: create_floatingip() got an unexpected keyword argument 'ext_network'

[1] http://artifacts.opnfv.org/functest/JOGVTC31O1IL/functest-opnfv-functest-smoke-latest-rally_sanity-run-905/rally_sanity/rally_sanity.html#/NovaServers.boot_server_associate_and_dissociate_floating_ip/failures

Change-Id: I7157847980b417b814d15e4bf3bbac3c4f114690
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Add Patrole compute testing

It stops leveraging on policy json files as it fails for Nova.

Change-Id: Ic70997d0ff8151956dc189739b8468def048aad7
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Skip Patrole test_show_auto_allocated_topology

test_show_auto_allocated_topology doesn't clean its resources:
- network: auto_allocated_network
- subnets: auto_allocated_subnet_v4 & auto_allocated_subnet_v6
- router: auto_allocated_router

This bug easily breaks testing in a quick run when allocating the
subnet due to insufficient prefix space to allocate subnet size /26.

https://build.opnfv.org/ci/job/functest-latest-daily/746/

Change-Id: I1a1bd22068bca0720016dab8a15ad7dd576ee152
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Use alpine edge

Last upper-constraints updates force Alpine edge.
https://build.opnfv.org/ci/job/functest-core-docker-build-amd64-master/941/console

Change-Id: If2c7d434cfa45e676e3d236a2f59f916b43b5cd1
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Use default ssh_timeout (300) in Functest

196 is used in OpenStack gates but it doesn't fit the Functest model.
Functest runs all test cases in parallel vs the same SUT.

https://github.com/openstack/tempest/blob/master/tempest/lib/common/ssh.py#L38

Change-Id: I2db072a3cb691bc200cd4357852cdd8ceda91a1b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Add tempest_cyborg in Functest ansible role

Change-Id: I58ae820e10af29be757616fb5faf716453169e25
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Add tempest_cyborg based on cyborg tempest plugin

Change-Id: I61e982011040ee8b0540910939983777280445ca
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update Cirros to O.5.1

It conforms to the OpenStack gates which have just updated to Cirros
0.5.1

Change-Id: If2ec4b2da2152c244af7ea86a57054fea195c4ce
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update microversions (Train) in CNTT testing

Baraque selects Train:
https://github.com/cntt-n/CNTT/pull/1782
https://github.com/cntt-n/CNTT/pull/1783

Change-Id: Id9ee50133a3529e4c905b60f993762205178fe7c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Enable back manage_snapshot testing in CNTT

Baraque selects Train:
https://github.com/cntt-n/CNTT/pull/1782
https://github.com/cntt-n/CNTT/pull/1783

Change-Id: Ie715a86cf6129ab5b51bf9e0ec07f326c0f86ccb
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Enable back RemoteStack testing from tempest_heat_cntt

Baraque selects Train:
https://github.com/cntt-n/CNTT/pull/1782
https://github.com/cntt-n/CNTT/pull/1783

Change-Id: I9650f5b9041a59b68af34c7410c64515e498a226
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update Patrole tempest list if OVN

Change-Id: I66811c31fac4a83ed9634e928b9af8fa6e633034
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update tempest blacklist if OVN

It takes into account last Patrole changes.

Change-Id: I552442864297b51a1cd7885b357b139b664e536a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Remove distlib from OpenStack constraints

It raised side effects wih Alpine [1].

[1] https://build.opnfv.org/ci/job/functest-core-docker-build-amd64-master/933/console

Change-Id: Ic922c2b56afe67f2f3e436c562c045f4d0fdbef4
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Reduce Patrole extra logic

It also reduces the skipped test list taking into account Patrole
improvements.

Change-Id: I23a8c0610a8a4258aa806451f52796f3e7bc46ca
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Avoid compiling numpy if arm

It reaches Travis-ci timeouts

Change-Id: I30405cc1cd603a9dee96674d350d04eeb39e6add
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Patrole user -> member

Change-Id: I050caff2d255118edff48403a87305cfe336e20a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Create all /etc dirs per service

Change-Id: I6bb0e68b817a3776e0ca051eea93b199db0bac1b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Fix tempest_scenario

It completes "min_computes_nodes is also used as max values".

Change-Id: I72c67e17015bc35ac5e14f20839cf74353695373
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Generate all policies for patrole

Change-Id: Ica6455e3467ef2695d489170bd8f94553a6fdf06
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

min_computes_nodes is also used as max values

Its max value is now 10 according to the default quota.

Change-Id: Ie7bfcb224e821c15d38d0cba1d9f080548613e79
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Prepare full patrole testing

It adds the remaining services which didn't work before.
It may be needed for CNTT (see security discussion in vevent).

Change-Id: I36b3ad8bbfc8ffdcc427c3d97b92c78da27915c0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Add patrole user and reader testing

The regexes will be updated in a second change (chicken and egg
between Functest and Releng)

Change-Id: I4273a80799730fe30c941a7b2966cfee46f24d4a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update tempest_neutron_cntt regex and test count

It takes latest updates into account [1][2].

[1] http://artifacts.opnfv.org/functest/4W4YF3954M7H/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-85/tempest_neutron_cntt/tempest-report.html
[2] http://artifacts.opnfv.org/functest/K4DV91F2Y9V0/functest-ovn-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-76/tempest_neutron_cntt/tempest-report.html

Change-Id: Ia9b57e708522926fde9f8c44e6a0e936e97f304f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Simplify hypervisor list logic in vmtp

It completes the previous change "Allow listing hypervisors in VMTP" [1].

[1] https://gerrit.opnfv.org/gerrit/c/functest/+/70359

Change-Id: I12a19fd473076cfc3262f096cac1615c183710b1
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Remove build data in CNTT playbook

It's useless from a CNTT point of view.
It remains in the classical IaaS verification to ease creating
Functest Jenkins Jobs.

Change-Id: Icb13ab7d0e43b4cf968cab4041b6827c11c20cbc
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Allow listing hypervisors in VMTP

It allows listing the compute under tests via VMTP_HYPERVISORS.
They are no additional log in Functest and the note in VMTP is still
valid [1].

[1] https://opendev.org/x/vmtp/src/branch/master/vmtp/vmtp.py#L833

Change-Id: Ic9fbda3901d796c0ed94e42e2a14c468598f4311
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Increase timeout in shaker and vmtp

Running them in parallel of rally full and jobs asks for a increase.

Change-Id: Ib2369d384ac839fb46031411e1985ac402da0c1e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Reuse OpenStack master upper-constaints.txt

https://github.com/openstack/requirements/commit/c2f72e26a42eceb4b64f80d04db8253a4fbdb793

Change-Id: I0318c8614b09dc7cff1ffba4b813e55483df7eb3
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Stop leveraging on neutron plugins and extra

Neutron leverages on the upstream Rally task in Stein and newer.
Leveraging on the upstream task may fix the side effects which
sometimes occured on Vmtp and Shaker [1] and filled better the CNTT
model.

[1] https://build.opnfv.org/ci/view/functest/job/functest-hunter-daily/574/

Change-Id: I5ee2e3a3ca956cf57365bf4579534177e87dc08b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Install py3-wheel to avoid legacy installation

Change-Id: I5cfd3e370ef16785eeceaafefa18b45d2221c04b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Compile lxml with optimization in smoke

Else cross compilation raises travis-ci.org's timeouts [1]

[1] https://travis-ci.org/github/collivier/functest/jobs/698227192

Change-Id: I985321ac68a7a94cb8483d831f03222d44b3be57
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Clone repositories instead of wgetting constraints

It stops leveraging on wget to get constraints as it fails since
opendev was created. Then functest and requirements are no longer
removed for functest-core which simplifies all child Dockerfile.

It also fix the current incompatibilities in OpenStack constraints
ERROR: cliff 3.2.0 has requirement cmd2!=0.8.3,<0.9.0,>=0.8.0, but you'll have cmd2 1.1.0 which is incompatible.
https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L230
https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L416

Change-Id: I7c044208ef26ad047fb841710b422acb6c9f0320
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Skip EncryptionVolTypeTest in Iaas verification and CNTT Conformance

Attaching an encryted volume is optional in CNTT [1].

This test fails if the volume type already exists and if it runs
twice in parallel [2]. As this single test doesn't fullfil the Functest
requirements, it's skiped in the classical verification as well.

[1] https://github.com/cntt-n/CNTT/blob/master/doc/ref_arch/openstack/chapters/chapter05.md
[2] https://storyboard.openstack.org/#!/story/2007804

Change-Id: Ib6fc843d71704aab1bfc536133544eb3fa4f3e72
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Remove RemoteStack testing from tempest_heat_cntt

Trusts redelegation is supported by Train [1][2] which forces to skip
RemoteStack as CNTT Baldy is designed for Pike (the support is
incomplete in case of trust [3]).

[1] https://docs.openstack.org/releasenotes/heat/train.html#relnotes-13-0-0-stable-train
[2] https://github.com/openstack/heat/commit/e377658586c737150dad1bfd80b7b2267d46be51
[3] https://bugs.launchpad.net/heat/+bug/1701498

Change-Id: Ib4f66170b9b1f50b8224705edb4820e5a4cb0167
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Add missing tempest_octavia in patches

Change-Id: I0fae7487955895bf8f41046043425849f8fd0bf2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update all patches proposed by Functest

It mostly takes into account the last changes in test case names and
new cntt-related test cases.

Change-Id: I271fd02095b57f3b19b5ff95299fb4b257163c1f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update aarch64 configs

Change-Id: Ib53b9f97f30bf86d33857910ab9bffa1ce0d1e9e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Merge "Update functest/ci/testcases.yaml"

Merge "Stop increase thread stack size"

Update functest/ci/testcases.yaml

It takes all CNTT changes into account.

Change-Id: Ib6ddbd12695266fc78fd8ae4cce451bf8a4b95ad
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Stop increase thread stack size

It seems useless and then it limits the diff to Rally.

Change-Id: Ib745c7aad20ab5ab7fd4f8a5c7deb9d4aff30881
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update to alpine3.12

https://alpinelinux.org/posts/Alpine-3.12.0-released.html

Change-Id: Ie7c72abd10d70b90d65f260d595c0ca47b69a8c0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Increase tempest_neutron's tests_count

http://artifacts.opnfv.org/functest/Q3047D5AS3HJ/functest-ovn-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-40/tempest_neutron_cntt/tempest-report.html
http://artifacts.opnfv.org/functest/03PRDE6TZC14/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-49/tempest_neutron_cntt/tempest-report.html

Change-Id: I4be6d2f58e50572a3e8b90ede0055f36bc1d6886
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Switch back to concurrency = 4 in tempest testcases

3 seems good enough and stable which allows trying 4 as before

Change-Id: Id1f17ebf73f9faa13ed3ceb327ccb4ecf88c2bf2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Run VNF in parallel in Functest default playbook

They pass in parallel if the bandwidth is good enough.
https://build.opnfv.org/ci/view/functest/job/functest-latest-daily/679/

Change-Id: I2f466f7f25898d8af3004da1d3ef2208e4410f7c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Rename patch name and removes debug logs

The debug logs works only vs py3.8 and breaks backport to Iruya.

Change-Id: Ie91a8a876e263a83b6ae3fbfb5bb05644b87b970
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Switching to Threading instead of multiprocessing

Change-Id: Id5059a06447357f4c9b058bad374ed6cbe4d742c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Increase tests_count into tempest_neutron_cntt

http://artifacts.opnfv.org/functest/KJSOWXFF5CIQ/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-41/tempest_neutron_cntt/tempest-report.html

Change-Id: I04f0c3ae695f17e2bd56add11410e24ee25967d2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Switch to queue.Queue() in Rally

It's still failing.

Change-Id: I20fdf5a8e05bccc30a760934f6c0e3cac16d763c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Switch to SimpleQueue()

Change-Id: I1f3cb636813b9c3c10b5c829e35bbdeea02a318c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Switch to fork in Python multiprocessing (python2 default)

Change-Id: Id93da8c0fc710e1491adeb2b42e6da99974a0580
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Take last upstream changes into account in the regexes

http://artifacts.opnfv.org/functest/1NCC8YUFKE7X/functest-opnfv-functest-smoke-cntt-latest-tempest_slow_cntt-run-64/tempest_slow_cntt/tempest-report.html

Change-Id: I196421d96ada3ec1eb8fc718f72c3e796a0bd3b4
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Retry increasing Stack size

Change-Id: I61cb70a74179dea6a02e81989695325fe6331e10
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Skip manage_snapshot in CNTT

As detected by Orange's RC Field trial, Ceph doesn't support this
feature before Rocky [1].
"AttributeError: 'RBDDriver' object has no attribute 'unmanage_snapshot'"

manage_snapshot testing is fully passing in Functest SUT because CNTT
is verified vs OpenStack Rocky and newer.

Ceph is selected by RI as backend and is widely reused by the ecosytem.
It's similar to [2].

[1] https://bugs.launchpad.net/cinder/+bug/1645288
[2] https://gerrit.opnfv.org/gerrit/c/functest/+/69888

Change-Id: I3b7f88f5bed9a47a1e26e16bbbf74fea24ca76c2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Remove test_gnocchi.GnocchiGabbiTest.test_live

It sometimes fails in our gates and it seems unverified in OpenStack [1].
"Archive policy gabbilive is still in use"

[1] http://artifacts.opnfv.org/functest/DET32CBF0L0F/functest-opnfv-functest-smoke-jerma-tempest_telemetry-run-29/tempest_telemetry/tempest-report.html

Change-Id: Id5e5ad5ff0f7bade7932a085df28cc35b0ac7984
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update tests_count in tempest_full_cntt

http://artifacts.opnfv.org/functest/R1927JRCFJR5/functest-opnfv-functest-smoke-cntt-latest-tempest_full_cntt-run-62/tempest_full_cntt/tempest-report.html

Change-Id: Ia4b5bbdb92e7877d81acdcf9c4f3a9745badfb36
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Precise validation configs in Neutron

ServerActionsTestJSON.test_reboot_server_hard [1] sometimes fails in
gates. We may increase timeouts if it stills fails.

[1] http://artifacts.opnfv.org/functest/LIQH38CPR30X/functest-opnfv-functest-smoke-hunter-tempest_full-run-649/tempest_full/tempest-report.html

Change-Id: Ia27c0faead1df5c2b33a3e1a272e5a1010a5a83e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 7ccc1b0793e32cad3912e63bbca971d5c3ecd724)

Prevent microversion conflicts

CNTT selects Pike as baseline and Hunter matches Rocky.

Change-Id: I8a1aa8385adce1ebb9d74bcb897effaf5b434e5a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 82ceefd8f1609508a476aa2e4573e59926653165)

Add other debug messages in Rally

It would allow debugging our Rally failures.

Change-Id: I749a040fe76261e997d3a8dff4965867d54f7c42
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Increase tempest_neutron test_count to 512

http://artifacts.opnfv.org/functest/V0A7B0J1PIRL/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-17/tempest_neutron_cntt/tempest-report.html

Change-Id: I95dc0d8dec370dedb501878cbedfe5567ab86afd
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update shaker image

It updates pyshaker to 1.3.4 and adds fio which should be useful for
the next disk benchmarking testcases.

Change-Id: I662599745590eae71922ea5ef3f3876ad1019387
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Fix load_balancer options if ovn

Change-Id: I362b70cdfe753f9c57f30d61662e9b08564ebfe9
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Enable network_v6 testing in CNTT

req.inf.ntw.16 is a MUST requirement.

Fixes cntt-n/CNTT#1556

Change-Id: Id48595f5f4710ea9cbf31edaa4ae0ed31f5278ba
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update all results printed in docs

Change-Id: If69f74031c218e513464fda5be4e6fb86266292a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update octavia regexes

It stops using noop drivers and adds all scenarios except:
 - .*api.v2.test_availability_zone
 - .*api.v2.test_availability_zone_profile
 - .*api.v2.test_member.MemberAPITest.test_member_ipv4_create
 - .*api.v2.test_member.MemberAPITest.test_member_ipv6_create

It doesn't run ipv6 which asks for a deep dive (ipv6 from containers?)
It increases threads to 3 as the other testcases.

Change-Id: I36d9d764a5ab92efe89196181bffa3ad0a57227c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Stop overriding tempest_conf.yaml in CNTT

It's useless as they are skipped by regexes (vnc_console).

Change-Id: If3cea451c95179e7e200708e1f3b5e9534be9e1d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Avoid useless ssh in juju_epc

scp + ssh calls are replaced by the right scp call.

Change-Id: Iac2be63fb380486393af65dda8f7fd80bdd41d5f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Skip latest tests in tempest_full_cntt

http://artifacts.opnfv.org/functest/NQ4IHN2NFL69/functest-opnfv-functest-smoke-cntt-latest-tempest_full_cntt-run-41/tempest_full_cntt/tempest-report.html

Change-Id: Ia2a4e4229f02f5909ac87b5f6175230b194bc205
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Remove py27 in .travis-ci

py27 cannot be gated due to Rally.
https://travis-ci.org/github/collivier/functest/jobs/681899038

Change-Id: Ifeef8b99050182c7629b0767920724d94cd86c48
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Exclude tests.scenario.test_octavia_lbaas in CNTT regexes

It may be skipped as well (the behavior differs if it's enabled or not
via config).

Change-Id: Ic23119001b7478e466782a58d8757117bf16e5c3
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Rename barbican->tempest_barbican & octavia->tempest_octavia

It unifies all test case names.

Change-Id: I38576f2d6bae6ef0236b6acfbd30c0446d3a2eca
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Remove neutron_trunk; networking_bgpvpn and networking_sfc

They were all merged in neutron-tempest-plugin.

Change-Id: Iefe3bf98fc41d0dab5a7cec640337c672a7b1a69
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Skip LoadBalancerTest in Functest Gates

It seems failing due to octavia noop driver as in-used.

Change-Id: I53e1fe1cfe4930f92c1a4460e89ed6e21eb3f922
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Debug rally race conditions

It will be reverted as soon as the root cause is found.

Change-Id: Icbff74c5fb940c1a201cbdc3fb2137c91e1ed460
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update OpenStack release detection

Change-Id: Icad13836d88583ee676525d4743f12a5b888ebe2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Add tempest_telemetry

It runs all telemetry-tempest-plugin tests except test_autoscaling
which is currently failing in our gates.

Change-Id: I2c712eda71799e96735419e1c547a775be659156
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Updated from global requirements

Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
Change-Id: I3fc23ba64da0440a5fb571b34c561337415eba77

Rename neutron_tempest_plugin_api to tempest_neutron

It conforms with the other test case names

Change-Id: Ia8836375c26ea4f1bff3637f7078e5f70f83c5fb
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Complete "Skip CfnInitIntegrationTest in tempest_heat"

http://artifacts.opnfv.org/functest/SXA8X2BBP3L4/functest-opnfv-functest-smoke-cntt-latest-tempest_heat_cntt-run-1/tempest_heat_cntt/tempest-report.html

Change-Id: Ie26d561fbb490bcaafe7c8ffddebc28bd72ff51c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Increase neutron-tempest-api test_count to 510

http://artifacts.opnfv.org/functest/3L7CL8K6CU2N/functest-ovn-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-29/neutron-tempest-plugin-api-cntt/tempest-report.html

Change-Id: I077464ed0fc734b97de089f549fb66ab137fccdb
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Move static data to tempest_conf.yaml

Change-Id: I99f33f2e5f27c22c597f7c4f2f864710ceb1350e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Skip CfnInitIntegrationTest in tempest_heat

It sometimes fails due to other test resources [1].

[1] http://artifacts.opnfv.org/functest/83IV2XRCU05K/functest-opnfv-functest-smoke-latest-tempest_heat-run-53/tempest_heat/tempest-report.html

Change-Id: Ied91781c784370b424653e37c090a909f16b6585
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Revert "Increase Thread stack size"

It still fails [1].

[1] https://build.opnfv.org/ci/job/functest-ovn-opnfv-functest-benchmarking-latest-rally_jobs-run/28/console

Change-Id: I8560edb7dfbba804a94bb1453d5a489138d6a152
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Add tempest_heat in CNTT smoke

It increases Heat coverage [1].

[1] http://artifacts.opnfv.org/functest/24ELW4C5RKD4/functest-ovn-opnfv-functest-smoke-latest-tempest_heat-run-34/tempest_heat/tempest-report.html

Change-Id: I6630e4827a59b2ded34e5c271b9155df3b25f588
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Increase Thread stack size

It sets Glibc default value.
https://wiki.musl-libc.org/functional-differences-from-glibc.html
https://bugs.python.org/issue32307

Change-Id: I647569354b3a30f7de4116cab53c9e2aeff91a59
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Increase test_count in tempest_cinder

http://artifacts.opnfv.org/functest/C5BKAKPJIZVJ/functest-opnfv-functest-smoke-cntt-latest-tempest_cinder_cntt-run-26/tempest_cinder_cntt/tempest-report.html

Change-Id: I5c0a7db033024259180610b83b07c055f0393e4c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Use ujson version asked by OpenStack requirement

It's newer than the former tag selected [1].

[1] https://github.com/openstack/requirements/blob/master/upper-constraints.txt

Change-Id: Ie89136da364d7e972a1d7f49c71bdfbbdd9671d8
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Publish "CNTT RC April 2020 and beyond"

Please see the following rendering if it helps
http://testresults.opnfv.org/functest/vevent202004/

Change-Id: Iaf8baa017666e2f36792c1daf48a721ff6f3f044
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>