Outdated need to install python2.7-dev in CircleCI

Change-Id: I6e25c16659f4a5653afbf9ed66d809cb638bce65
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit bdc882d05a70ec60fa6a4c9f8122ec29cae3f407)

Sync requirements

Change-Id: Id6fc0ff83663988b7d6e5058f067bf0c48005a78
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Set defaut provider to local back

It apply the same rules as sonuoboy.
Any user is free to set skeleton or something else via testcases.yml.

Change-Id: Id999dfd5fbcf9bde9430e0f35c63c41e80c8044f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit b100b1254e06719bf1c779b9d53d0ac2da67fca4)
(cherry picked from commit 329b58f2b897245dd91aedb280fec8db23ae25e1)

Fallback to item calculation

There is no final result if simple tag is called.

Change-Id: If86c5115e917e5790799fac4308cf08d57e87905
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit e2f255aaaddeb0b7bebd7a158b535aa0ecdf55e0)

Update CNF_TESTSUITE_TAG to v0.15.0

Change-Id: Icf13d49ec3c4fd65b0d089714728d67813633e49
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit fbd14c587ba477b7c9e3dcadabb295edbcc6ffbb)

Blacklist 2 features (out of Anuket)

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_instrumentation-run/38/console
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_node-run/38/console

Change-Id: Ieabfd42bdd98df9bf0c6fe3970045a07c2b9a38d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Skip Feature:CSIStorageCapacity in sig_storage

Change-Id: I701423895b3aabd51b95a8e7aff7dca15e1d5808
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit c1bc5af679651f035401c32f161620763f80763e)

Remove sig_network_serial

PerformanceDNS is now optional [1] and DNS configMap nameserver makes
operations on coredns which would break an living deployment [2].

sig_network_serial is now enpty and then can be safely removed.

[1] https://github.com/cntt-n/CNTT/pull/2580
[2] https://github.com/kubernetes/kubernetes/blob/c262d09bb7d22d62665aa215b5c6d1dd2a865007/test/e2e/network/dns_common.go#L238

Change-Id: I525cf894699a1d56f0b3cbb8e630d40ea3549b51
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 775135dd1ef01ab50a22846067bfdd87903f836e)

Remove SIG CLI testing

It conforms to https://github.com/cntt-n/CNTT/pull/2579

Change-Id: Ic5d6e6914fc34e8a5de9403c1b72ad251d97d759
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 0a68c0cb77c6d9967e5e33e59a737ef686b2e916)

Skip git_repo testing

As discussed in Kind [1], this test is for the deprecated in-tree GitRepo volume type.
"we shouldn't expend resources on these volume types".

[1] https://github.com/kubernetes-sigs/kind/issues/2356
[2] https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_storage_serial-run/16/console

Change-Id: I9951f49208376652e8d580b3086f72994424cf80
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 89dfd5adb8a1e4ac8a1278e421ef7fd90847c65e)

Print cnf_testsuite console on exceptions

Change-Id: I3c20993e1cbb644546ac89a8b4b7d1cfd98b0a80
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit f6d5010cd34db6cd639a7fcf5b6183aa117acacf)

Fix previous backport

It reverts IPv6DualStack to IPv6DualStackAlphaFeature

Change-Id: If7795d6e7ec37087079b2b7b598cae607b2873bd
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 573073379f57ca95a60ff6fa56ddef103810e803)

Make clear all skipped Features according to CNTT RA2 Chapter6

Change-Id: I57d414902e973d7bcbb67ba12d10b727b2ec4785
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 9e025ffa82fad7a5b49663a30987dcf03bb44ca3)

Allow passing extra opts to e2e.test

E2E_TEST_OPTS now allows setting runtime opts.
For instance, Functest K8s gates ask for -container-runtime containerd

Change-Id: I4f6e1caf57590b16d0c01324dadb040f67f3cee4
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 6ec9d5167032ed82e32cde592f98a4bcffca4c00)

Update cnf_testsuite to v0.11.2

https://github.com/cncf/cnf-testsuite/releases/tag/v0.11.2

Change-Id: Id3a5ea98260859f7fe5aba6ecf02d09905610dda
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit c35c6d443eb93db318c3436d995809dfe7e8389f)

Install libstdc++

Error loading shared library libstdc++.so.6

Change-Id: Ica164f5fddabd5508200ad0017e81de041c40b22
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Install matplotlib as needed by netperf

Change-Id: I55d03dbc5a35adcbf9b88d873b3f2bb7851d0d86
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Set --use-deprecated=legacy-resolver in pip calls

Change-Id: I3fbf72043b8c9145364eecfd091bba6553424b67
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Benchmark Kubernetes Networking Performance

https://github.com/kubernetes/perf-tests/tree/master/network/benchmarks/netperf

Change-Id: I6facd567f1c52c5949b53484a1fb107dcf34d622
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 3b5c1b115e234d636cb4f2a17d27ced872fee924)

Update test case results

Change-Id: I728e4d81b86f5ec851b513fd88f7501230c5350a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Skip [Feature:Federation] in sig_network_serial

Change-Id: I005d6207311e661a0ce6695f24e3b91703926692
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Fix cnf-conformance results filename

It also decreases criteria to 1 due to massive changes (dockerd).

Change-Id: Ic420139e6355debae183a77d6b0c574801b05076
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update cnf-conformance to cnf-testsuite

https://github.com/cncf/cnf-conformance
https://github.com/cncf/cnf-testsuite

Change-Id: I77983e0f04781a92b34afcf82f21366384782964
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 959da00d3e1c4d406bc59995c0ea93add1ec48d3)

Reduce number of threads

It passed successfully with 10 threads (more doesn't reduce the
overall duration so much).

Change-Id: I9013897883a40ebaf699c9a180f1bc8a6a34505c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Mock os.makedirs

Change-Id: I5f9fd24421727cea474715f502670ae6f2c80b76
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 32d2ed700c449ba33a4060f55c2adadc2e76b070)

Install setuptools_scm before xrally

Else upper-constraints basically fails.

Change-Id: I022f101e2ed44708afd0b88d47b77d2f7ca85458
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 75170fc0b528c7e182b8164d5905fcbaa6957faa)

Skip Network.should.set.TCP.CLOSE_WAIT.timeout

https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-release/release-branch-jobs/1.21.yaml

Change-Id: I9d8065fe84d3f6f5e74bf9b46bc8c85ad985c8b1
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 16f4a0c29fcfd1c6dc577394d9627845f559f177)

Remove sig_node_serial as it's fully disruptive

There is no other test when NoExecuteTaintManager is skipped.
http://artifacts.opnfv.org/functest-kubernetes/T94WVHSMW8ZH/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_scheduling_serial-run-6/sig_scheduling_serial/e2e.log

Change-Id: I09a81ec973b26fc6de32df6788e73c7e529ecbf0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit a6dda83e6d6da08f776010a7cdebff77c3c7c2c0)

Skip SetHostnameAsFQDN in sig-node

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_node-run/14/console

Change-Id: I55c9d3404f9bca64354a6276881a71624281a59b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Disable Feature:StackdriverLogging in sig_instrumentation

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_instrumentation-run/8/console

Change-Id: I6634ec335b632ac6c7efb9b2078eb1c5f3d9be31
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Disable a few sig-auth features

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_auth-run/8/console

Change-Id: I8a1523ba293f40dc8533428f9d75b29560a817cc
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Update test case list

Change-Id: Ide5fdd5967ce11847f9672259b79b6068c228848
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit c1ea4de55608b931462cd0a8d2a3a8325e5888e5)

Add k8s_io and serial test cases

It completes the first sig test cases.

Change-Id: I6b018177e93efe60f9de67f384e388d2a584536c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Merge sig_network_features in sig_network

Change-Id: I33516a7b0f280355c40aa821738a0625e4fae93a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit adcbd42cabfa42a71410dd0c76ece0f367d338fd)

Spit conformance into 2 test cases

It hugely decreases the overall test duration.
(10 + 15 << 120)

Change-Id: I534f6d09956bb51c32ad35c5a1baff8ebbfabb3d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit a33e339a001d62352e15a3cc1546bb4d4d482e59)

Fix etcd image repository in etcd-depl.yaml

Change-Id: I4447b4030b141b691b3a2cdf05c97c79de9eb72a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit b134d6735e722261bfe10c1ba83112e9cf95f85c)

Fix name in chronos-depl.yaml

Change-Id: Ia6972a216daaa9d807058b34b55615cab2c09b64
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 2eaa659eee7c83fd9221f88a207514e8e909943f)

Run opnfv/functest-kubernetes-smoke sequentially

Now multithreading is enabled at test case level.

Change-Id: I17a9bdb07216cd645483a0a893da502abee2ade0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit bbc8501437c1112d2e5c53281622d3dbe25b0d61)

Properly manage e2e parallel testing

It sets nodes=25 to run all tests in parallel.
It also beautifies all testcases.yml

Change-Id: I3ecd8899e957c0efbef34ab0409c2b4d557c9425
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 4004d776b01a1b038df52ca45b18430da1b6b1f5)

Rather use skeleton instead of local

“skeleton”: cluster is accessed via the Kubernetes API and nothing
else.

https://kubernetes.io/blog/2019/03/22/kubernetes-end-to-end-testing-for-everyone/

Change-Id: Iea8b65b2ac55d40572c0078b7a9a956d1c54d62a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 17929b0d015034bad2796fb733627fe5b23bbb2d)

 Provide support for air gapped env for e2e

 Sometimes, tested Kubernetes doesn't have direct access to Internet but
 access through repository mirrors.
 This patch handles this case for e2e test case.

Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com>
Change-Id: Iaa2960ad03aae1fff859d41bcd6ddf3176658f8e
(cherry picked from commit 27dbb7c14bac45a9af8d18e1913a2be6815cea43)

Export GINKGO_PARALLEL=y

It would speedup the testing.

Change-Id: I084bcf0d44a060f65eb9549eab8dcbbf5a4cc9d5
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 65d860c6ee36641ad7d248f7dc8a6f63f3173301)

Provide support for air gapped env for ims

Sometimes, tested Kubernetes doesn't have direct access to Internet but
access through repository mirrors.

This patch handles this case for ims test cases.

Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com>
Change-Id: I3b5e78fcf43ca7af6bcfd2d87fa4bb0705373697
(cherry picked from commit 9c95b5f46e57365c1f6e7e5c9073f7fe2ad48a66)

Disable sig_scheduling as serial

All remaining sig-scheduling tests are serial and for the time
Functest doesn't run serial test cases.

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_scheduling-run/13/console

Change-Id: I3f09077a1873b92120daa3a97cbb6f68558e2b4d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit f28b4673a7b89f124d225e629f0c5267e6008c14)

Remove the useless Feature prefix

Change-Id: Ibec3e1cee170a91a4333b3c625ae39cd0c4bbe11
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Exlude Serial and Feature:Feature:StorageVersionAPI

Serial asks to split the testcases into 2 runs.

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_api_machinery-run/7/console

Change-Id: I58f9d9023f596141bbefae8339043f813a124854
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Blacklist all alpha testing

Change-Id: Ifb9edb0d70f285825eb2cfee05a53590367b2650
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Leverage py3-setuptools_scm as proposed by Alpine

Last version setuptools-scm asks for very new setuptools.
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-xrally_kubernetes-run/194/console

Change-Id: Ib1f4b4b49c036e0018fd695b52013bccdaf4998f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Provide support for air gapped env for security

Sometimes, tested Kubernetes doesn't have direct access to Internet but
access through repository mirrors.
This patch handles this case for security test cases.

Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com>
Change-Id: I699d065ee691596c4a5ccf06c22ea76ef00fe497
(cherry picked from commit af623f6b45c9357b0e33210f54db64b270d162bf)

Add missing blackslashing

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-leguer-sig_apps-run/2/console

Change-Id: I5cd41f2da49e59acd6bd06ded0c4e2178e7402b7
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 0b79c61af319c6f3beddf79e586e51e4675f3fc1)

Add sig_node

It runs sig-node tests which are not included in K8s conformance.
It adds 38 single tests.

Change-Id: Idbd172517e705402632a7fadb4d4639b5f1ee2d1
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 56da8a0382889fa0558da48375d2319bfedd39a4)

Add sig_cluster_lifecycle

It runs sig-cluster-lifecycle tests which are not included in K8s conformance.
It adds 4 single tests.

Change-Id: I834523d753eec9bd570b8d4cc1cf3550b4a19bab
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit f3d390ddba5021ce45dd763e2bf953994a487337)

Add sig_instrumentation

It runs sig-instrumentation tests which are not included in K8s conformance.
It adds 6 single tests.

Change-Id: Ic320f3bc7daed0aa2846a7ea672f13f3bec45f12
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 39511d8576f49b578882757fcc1e33a4bb2c9832)

Add sig_scheduling

It runs sig-scheduling tests which are not included in K8s conformance.
It adds 19 single tests.

Change-Id: I9b34c327cc1079e401771b1fa77a6baeef24251c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 8bfdc8569b9eaffb96562e7753423d3ded218ed4)

Add sig_cli

It runs sig-cli tests which are not included in K8s conformance.
It adds 34 single tests.

Change-Id: I4beaf9d08d151d06fc3f3c05770c51dcc68f762a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 6eaf848638d7a1a7b4c4324837da9b8562ec7136)

Add sig-apps

It runs sig-apps tests which are not included in K8s conformance.

Change-Id: Ic02e75b94e6cd923f05af16eb98706599fd26f59
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 985b7955d065065d191702f1355d7ca114f0c827)

Add sig-auth

It runs sig-auth tests which are not included in K8s conformance.

Change-Id: I5684dbcb5d61866f5ae1cb483f3a119b8d3c7807
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit b5c7bc0107a443722004ecced91118042965f327)

Add sig-api-machinery

It runs sig-api-machinery which are not included in K8s conformance.
It adds 97 single tests.

Change-Id: I76949b75e2d6c59d21f459f8fc358e806af6f23d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 9c08bb471502b80df2470e577376f25400913939)

Provide support for air gapped env for rally

Sometimes, tested Kubernetes doesn't have direct access to Internet but
access through repository mirrors.
This patch handles this case for rally test case.

There's two ways for providing the repository mirrors:

- Give an environment variable (`MIRROR_REPO`) which gives a repository
  mirro with all needed images.
- Gives an environment variable per needed repo:
  - `DOCKERHUB_REPO` for DockerHub repository (`docker.io`)
  - `GCR_REPO` for Google Cloud repository (`gcr.io`)
  - `K8S_GCR_REPO` for Kubernetes repository (`k8s.gcr.io`)

Needed images list has also been extracted so Kubernetes administrator can
easily upload these images to the mirror if the mirror also doesn't have
access to Internet.

Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com>
Change-Id: I2ea6622b79f7e3c3c63c1441c4dab48e9bc4fb1a
(cherry picked from commit b26ac308d4d441195fcd804d4d0ff36356fa8a90)

Fix k8s.gcr.io/etcd tags

Kali is used to verify both v1.18 and v1.19 which respectively ask for
k8s.gcr.io/etcd:3.4.3 and k8s.gcr.io/etcd:3.4.13-0.

Co-Authored-By: Dominique Lagadec <dominique.lagadec@ericsson.com>
Change-Id: I739202b0e58352fd3308d705807910cd1575d28f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Rename sig_network_feature -> sig_network_features

It also takes into account the last results (SCTP seems falky):
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_network-run/1/console
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_network_feature-run/3/console
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_storage-run/1/console

Change-Id: I7d4b11462da33d20fd5ed00d870f1099b70f437b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 08558d2f47f7400176ad678ceaaff1b5be0b931c)
(cherry picked from commit dcb442250783d97b2bbfc6e5671a18cc90106792)

Run rally first

After all tests are based on e2e tests.

Change-Id: I727f40255ccc165f9a7da7fa7bdff1a629614d21
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 9e132759f6186672dd23dd31fac511bec128d386)

Add sig-storage

It runs the remaining sig-storage testing excluding the next labels:
Conformance (part of k8s_conformance), all features, all drivers and
Slow.

Change-Id: I486d8ecdfc46287f4774ba8acaa64d47fb1da287
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 8e6f69afa84ae428a013debcc7329be26a475695)

Add sig_network

It runs the remaining sig-network testing excluding the next labels:
Conformance (part of k8s_conformance), all features and load.balancer.

Change-Id: I708ff4690205196e5f69382890a39ea4fd5b6942
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 80f0edf3194493620d1cfab79efbcdc8a136d799)

Add sig_network_feature

It verifies a couple of features by running a subset of e2e tests:
Ingress, Networking-IPv4, Networking-Performance, PerformanceDNS, SCTP
and NoSNAT. It excludes by default NetworkPolicy, Networking-IPv6,
IPv6DualStackAlphaFeature and Example. A couple of features may be
automatically skipped like NEG and kubemci.

Change-Id: I6efac39bb99c1deb8efe8e82d976d69d6540a9d1
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit e0d151ccaf4b267682feec0d38e8b7de3eb3d61c)

Move former wiki to README.md

https://wiki.opnfv.org/pages/viewpage.action?pageId=20742412

Change-Id: I75a9303b2ee5b2df66e0169b37408bb9ad881d5a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 15f754678e4c8734c4600fea94fcbe66f2584515)
(cherry picked from commit a33e2fe3956ce5dcf0893454131121af8e73f863)

Upgrade system first

Change-Id: I0120e6a368b033f2fb47257ff9dcc3fd8a58d243
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 9ebe47c8458c99066fd71a65007fff923004d209)

Allow parameterizing K8s versions

ONAP integration team asks to support all K8s versions.
A couple of OPNFV releases will allow testing 2 K8s versions.

Change-Id: Idb43cff0cc03cc9c17b290f7f1867e2c380c6319
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 0163bb977604a32a0c46e10a97313ce0feea7b5f)

Update cnf-conformance to v0.9.19

It also adds configs needed by resilience.

https://github.com/cncf/cnf-conformance/releases/tag/v0.9.19

Change-Id: I94b1efc47c5ceef96bc8879a6c71f92db1f63bd2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 7385caf54c27b471bda8599d491aa3aee21b7b89)

Use pip==20.2.4 in tox venv

pip 20.3 is disruptive and would lead to big changes in stable
branches.

Change-Id: Ic3770f4098c52b0d26bc62291c9e79e8ee81d107
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 0c99749bb65cadd0142c90cd73e13802d48cb7f6)

Allow offline testing via xrally_kubernetes

It locally patches xrally_kubernetes before [1] is merged upstream.
It adds docker.io/appropriate/curl:edge in download_images.sh too.

[1] https://github.com/xrally/xrally-kubernetes/pull/51

Co-Authored-By: Tamas Lendvay <tamas.lendvay@nokia.com>
Change-Id: I82ea2c777cd4d8f24c38dbf32d66081846e3e9af
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit ea2715240015a482c5321d309257d801a412f02d)

Reduce cnf-conformance criteria

It's considered as passed vs 95 out of 105 [1].

[1] http://artifacts.opnfv.org/functest-kubernetes/ACI2B7JDLFHI/functest-kubernetes-opnfv-functest-kubernetes-cnf-kali-cnf_conformance-run-16/functest-kubernetes.log

Change-Id: Ib16927cf269784daba1085a3e97fb611e869928f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 53b16e8f00e5a60db2e6db7f375bd06f184a5cc8)

Update cnf_conformance to v0.9.4

It removes all hacks bypassing a few former bugs (e.g. [1]) and adapts
to the new cnf-conformance result output logic.

It runs all workload tests as it's now proposed (k8s conformance
is already part of smoke) and select the latest coredns
cnf-conformance.yml taking latest helm repo changes into account.

It sets criteria 95 as all steps are considered as passed and a
overall result 100/105.

[1] https://github.com/cncf/cnf-conformance/issues/388

Change-Id: I95b7889b21b86961e7a1d0c84b13e0ae367cae44
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 854f164998793000fcf8019214b259b7283f5194)

Skip cnf_conformance till helm repos are fixed

Change-Id: Ie5b9aff6f56ca5cfd8501f142969f8a6246c4151
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 30a5a084fa9e0c8677d70d8b1d7a696ea06b49b9)

Add timeout as needed by Gitlab CI/CD

Gitlab CI/CD sets 1h as default timeout.
The timeout is valid for both test cases even if only k8s_conformance
reaches 1h (XtestingCI design).

Change-Id: Ieaef1cefb3dfc7f9ca47b1ce8ec082409fa52604
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit f38caec124b925f8c633f9d8b4474bd1085638a2)

Split Gates in a dedicated site.yml

Change-Id: Iae0e61696987696fda3d9da05d75c5d87908b92a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Split kube_bench_master and kube_bench_node in site.cntt.yml

Change-Id: Ifb9a5ac2fac7286d5f3f5e677fc2a7ecda6ade25
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 687723872c04c4b889cf4b529f6534776b16908d)

Stop hardcoding ims- as generate_name

Change-Id: I3ea22a4050ff1eb609cffb61edc41c49fab44366
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 3a3bd294ea8101c22896d8601fe2723861f73124)

Redirect helm false warnings to logs

Helm fails when detecting arch if qemu.
time="2020-09-17T16:25:15Z" level=error msg="failure getting variant" error="getCPUInfo for pattern: Cpu architecture: not found"

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-arm64-latest-helm_vims-run/1/console

Change-Id: Ida37062a780f9e2acf60035bf2d41fb8fbb61173
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit aed4dd8a97dbfcb9d0558f2df8f8ef8696f2c421)

Add helm_vims in site.cntt.yml

https://github.com/cntt-n/CNTT/pull/2022

Change-Id: Icb4bdfa0c1ecca9d0c5b4d5beef7ff4c5edb289e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 63476e09e51c5c847df47b0855ee9d1b0ea1add1)

Publish helm_vims also based on clearwater-docker

It leverages helm rather than kubectl as proposed by k8s_vims.
A new abstract class Vims is added to factorize code between both
testcases.

Change-Id: Ie5de8d62e25e74f73f8e32167228a08e82989abd
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 32dfc10f56ead9abea8798e1ce16fe87652ca480)

Increase timeout in vIMS

It depends on bandwidth.
30 minutes should be enough in all scenarios

Change-Id: Ieaad053c8f8a270e5da08275de217b15fdf17f44
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit f6e86e451a0f5bf0e9775482c487d86249b935bd)

Don't run disruptive hunter checks

Change-Id: I52cb8303950269946774546cf8e413166c70a33c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 4fa706f4b37d655f2239ad9381df8041e0701142)

Split kube-bench master and node

The former deployment asked for all-in-one.

Change-Id: I12e470cec9e82b82c6f3ea5ff2431087f5deb9be
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit bced94b6fe24c7e939fb22834deb77477e4a9bb9)

self.details must be a dict

Change-Id: I4f65a9eeb7eda471371668db9abfa49e2875c5b0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit b866c73d70079fdb52e5fc999f49a32d2af82349)

Improve kube_bench output

It also fills self.details.

Change-Id: Ie73215ebcbd34de9d457fd364de4ab9cbdf64319
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 823b3a005ace9372a080421b2ae19152505201e4)

Enhance kube-hunter result postprocessing

It fills self.details and checks if the test case passes according
to criteria (severity = high by default)

Change-Id: Ib20779b4b5dca078c65b546c8703bc99856c6f41
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 4f197110710e9f148eae4533792e8e7e2d72f053)

Avoid running VNF testing if deployment failed

Change-Id: Id7642e47ec2cc438c5fe3a0ec9447f72b358adab
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 3afa7d700eebf2bfcaabab7bdd3aa09198ba4fe0)

Add clusterIP: None in ims svc.yaml

Change-Id: I0709e83f32261af232c20f94c1edef7423e05527
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 70d96ca61f1f11aafcf9e6975e61bce41febd371)

Remove clusterIP: None in ims svc.yaml

It now works on minikube in addition to Functest gates.

Change-Id: I314989516f81dcddb3e615fe23480135f1d8cf9a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 757fcb55867aa8c0545543a87ec9741827e9ab50)

Allow setting e2e non blocking taints via env vars

Change-Id: I8132769ea23a376b2cae8b93beed27f5e66b9207
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit d4af425da9a60c0eb8e80cb74972b99e3deadeb8)

Add quay.io/coreos/etcd:v2.2.5 in download_images.sh

Change-Id: I4b5da162dcb1efff810514498a20250c05afb3ba
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit ef4ef77dfda94d02e2d933318c1b0072da224e43)

Publish CNTT RC2 playbook

It runs the mandatory CNTT test cases [1].

[1] https://github.com/cntt-n/CNTT/commit/6af033c254727b7405fbfb2efbaa023e2b8ba876

Change-Id: I87d381c78049f7309c564791ed701cf446cc85d2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 463f233c0abb77f6758e7c39fa05b5c118fbaf1f)

Add helper to download all images

It eases offline testing.
The default command leverages kind as in-use in Functest.

Change-Id: I66a20ede5e857f80cf49b9b708843034bc3b9904
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 9913456884a19525bae18226ef0a6233f00993b2)

Set all image tags

It allows offline testing via xrally_kubernetes, k8s_vims, kube_bench
and kube_hunter.

Change-Id: I3084abec19f06a894d0083ecb3ed61882eddd785
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 8d748b9efc9ed4ed3edb4eaaf09909e41fe29577)

Remove latest in all images

The Kubernetes default pull policy is IfNotPresent unless the image tag
is :latest in which case the default policy is Always. IfNotPresent
causes the Kubelet to skip pulling an image if it already exists [1]

[1] https://kind.sigs.k8s.io/docs/user/quick-start/

Change-Id: I83dac6165d2bbef165ca852dd03e5b76a5356f2f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 44492e0da74473460ff6a9f1323c5df3a2af943e)

Remove all imagePullPolicy: Always in ims depl yaml

It forbids offline testing.

Change-Id: I6790f64fd1382671300374b339ae038ef015cfab
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 6371491ee2bc134c60abaa6a1deec18b9ffe22e2)

Update Kali to v1.19

It follows the last stable version as expected by RA2 Baraque.

Change-Id: I962663516c38a2a879c3f263a8df2d1de949ca14
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

Override default Xtesting logs in cnf-conformance

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/7/console

Change-Id: Ia6845525e62274f7792b4cb1909f36204fb51418
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit e09c7eeab3a1bdc361769e469c0ec7a7b5718ebf)

Decreate cnf-conformance criteria

conf-confirmance is beta.
Let's decrease criteria to allow gating [1].

[1] https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/4/console

Change-Id: I77068fd027b2ae042e0ff9574ed608c8a3d77ade
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 12d6bc18b178d4a8a4f64574bd6549341268645a)