functest-kubernetes.git
3 years agoAdd missing blackslashing 45/72245/1
Cédric Ollivier [Mon, 22 Mar 2021 08:37:13 +0000 (09:37 +0100)]
Add missing blackslashing

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-leguer-sig_apps-run/2/console

Change-Id: I5cd41f2da49e59acd6bd06ded0c4e2178e7402b7
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig_node 41/72241/1
Cédric Ollivier [Sun, 21 Mar 2021 13:46:40 +0000 (14:46 +0100)]
Add sig_node

It runs sig-node tests which are not included in K8s conformance.
It adds 38 single tests.

Change-Id: Idbd172517e705402632a7fadb4d4639b5f1ee2d1
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig_cluster_lifecycle 26/72226/1
Cédric Ollivier [Sun, 21 Mar 2021 12:58:13 +0000 (13:58 +0100)]
Add sig_cluster_lifecycle

It runs sig-cluster-lifecycle tests which are not included in K8s conformance.
It adds 4 single tests.

Change-Id: I834523d753eec9bd570b8d4cc1cf3550b4a19bab
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig_instrumentation 25/72225/1
Cédric Ollivier [Sun, 21 Mar 2021 12:47:36 +0000 (13:47 +0100)]
Add sig_instrumentation

It runs sig-instrumentation tests which are not included in K8s conformance.
It adds 6 single tests.

Change-Id: Ic320f3bc7daed0aa2846a7ea672f13f3bec45f12
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig_scheduling 24/72224/1
Cédric Ollivier [Sun, 21 Mar 2021 12:30:30 +0000 (13:30 +0100)]
Add sig_scheduling

It runs sig-scheduling tests which are not included in K8s conformance.
It adds 19 single tests.

Change-Id: I9b34c327cc1079e401771b1fa77a6baeef24251c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig_cli 23/72223/1
Cédric Ollivier [Sun, 21 Mar 2021 12:18:49 +0000 (13:18 +0100)]
Add sig_cli

It runs sig-cli tests which are not included in K8s conformance.
It adds 34 single tests.

Change-Id: I4beaf9d08d151d06fc3f3c05770c51dcc68f762a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig-apps 22/72222/1
Cédric Ollivier [Sun, 21 Mar 2021 11:59:13 +0000 (12:59 +0100)]
Add sig-apps

It runs sig-apps tests which are not included in K8s conformance.

Change-Id: Ic02e75b94e6cd923f05af16eb98706599fd26f59
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig-auth 21/72221/1
Cédric Ollivier [Sun, 21 Mar 2021 11:52:46 +0000 (12:52 +0100)]
Add sig-auth

It runs sig-auth tests which are not included in K8s conformance.

Change-Id: I5684dbcb5d61866f5ae1cb483f3a119b8d3c7807
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig-api-machinery 20/72220/1
Cédric Ollivier [Sun, 21 Mar 2021 09:47:27 +0000 (10:47 +0100)]
Add sig-api-machinery

It runs sig-api-machinery which are not included in K8s conformance.
It adds 97 single tests.

Change-Id: I76949b75e2d6c59d21f459f8fc358e806af6f23d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoProvide support for air gapped env for rally 62/72162/4
Sylvain Desbureaux [Tue, 16 Mar 2021 13:09:29 +0000 (14:09 +0100)]
Provide support for air gapped env for rally

Sometimes, tested Kubernetes doesn't have direct access to Internet but
access through repository mirrors.
This patch handles this case for rally test case.

There's two ways for providing the repository mirrors:

- Give an environment variable (`MIRROR_REPO`) which gives a repository
  mirro with all needed images.
- Gives an environment variable per needed repo:
  - `DOCKERHUB_REPO` for DockerHub repository (`docker.io`)
  - `GCR_REPO` for Google Cloud repository (`gcr.io`)
  - `K8S_GCR_REPO` for Kubernetes repository (`k8s.gcr.io`)

Needed images list has also been extracted so Kubernetes administrator can
easily upload these images to the mirror if the mirror also doesn't have
access to Internet.

Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com>
Change-Id: I2ea6622b79f7e3c3c63c1441c4dab48e9bc4fb1a

3 years agoFix k8s.gcr.io/etcd tag (3.4.13-0) 57/72157/1
Cédric Ollivier [Mon, 15 Mar 2021 08:49:32 +0000 (09:49 +0100)]
Fix k8s.gcr.io/etcd tag (3.4.13-0)

Change-Id: Idbc520f1b196fd9d66dc57d32a543a6af7b73598
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agonit: highlight the release 89/72089/1
Cédric Ollivier [Wed, 17 Feb 2021 21:48:36 +0000 (22:48 +0100)]
nit: highlight the release

Change-Id: I09e3708d4afb477bdc767f124a778d0176f448a8
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoMove former wiki to README.md 87/72087/1
Cédric Ollivier [Wed, 17 Feb 2021 21:37:28 +0000 (22:37 +0100)]
Move former wiki to README.md

https://wiki.opnfv.org/pages/viewpage.action?pageId=20742412

Change-Id: I75a9303b2ee5b2df66e0169b37408bb9ad881d5a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd Cargo as build dev 79/71979/1
Cédric Ollivier [Thu, 11 Feb 2021 07:53:41 +0000 (08:53 +0100)]
Add Cargo as build dev

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-core-latest-build/2/console

Change-Id: Ic4ac415de09226d4ce2bb59a2790be5364efbf35
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoForce legacy pip resolver 78/71978/1
Cédric Ollivier [Wed, 10 Feb 2021 23:22:54 +0000 (00:22 +0100)]
Force legacy pip resolver

Change-Id: I6e3ad097333b529e4d80cbf52905e4e75cb96227
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoFix alpine version 56/71956/1
Cédric Ollivier [Wed, 10 Feb 2021 13:32:20 +0000 (14:32 +0100)]
Fix alpine version

It also install rust as it's now a build dependency.

Change-Id: I8fb11095b3fa29d40ec415c111015abd2dea2b34
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoUpgrade system first 50/71950/1
Cédric Ollivier [Wed, 10 Feb 2021 10:43:34 +0000 (11:43 +0100)]
Upgrade system first

Change-Id: I0120e6a368b033f2fb47257ff9dcc3fd8a58d243
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAllow parameterizing K8s versions 45/71845/1
Cédric Ollivier [Thu, 4 Feb 2021 11:57:09 +0000 (12:57 +0100)]
Allow parameterizing K8s versions

ONAP integration team asks to support all K8s versions.
A couple of OPNFV releases will allow testing 2 K8s versions.

Change-Id: Idb43cff0cc03cc9c17b290f7f1867e2c380c6319
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoRename sig_network_feature -> sig_network_features 00/71800/2
Cédric Ollivier [Tue, 26 Jan 2021 08:29:17 +0000 (09:29 +0100)]
Rename sig_network_feature -> sig_network_features

It also takes into account the last results (SCTP seems falky):
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_network-run/1/console
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_network_feature-run/3/console
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_storage-run/1/console

Change-Id: I7d4b11462da33d20fd5ed00d870f1099b70f437b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoExport GINKGO_PARALLEL=y 97/71797/2
Cédric Ollivier [Mon, 25 Jan 2021 17:42:13 +0000 (18:42 +0100)]
Export GINKGO_PARALLEL=y

It would speedup the testing.

Change-Id: I084bcf0d44a060f65eb9549eab8dcbbf5a4cc9d5
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoRun rally first 96/71796/1
Cédric Ollivier [Mon, 25 Jan 2021 17:00:10 +0000 (18:00 +0100)]
Run rally first

After all tests are based on e2e tests.

Change-Id: I727f40255ccc165f9a7da7fa7bdff1a629614d21
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig-storage 95/71795/1
Cédric Ollivier [Mon, 25 Jan 2021 16:32:36 +0000 (17:32 +0100)]
Add sig-storage

It runs the remaining sig-storage testing excluding the next labels:
Conformance (part of k8s_conformance), all features, all drivers and
Slow.

Change-Id: I486d8ecdfc46287f4774ba8acaa64d47fb1da287
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig_network 94/71794/1
Cédric Ollivier [Mon, 25 Jan 2021 11:20:22 +0000 (12:20 +0100)]
Add sig_network

It runs the remaining sig-network testing excluding the next labels:
Conformance (part of k8s_conformance), all features and load.balancer.

Change-Id: I708ff4690205196e5f69382890a39ea4fd5b6942
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAdd sig_network_feature 93/71793/1
Cédric Ollivier [Mon, 25 Jan 2021 09:13:44 +0000 (10:13 +0100)]
Add sig_network_feature

It verifies a couple of features by running a subset of e2e tests:
Ingress, Networking-IPv4, Networking-Performance, PerformanceDNS, SCTP
and NoSNAT. It excludes by default NetworkPolicy, Networking-IPv6,
IPv6DualStackAlphaFeature and Example. A couple of features may be
automatically skipped like NEG and kubemci.

Change-Id: I6efac39bb99c1deb8efe8e82d976d69d6540a9d1
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoUpdate to alpine:3.13 90/71790/1
Cédric Ollivier [Thu, 21 Jan 2021 15:49:53 +0000 (16:49 +0100)]
Update to alpine:3.13

Change-Id: I6087facf8ac62de3460861fe69e8f5270cf5d922
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoUpdate cnf-conformance to v0.9.19 72/71772/1
Cédric Ollivier [Sun, 10 Jan 2021 14:03:44 +0000 (15:03 +0100)]
Update cnf-conformance to v0.9.19

It also adds configs needed by resilience.

https://github.com/cncf/cnf-conformance/releases/tag/v0.9.19

Change-Id: I94b1efc47c5ceef96bc8879a6c71f92db1f63bd2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoSwitch to latest (v1.21.0-alpha.0) 70/71770/1
Cédric Ollivier [Sat, 9 Jan 2021 21:53:22 +0000 (22:53 +0100)]
Switch to latest (v1.21.0-alpha.0)

Change-Id: I30d7e4c8861811bf405ae79eb6d6c7bf82226dd6
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoUpdate to latest Kubernetes release 44/71744/1
Cédric Ollivier [Fri, 11 Dec 2020 16:16:18 +0000 (17:16 +0100)]
Update to latest Kubernetes release

Change-Id: Id2c05f8b425eeba1924bc2bc2688233beeb7457b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoUse pip==20.2.4 in tox venv 36/71736/1
Cédric Ollivier [Thu, 10 Dec 2020 18:11:46 +0000 (19:11 +0100)]
Use pip==20.2.4 in tox venv

pip 20.3 is disruptive and would lead to big changes in stable
branches.

Change-Id: Ic3770f4098c52b0d26bc62291c9e79e8ee81d107
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoSync with xrally_kubernetes 30/71530/1
Cédric Ollivier [Fri, 27 Nov 2020 08:06:16 +0000 (09:06 +0100)]
Sync with xrally_kubernetes

As upstream is fixed [1][2], it can partially revert
- Force Kubernetes 11.0 to please xrall_kubernetes
- Allow offline testing via xrally_kubernetes

[1] https://github.com/xrally/xrally-kubernetes/commit/2ffa85af2bff3438b6b23034b6ec6ee1de481090
[2] https://github.com/xrally/xrally-kubernetes/commit/8caf16cd017d475360b2345b3091a9881cd884d0

Change-Id: I43ca1966dcf61e55c45f13adf27ba5b0035d8fda
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoAllow offline testing via xrally_kubernetes 54/71454/3
Cédric Ollivier [Sun, 22 Nov 2020 11:09:12 +0000 (12:09 +0100)]
Allow offline testing via xrally_kubernetes

It locally patches xrally_kubernetes before [1] is merged upstream.
It adds docker.io/appropriate/curl:edge in download_images.sh too.

[1] https://github.com/xrally/xrally-kubernetes/pull/51

Co-Authored-By: Tamas Lendvay <tamas.lendvay@nokia.com>
Change-Id: I82ea2c777cd4d8f24c38dbf32d66081846e3e9af
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoReduce cnf-conformance criteria 48/71448/1
Cédric Ollivier [Sun, 22 Nov 2020 00:40:44 +0000 (01:40 +0100)]
Reduce cnf-conformance criteria

It's considered as passed vs 95 out of 105 [1].

[1] http://artifacts.opnfv.org/functest-kubernetes/ACI2B7JDLFHI/functest-kubernetes-opnfv-functest-kubernetes-cnf-kali-cnf_conformance-run-16/functest-kubernetes.log

Change-Id: Ib16927cf269784daba1085a3e97fb611e869928f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoUpdate cnf_conformance to v0.9.4 41/71441/2
Cédric Ollivier [Sat, 21 Nov 2020 12:06:25 +0000 (13:06 +0100)]
Update cnf_conformance to v0.9.4

It removes all hacks bypassing a few former bugs (e.g. [1]) and adapts
to the new cnf-conformance result output logic.

It runs all workload tests as it's now proposed (k8s conformance
is already part of smoke) and select the latest coredns
cnf-conformance.yml taking latest helm repo changes into account.

It sets criteria 95 as all steps are considered as passed and a
overall result 100/105.

[1] https://github.com/cncf/cnf-conformance/issues/388

Change-Id: I95b7889b21b86961e7a1d0c84b13e0ae367cae44
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoSkip cnf_conformance till helm repos are fixed 33/71433/1
Cédric Ollivier [Fri, 20 Nov 2020 11:19:32 +0000 (12:19 +0100)]
Skip cnf_conformance till helm repos are fixed

Change-Id: Ie5b9aff6f56ca5cfd8501f142969f8a6246c4151
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
3 years agoForce Kubernetes 11.0 to please xrall_kubernetes 32/71432/1
Cédric Ollivier [Fri, 20 Nov 2020 11:13:45 +0000 (12:13 +0100)]
Force Kubernetes 11.0 to please xrall_kubernetes

It will be reverted once xrally_kubernetes is ready.

Change-Id: If0ad5cd018ec4edeb8c150d789df051b3f55baed
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoAdd timeout as needed by Gitlab CI/CD 80/71380/3
Cédric Ollivier [Thu, 12 Nov 2020 16:26:25 +0000 (17:26 +0100)]
Add timeout as needed by Gitlab CI/CD

Gitlab CI/CD sets 1h as default timeout.
The timeout is valid for both test cases even if only k8s_conformance
reaches 1h (XtestingCI design).

Change-Id: Ieaef1cefb3dfc7f9ca47b1ce8ec082409fa52604
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit f38caec124b925f8c633f9d8b4474bd1085638a2)

4 years agoSplit Gates in a dedicated site.yml 48/71348/2
Cédric Ollivier [Mon, 9 Nov 2020 15:28:49 +0000 (16:28 +0100)]
Split Gates in a dedicated site.yml

Change-Id: Idd4458cc003b1baef717bb87846081b9afe3976f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoSplit kube_bench_master and kube_bench_node in site.cntt.yml 72/71172/1
Cédric Ollivier [Wed, 23 Sep 2020 07:12:48 +0000 (09:12 +0200)]
Split kube_bench_master and kube_bench_node in site.cntt.yml

Change-Id: Ifb9a5ac2fac7286d5f3f5e677fc2a7ecda6ade25
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoStop hardcoding ims- as generate_name 71/71171/1
Cédric Ollivier [Tue, 22 Sep 2020 14:47:04 +0000 (16:47 +0200)]
Stop hardcoding ims- as generate_name

Change-Id: I3ea22a4050ff1eb609cffb61edc41c49fab44366
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoRedirect helm false warnings to logs 60/71160/1
Cédric Ollivier [Sun, 20 Sep 2020 16:28:36 +0000 (18:28 +0200)]
Redirect helm false warnings to logs

Helm fails when detecting arch if qemu.
time="2020-09-17T16:25:15Z" level=error msg="failure getting variant" error="getCPUInfo for pattern: Cpu architecture: not found"

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-arm64-latest-helm_vims-run/1/console

Change-Id: Ida37062a780f9e2acf60035bf2d41fb8fbb61173
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoAdd helm_vims in site.cntt.yml 44/71144/1
Cédric Ollivier [Wed, 16 Sep 2020 09:34:51 +0000 (11:34 +0200)]
Add helm_vims in site.cntt.yml

https://github.com/cntt-n/CNTT/pull/2022

Change-Id: Icb4bdfa0c1ecca9d0c5b4d5beef7ff4c5edb289e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoPublish helm_vims also based on clearwater-docker 31/71131/1
Cédric Ollivier [Tue, 15 Sep 2020 21:18:16 +0000 (23:18 +0200)]
Publish helm_vims also based on clearwater-docker

It leverages helm rather than kubectl as proposed by k8s_vims.
A new abstract class Vims is added to factorize code between both
testcases.

Change-Id: Ie5de8d62e25e74f73f8e32167228a08e82989abd
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoIncrease timeout in vIMS 30/71130/1
Cédric Ollivier [Tue, 15 Sep 2020 08:03:30 +0000 (10:03 +0200)]
Increase timeout in vIMS

It depends on bandwidth.
30 minutes should be enough in all scenarios

Change-Id: Ieaad053c8f8a270e5da08275de217b15fdf17f44
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoDon't run disruptive hunter checks 21/71121/1
Cédric Ollivier [Mon, 14 Sep 2020 19:47:11 +0000 (21:47 +0200)]
Don't run disruptive hunter checks

Change-Id: I52cb8303950269946774546cf8e413166c70a33c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoSplit kube-bench master and node 07/71107/3
Cédric Ollivier [Sun, 13 Sep 2020 12:53:26 +0000 (14:53 +0200)]
Split kube-bench master and node

The former deployment asked for all-in-one.

Change-Id: I12e470cec9e82b82c6f3ea5ff2431087f5deb9be
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoself.details must be a dict 02/71102/1
Cédric Ollivier [Sun, 13 Sep 2020 08:44:38 +0000 (10:44 +0200)]
self.details must be a dict

Change-Id: I4f65a9eeb7eda471371668db9abfa49e2875c5b0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoImprove kube_bench output 97/71097/1
Cédric Ollivier [Sat, 12 Sep 2020 14:12:50 +0000 (16:12 +0200)]
Improve kube_bench output

It also fills self.details.

Change-Id: Ie73215ebcbd34de9d457fd364de4ab9cbdf64319
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoEnhance kube-hunter result postprocessing 88/71088/1
Cédric Ollivier [Sat, 12 Sep 2020 11:52:47 +0000 (13:52 +0200)]
Enhance kube-hunter result postprocessing

It fills self.details and checks if the test case passes according
to criteria (severity = high by default)

Change-Id: Ib20779b4b5dca078c65b546c8703bc99856c6f41
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoAvoid running VNF testing if deployment failed 87/71087/1
Cédric Ollivier [Sat, 12 Sep 2020 09:31:37 +0000 (11:31 +0200)]
Avoid running VNF testing if deployment failed

Change-Id: Id7642e47ec2cc438c5fe3a0ec9447f72b358adab
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoAdd clusterIP: None in ims svc.yaml 82/71082/1
Cédric Ollivier [Fri, 11 Sep 2020 22:42:07 +0000 (00:42 +0200)]
Add clusterIP: None in ims svc.yaml

Change-Id: I0709e83f32261af232c20f94c1edef7423e05527
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoRemove clusterIP: None in ims svc.yaml 76/71076/1
Cédric Ollivier [Thu, 10 Sep 2020 19:54:00 +0000 (21:54 +0200)]
Remove clusterIP: None in ims svc.yaml

It now works on minikube in addition to Functest gates.

Change-Id: I314989516f81dcddb3e615fe23480135f1d8cf9a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoAllow setting e2e non blocking taints via env vars 69/71069/1
Cédric Ollivier [Thu, 10 Sep 2020 15:07:53 +0000 (17:07 +0200)]
Allow setting e2e non blocking taints via env vars

Change-Id: I8132769ea23a376b2cae8b93beed27f5e66b9207
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoAdd quay.io/coreos/etcd:v2.2.5 in download_images.sh 65/71065/1
Cédric Ollivier [Thu, 10 Sep 2020 09:41:10 +0000 (11:41 +0200)]
Add quay.io/coreos/etcd:v2.2.5 in download_images.sh

Change-Id: I4b5da162dcb1efff810514498a20250c05afb3ba
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoPublish CNTT RC2 playbook 55/71055/1
Cédric Ollivier [Wed, 9 Sep 2020 17:25:12 +0000 (19:25 +0200)]
Publish CNTT RC2 playbook

It runs the mandatory CNTT test cases [1].

[1] https://github.com/cntt-n/CNTT/commit/6af033c254727b7405fbfb2efbaa023e2b8ba876

Change-Id: I87d381c78049f7309c564791ed701cf446cc85d2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoAdd helper to download all images 51/71051/3
Cédric Ollivier [Tue, 8 Sep 2020 21:27:50 +0000 (23:27 +0200)]
Add helper to download all images

It eases offline testing.
The default command leverages kind as in-use in Functest.

Change-Id: I66a20ede5e857f80cf49b9b708843034bc3b9904
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoSet all image tags 45/71045/2
Cédric Ollivier [Tue, 8 Sep 2020 20:42:48 +0000 (22:42 +0200)]
Set all image tags

It allows offline testing via xrally_kubernetes, k8s_vims, kube_bench
and kube_hunter.

Change-Id: I3084abec19f06a894d0083ecb3ed61882eddd785
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoRemove latest in all images 39/71039/1
Cédric Ollivier [Tue, 8 Sep 2020 18:45:18 +0000 (20:45 +0200)]
Remove latest in all images

The Kubernetes default pull policy is IfNotPresent unless the image tag
is :latest in which case the default policy is Always. IfNotPresent
causes the Kubelet to skip pulling an image if it already exists [1]

[1] https://kind.sigs.k8s.io/docs/user/quick-start/

Change-Id: I83dac6165d2bbef165ca852dd03e5b76a5356f2f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoRemove all imagePullPolicy: Always in ims depl yaml 16/71016/1
Cédric Ollivier [Tue, 8 Sep 2020 09:11:44 +0000 (11:11 +0200)]
Remove all imagePullPolicy: Always in ims depl yaml

It forbids offline testing.

Change-Id: I6790f64fd1382671300374b339ae038ef015cfab
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoOverride default Xtesting logs in cnf-conformance 02/71002/1
Cédric Ollivier [Mon, 7 Sep 2020 08:24:22 +0000 (10:24 +0200)]
Override default Xtesting logs in cnf-conformance

https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/7/console

Change-Id: Ia6845525e62274f7792b4cb1909f36204fb51418
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoDecreate cnf-conformance criteria 84/70984/1
Cédric Ollivier [Sun, 6 Sep 2020 13:32:11 +0000 (15:32 +0200)]
Decreate cnf-conformance criteria

conf-confirmance is beta.
Let's decrease criteria to allow gating [1].

[1] https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/4/console

Change-Id: I77068fd027b2ae042e0ff9574ed608c8a3d77ade
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoSlightly generalize ginkgo params 78/70978/1
Cédric Ollivier [Sun, 6 Sep 2020 11:57:11 +0000 (13:57 +0200)]
Slightly generalize ginkgo params

Change-Id: I51b013056bb5b88ef2e1de31e9171129e8ce4475
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoPublish k8s_quick as proposed by sonobuoy 73/70973/1
Cédric Ollivier [Sun, 6 Sep 2020 11:28:05 +0000 (13:28 +0200)]
Publish k8s_quick as proposed by sonobuoy

https://github.com/vmware-tanzu/sonobuoy

Change-Id: I7b1777cf7bc0d6cd04ea5cccc08b7c29887ae874
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoConform with latest sonubuoy's non disruptive list 70/70970/1
Cédric Ollivier [Sun, 6 Sep 2020 10:02:07 +0000 (12:02 +0200)]
Conform with latest sonubuoy's non disruptive list

https://github.com/vmware-tanzu/sonobuoy/blob/master/pkg/client/mode.go#L43

Change-Id: Ie6c331168e1186b33c4cabda6744e1bb2c6346a0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoUpdate Xtesting to 0.91.0 65/70965/1
Cédric Ollivier [Sun, 6 Sep 2020 08:34:06 +0000 (10:34 +0200)]
Update Xtesting to 0.91.0

Change-Id: I384478ec9a0926b168f2372c8007b3eceeadb6da
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoAllow skipping arm and arm64 if cnf-conformance 17/70917/1
Cédric Ollivier [Fri, 4 Sep 2020 19:24:55 +0000 (21:24 +0200)]
Allow skipping arm and arm64 if cnf-conformance

cnf-conformance doesn't support this architectures.
It also improve cleaning to publish artifacts [1].

[1] https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/2/console

Change-Id: Ie64e4a1b8b496a050a1798eaf4e5ef6397c011d0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoAdd cnf-conformance in functest-kubernetes-cnf 14/70914/3
Cédric Ollivier [Wed, 2 Sep 2020 17:46:25 +0000 (19:46 +0200)]
Add cnf-conformance in functest-kubernetes-cnf

It selects cnf-conformance v0.7.2-beta1 (current latest release) and
the default CNF as proposed by [1].

The end user is free to override the default yml files on purpose (e.g.
docker volumes). Tag (default: all) could be modified too via
testcases.yaml

[1] https://hackmd.io/@vulk/SkY54QnsU

Change-Id: Iec2a830c441e523e067f6eb114ecb205c49c6b93
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoIncrease cassandra timeouts 98/70898/1
Cédric Ollivier [Sun, 30 Aug 2020 11:39:42 +0000 (13:39 +0200)]
Increase cassandra timeouts

homestead-prov sometimes fails reaching cassandra
30-08-2020 10:33:44.064 UTC ERROR base.py:327: Translating internal <class 'telephus.cassandra.ttypes.TimedOutException'> error into a 503 status code
30-08-2020 10:34:32.257 UTC ERROR base.py:327: Translating internal <class 'telephus.cassandra.ttypes.TimedOutException'> error into a 503 status code

Change-Id: I48566e39dca3dc9644733a76140a76300bbfd84e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoStop pushing all details in Test DB 64/70864/1
Cédric Ollivier [Sun, 23 Aug 2020 02:53:16 +0000 (04:53 +0200)]
Stop pushing all details in Test DB

It raises 413 Client Error: Request Entity Too Large for url [1]

[1] https://build.opnfv.org/ci/job/functest-kubernetes-pi-ollivier-functest-kubernetes-benchmarking-latest-xrally_kubernetes_full-run/1/console

Change-Id: I165f7aec53d8ce1e567491a5b5559f00b1fe4a4c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoSet output logs in Rally testcases 59/70859/2
Cédric Ollivier [Sun, 23 Aug 2020 02:37:08 +0000 (04:37 +0200)]
Set output logs in Rally testcases

https://build.opnfv.org/ci/job/functest-kubernetes-pi-ollivier-functest-kubernetes-benchmarking-latest-xrally_kubernetes_full-run/1/console

Change-Id: I13685d74ce9e286520af6f1e5cd6ce9dafc3d5ea
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoFix kube-hunter command 58/70858/1
Cédric Ollivier [Sun, 23 Aug 2020 02:26:02 +0000 (04:26 +0200)]
Fix kube-hunter command

https://github.com/aquasecurity/kube-hunter/commit/3e06647b4c09257cb994bbdd174ee621e2af5406

Change-Id: Idf470f0161aaeb7a326a3e2a4e680445d9f00eac
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoFix last hardcoded concurrencies 51/70851/1
Cédric Ollivier [Sat, 22 Aug 2020 16:41:38 +0000 (18:41 +0200)]
Fix last hardcoded concurrencies

Change-Id: Ib2b7bb6de8cd6524fcc52baf48ebf5c4835c396b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoAdd xrally_kubernetes_full 44/70844/1
Cédric Ollivier [Sat, 22 Aug 2020 14:36:19 +0000 (16:36 +0200)]
Add xrally_kubernetes_full

It iterates 10 times all operations run by xrally_kubernetes.
Now times, concurrenty and count can be set via testcases.yaml.

It creates a new containers functest-kubernetes-benchamrking as
Functest.

Change-Id: Ibfd493b987a359a3f16c2d8db0aef6b8e90b7e92
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoMerge "Add tempo waiting for ellis account"
Cedric Ollivier [Tue, 18 Aug 2020 07:56:39 +0000 (07:56 +0000)]
Merge "Add tempo waiting for ellis account"

4 years agoAdd tempo waiting for ellis account 20/70820/1
Cédric Ollivier [Mon, 17 Aug 2020 19:26:16 +0000 (21:26 +0200)]
Add tempo waiting for ellis account

Change-Id: I63c6af9d0e20aab426e7f875cdb4f86d206f49aa
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoSafely decode e2e.test output 75/70775/3
Cédric Ollivier [Thu, 13 Aug 2020 23:04:04 +0000 (01:04 +0200)]
Safely decode e2e.test output

https://build.opnfv.org/ci/job/functest-kubernetes-pi-ollivier-functest-kubernetes-healthcheck-arm-latest-k8s_smoke-run/2/console

Change-Id: I57f3ae0051fb55c8c80de81e79168e61c06d85bb
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoSwitch to latest K9S v1.19 rc 03/70803/1
Cédric Ollivier [Mon, 17 Aug 2020 08:12:17 +0000 (10:12 +0200)]
Switch to latest K9S v1.19 rc

Change-Id: I3a8b45e8347b5a4eea3398cb2f84796b2893cace
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoFix arm -> arm64 92/70792/1
Cédric Ollivier [Sun, 16 Aug 2020 11:47:36 +0000 (13:47 +0200)]
Fix arm -> arm64

Change-Id: I3d7d16ec7d8a7712a279258f817d811f0d59187f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoLeverage alpine rather than golang 87/70787/1
Cédric Ollivier [Sun, 16 Aug 2020 09:56:12 +0000 (11:56 +0200)]
Leverage alpine rather than golang

golang is now useless as we stop compiling kubernetes binaries.

Change-Id: I7b9416281de069d1f454b13eff6e9e6107f7471e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoStop compiling K8s testing binaries 86/70786/1
Cédric Ollivier [Fri, 14 Aug 2020 11:58:13 +0000 (13:58 +0200)]
Stop compiling K8s testing binaries

It reaches Travis CI timeouts and is quite useless.
PyNaCl is now installed via Alpine packages to save CI.

Change-Id: Ifdf3f46f162a811ed95e3ca07ea7c3d89bc77a91
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoUpdate golang to 1.15 in site.yaml 78/70778/1
Cédric Ollivier [Fri, 14 Aug 2020 07:55:35 +0000 (09:55 +0200)]
Update golang to 1.15 in site.yaml

Change-Id: I8891ab01dcbeee7711fcb704535ce1621d0dc7c3
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoUpdate golang to 1.15 76/70776/1
Cédric Ollivier [Fri, 14 Aug 2020 07:29:36 +0000 (09:29 +0200)]
Update golang to 1.15

As asked by Kubernetes [1]

[1] https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-healthcheck-latest-gate/60/console

Change-Id: Ic57fe67f6568d19615599f7762a18170158c6771
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoConform with Functest pylint config 69/70769/1
Cédric Ollivier [Thu, 13 Aug 2020 20:01:31 +0000 (22:01 +0200)]
Conform with Functest pylint config

Change-Id: Ie12d17a32fe8ee7ee212a6a4acf6aec107377390
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoMerge "Make K8s security tests namespace aware"
Cedric Ollivier [Thu, 13 Aug 2020 15:16:10 +0000 (15:16 +0000)]
Merge "Make K8s security tests namespace aware"

4 years agoEase modifying the test list in E2E testing 59/70759/3
Cédric Ollivier [Thu, 13 Aug 2020 10:04:31 +0000 (12:04 +0200)]
Ease modifying the test list in E2E testing

It eases creating specific CNTT test cases based on K8s E2E testing.

Change-Id: I304960fda760ffc47d763d53511898699f63e356
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoMake K8s security tests namespace aware 64/70764/1
Cédric Ollivier [Thu, 13 Aug 2020 10:51:55 +0000 (12:51 +0200)]
Make K8s security tests namespace aware

It now creates a namespace to allow running the test cases twice in
parallel. It also overprotects clean operations to force a full delete.

Change-Id: Ie0becd8ea9126328e7280591bacc0d88e14dd031
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoMake K8s_vims namespace aware 54/70754/2
Cédric Ollivier [Thu, 13 Aug 2020 08:34:48 +0000 (10:34 +0200)]
Make K8s_vims namespace aware

It now creates a namespace to allow running the test case twice in
parallel (see DNS requests).

Change-Id: Ib47b3fb2d753f8259d9224cb79168cde68d8e854
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoRemove GOFLAGS in Dockerfile 40/70740/1
Cédric Ollivier [Tue, 11 Aug 2020 20:38:47 +0000 (22:38 +0200)]
Remove GOFLAGS in Dockerfile

Change-Id: Ifc6c63641b9512b4b930c1d7032cae55d2afb811
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoRemove useless repo in site.yaml 16/70716/1
Cédric Ollivier [Tue, 11 Aug 2020 17:23:25 +0000 (19:23 +0200)]
Remove useless repo in site.yaml

Change-Id: Ie5db13dda8dfcaa5bb26e39858448bddee8f6a2c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoStop building smoke if arm and arm64 15/70715/1
Cédric Ollivier [Tue, 11 Aug 2020 17:22:07 +0000 (19:22 +0200)]
Stop building smoke if arm and arm64

Change-Id: Ifef3eef410cdca58855bda471ef2be49bf3ba64f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoDon't build arm containers on TravisCI 13/70713/2
Cédric Ollivier [Tue, 11 Aug 2020 16:17:34 +0000 (18:17 +0200)]
Don't build arm containers on TravisCI

From the time being, they are reaching timeouts.

Change-Id: I5a2d6060f1f1042f630eeafa1876ed434e681e3a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoCompile all Functest K8s containers for arm v7 and v8 09/70709/1
Cédric Ollivier [Mon, 10 Aug 2020 19:40:11 +0000 (21:40 +0200)]
Compile all Functest K8s containers for arm v7 and v8

It removes sudo when useless and turns debug on when compiling K8s
binaries. It avoids TravisCI timeouts (No output has been received)[1]

[1] https://travis-ci.org/github/collivier/functest-kubernetes/jobs/716819529

Change-Id: I4719257596bf9a6510e2f772271b336847d1ccb5
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoOnly upgrade py3-distlib 84/70684/1
Cédric Ollivier [Thu, 6 Aug 2020 19:59:05 +0000 (21:59 +0200)]
Only upgrade py3-distlib

https://travis-ci.org/github/collivier/functest/jobs/715442363

Change-Id: Ib84615def124d38e29582b31af4229325fd3ab1a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoUpdated from global requirements 76/70676/2
Cédric Ollivier [Wed, 5 Aug 2020 23:40:15 +0000 (01:40 +0200)]
Updated from global requirements

Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
Change-Id: I3a60f9916ff96b1820c72bb9b45b3c919baeade1

4 years agoInstall py3-wheel in functest-kubernetes-core 74/70674/2
Cédric Ollivier [Wed, 5 Aug 2020 15:09:27 +0000 (17:09 +0200)]
Install py3-wheel in functest-kubernetes-core

It fixes all deprecated warnings.

Change-Id: Idd02668fac42a8e0c93d6df5a3e13b788716e873
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoInstall py3-distlib from Alpine Edge 78/70678/1
Cédric Ollivier [Thu, 6 Aug 2020 04:11:50 +0000 (06:11 +0200)]
Install py3-distlib from Alpine Edge

Alpine 3.12 doesn't offer a version compatible with latest OpenStack
constraints [1].

[1] https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-core-latest-gate/68/console

Change-Id: I70930f888d9612d44b73c5d87b9dc1663d02be1c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoBoot Ellis once homestead-depl is ready 17/70517/1
Cédric Ollivier [Thu, 9 Jul 2020 15:10:28 +0000 (17:10 +0200)]
Boot Ellis once homestead-depl is ready

Change-Id: If61a36020f3dfa5ba0a9858df769138257f9ec94
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoImprove vIMS service order 04/70504/1
Cédric Ollivier [Tue, 7 Jul 2020 14:28:10 +0000 (16:28 +0200)]
Improve vIMS service order

It seems fixing the last side effects.

Change-Id: Iee8506c2f0048f7c2619b772aea815288a40145e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoUpdate to go 1.14 02/70502/1
Cédric Ollivier [Tue, 7 Jul 2020 08:29:21 +0000 (10:29 +0200)]
Update to go 1.14

https://build.opnfv.org/ci/job/functest-kubernetes-healthcheck-docker-build-amd64-master/116/console

Change-Id: I8559ee71fe05682792728a62baa628c8199dfdd6
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoRemove energy from logging configs 94/70494/1
Cédric Ollivier [Tue, 7 Jul 2020 08:11:43 +0000 (10:11 +0200)]
Remove energy from logging configs

Change-Id: I35c15aa68fe1afadcdbf3d391febb166358c90be
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
4 years agoRemove distlib from OpenStack constraints 93/70493/1
Cédric Ollivier [Tue, 7 Jul 2020 07:51:07 +0000 (09:51 +0200)]
Remove distlib from OpenStack constraints

It raised side effects wih Alpine [1].

[1] https://build.opnfv.org/ci/job/functest-kubernetes-core-docker-build-amd64-master/126/console

Change-Id: If52bff670c7646e7cdc6703a6ed397a23a2be2b9
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>