X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=blobdiff_plain;f=docker%2Fsmoke%2Ftestcases.yaml;h=1266ae0938587934b2b0baa9f278a79b0acc65ae;hb=8998d6650bd3c3b3872e73b15f5f3330bef71d64;hp=ed0cc67430d948cac178e05d2c15e3fa26c634d3;hpb=9e132759f6186672dd23dd31fac511bec128d386;p=functest-kubernetes.git diff --git a/docker/smoke/testcases.yaml b/docker/smoke/testcases.yaml index ed0cc674..1266ae09 100644 --- a/docker/smoke/testcases.yaml +++ b/docker/smoke/testcases.yaml @@ -1,97 +1,346 @@ --- tiers: - - - name: smoke - description: >- - A set of conformance tests integrated from kubernetes project. - testcases: - - - case_name: xrally_kubernetes - project_name: functest - criteria: 100 - blocking: false - description: >- - All in one tasks for checking basic functionality of - Kubernetes cluster. - dependencies: - - DEPLOY_SCENARIO: 'k8-*' - run: - name: xrally_kubernetes - - - case_name: k8s_conformance - project_name: functest - criteria: 100 - blocking: false - description: >- - Conformance Tests are expect to pass on any Kubernetes - cluster, this tests are necessary demonstrate conformance - of the Kubernetes deployment. - dependencies: - - DEPLOY_SCENARIO: 'k8-*' - run: - name: e2e_testing - args: - focus: \[Conformance\] - skip: \[Disruptive\]|NoExecuteTaintManager - - - case_name: sig_network - project_name: functest - criteria: 100 - blocking: false - description: >- - It runs the remaining sig-network testing excluding the - next labels: Conformance (part of k8s_conformance), all - features and load.balancer. - dependencies: - - DEPLOY_SCENARIO: 'k8-*' - run: - name: e2e_testing - args: - focus: \[sig-network\] - skip: "\\[Disruptive\\]|\\[Flaky\\]|\ - \\[Conformance\\]|load.balancer|\\[Feature:.+\\]" - - - case_name: sig_network_feature - project_name: functest - criteria: 100 - blocking: false - description: >- - It verifies a couple of features by running a subset of e2e - tests: Ingress, Networking-IPv4, Networking-Performance, - PerformanceDNS, SCTP and NoSNAT. It excludes by default - NetworkPolicy, Networking-IPv6, IPv6DualStackAlphaFeature - and Example. A couple of features may be automatically - skipped like NEG and kubemci. - dependencies: - - DEPLOY_SCENARIO: 'k8-*' - run: - name: e2e_testing - args: - focus: "\\[Feature:Ingress\\]|\ - \\[Feature:Networking-IPv4\\]|\ - \\[Feature:Networking-Performance\\]|\ - \\[Feature:PerformanceDNS\\]|\ - \\[Feature:SCTP\\]|\\[Feature:NoSNAT\\]" - skip: "\\[Disruptive\\]|\\[Flaky\\]|\ - \\[Feature:NetworkPolicy\\]|\ - \\[Feature:Networking-IPv6\\]|\ - \\[Feature:IPv6DualStackAlphaFeature\\]|\ - \\[Feature:Example\\]" - - - case_name: sig_storage - project_name: functest - criteria: 100 - blocking: false - description: >- - It runs the remaining sig-storage testing excluding the - next labels: Conformance (part of k8s_conformance), all - features, all drivers and Slow. - dependencies: - - DEPLOY_SCENARIO: 'k8-*' - run: - name: e2e_testing - args: - focus: \[sig-storage\] - skip: "\\[Disruptive\\]|\\[Flaky\\]|\ - \\[Conformance\\]|\\[Feature:.+\\]|\\[Driver:.+\\]|\ - \\[Slow\\]" + - name: smoke + description: >- + A set of conformance tests integrated from kubernetes project. + testcases: + - case_name: xrally_kubernetes + project_name: functest + criteria: 100 + blocking: false + description: >- + All in one tasks for checking basic functionality of + Kubernetes cluster. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: xrally_kubernetes + - case_name: k8s_conformance + project_name: functest + criteria: 100 + blocking: false + description: >- + Conformance Tests are expect to pass on any Kubernetes + cluster, this tests are necessary demonstrate conformance + of the Kubernetes deployment. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + nodes: 10 + ginkgo: + focus: \[Conformance\] + skip: \[Serial\]|\[Disruptive\]|NoExecuteTaintManager + - case_name: k8s_conformance_serial + project_name: functest + criteria: 100 + blocking: false + description: >- + Conformance Tests are expect to pass on any Kubernetes + cluster, this tests are necessary demonstrate conformance + of the Kubernetes deployment. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + ginkgo: + focus: \[Serial\].*\[Conformance\] + skip: \[Disruptive\]|NoExecuteTaintManager + - case_name: sig_api_machinery + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-api-machinery testing excluding + the label Conformance (part of k8s_conformance) and + StorageVersionAPI. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + nodes: 10 + ginkgo: + focus: \[sig-api-machinery\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Serial\\]|\\[Conformance\\]|\ + \\[Feature:StorageVersionAPI\\]" + - case_name: sig_api_machinery_serial + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-api-machinery testing excluding + the label Conformance (part of k8s_conformance) and + StorageVersionAPI. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + ginkgo: + focus: \[sig-api-machinery\].*\[Serial\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Conformance\\]|\ + \\[Feature:StorageVersionAPI\\]" + - case_name: sig_apps + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-apps testing excluding the + following labels: Conformance (part of k8s_conformance) and + both StatefulUpgrade and StatefulSet features. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + nodes: 10 + ginkgo: + focus: \[sig-apps] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Serial\\]|\\[Conformance\\]|\ + \\[Feature:IndexedJob\\]|\ + \\[Feature:SuspendJob\\]|\ + \\[Feature:StatefulUpgrade\\]|\ + \\[Feature:StatefulSet\\]" + - case_name: sig_apps_serial + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-apps testing excluding the + following labels: Conformance (part of k8s_conformance) and + both StatefulUpgrade and StatefulSet features. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + ginkgo: + focus: \[sig-apps].*\[Serial\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Conformance\\]|\ + \\[Feature:StatefulUpgrade\\]|\ + \\[Feature:DaemonSetUpdateSurge\\]|\ + \\[Feature:StatefulSet\\]" + - case_name: sig_auth + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-auth testing excluding the + following labels: Conformance (part of k8s_conformance) and + both PodSecurityPolicy and BoundServiceAccountTokenVolume + features. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + nodes: 10 + ginkgo: + focus: \[sig-auth\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Serial\\]|\\[Conformance\\]|\ + \\[Feature:PodSecurityPolicy\\]|\ + \\[Feature:BoundServiceAccountTokenVolume\\]" + - case_name: sig_cli + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-cli testing excluding Conformance + (part of k8s_conformance). + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + nodes: 10 + ginkgo: + focus: \[sig-cli] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Serial\\]|\\[Conformance\\]" + - case_name: sig_cli_serial + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-cli testing excluding Conformance + (part of k8s_conformance). + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + ginkgo: + focus: \[sig-cli].*\[Serial\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Conformance\\]" + - case_name: sig_cluster_lifecycle + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-cluster-lifecycle testing + excluding Conformance (part of k8s_conformance). + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + nodes: 10 + ginkgo: + focus: \[sig-cluster-lifecycle\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Serial\\]|\\[Conformance\\]" + - case_name: sig_instrumentation + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-instrumentation testing excluding + Conformance (part of k8s_conformance) + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + nodes: 10 + ginkgo: + focus: \[sig-instrumentation\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Serial\\]|\\[Conformance\\]" + - case_name: sig_network + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-network testing excluding the + following labels: Conformance (part of k8s_conformance), + load.balancer, NetworkPolicy, Networking-IPv6, + IPv6DualStackAlphaFeature, SCTP and Example. A couple of features may + be automatically skipped like NEG and kubemci. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + nodes: 10 + ginkgo: + focus: \[sig-network\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Serial\\]|\\[Conformance\\]|load.balancer|\ + LoadBalancer|\\[Feature:NetworkPolicy\\]|\ + \\[Feature:Networking-IPv6\\]|\ + \\[Feature:IPv6DualStack\\]|\ + \\[Feature:SCTP\\]|\\[Feature:Example\\]|\ + Network.should.set.TCP.CLOSE_WAIT.timeout" + - case_name: sig_network_serial + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-network testing excluding the + following labels: Conformance (part of k8s_conformance), + load.balancer, NetworkPolicy, Networking-IPv6, + IPv6DualStackAlphaFeature, SCTP and Example. A couple of features may + be automatically skipped like NEG and kubemci. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + ginkgo: + focus: \[sig-network\].*\[Serial\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Conformance\\]|load.balancer|\ + LoadBalancer|\\[Feature:NetworkPolicy\\]|\ + \\[Feature:SCTP\\]|\\[Feature:Example\\]" + - case_name: sig_node + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-node testing excluding the + following labels: Conformance (part of k8s_conformance), + RuntimeClass, all RuntimeHandler, PodGarbageCollector, + RegularResourceUsageTrackingfeatures and + ExperimentalResourceUsageTracking features. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + nodes: 10 + ginkgo: + focus: \[sig-node] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Serial\\]|\\[Conformance\\]|\ + \\[NodeFeature:RuntimeHandler\\]|\ + \\[Feature:PodGarbageCollector\\]|\ + \\[Feature:RegularResourceUsageTracking\\]|\ + \\[Feature:GPUUpgrade\\]|\ + \\[Feature:ExperimentalResourceUsageTracking\\]|\ + RuntimeClass" + - case_name: sig_scheduling_serial + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-network testing excluding the + next labels: Conformance (part of k8s_conformance) and the + GPUDevicePlugin feature. + dependencies: + - DEPLOY_SCENARIO: 'k8-*' + run: + name: e2e_testing + args: + ginkgo: + focus: \[sig-scheduling\].*\[Serial\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Conformance\\]|\ + \\[Feature:GPUDevicePlugin\\]" + - case_name: sig_storage + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-storage testing excluding the + following labels: Conformance (part of k8s_conformance), + all features, all drivers, PersistentVolumes.NFS and Slow. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + nodes: 10 + ginkgo: + focus: \[sig-storage\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Serial\\]|\\[Conformance\\]|\ + \\[Feature:.+\\]|\\[Driver:.+\\]|\ + PersistentVolumes.NFS|\\[Slow\\]" + - case_name: sig_storage_serial + project_name: functest + criteria: 100 + blocking: false + description: >- + It runs the remaining sig-storage testing excluding the + following labels: Conformance (part of k8s_conformance), + all features, all drivers, PersistentVolumes.NFS and Slow. + dependencies: + - DEPLOY_SCENARIO: "k8-*" + run: + name: e2e_testing + args: + ginkgo: + focus: \[sig-storage\].*\[Serial\] + skip: "\\[Disruptive\\]|\\[Flaky\\]|\\[alpha\\]|\ + \\[Conformance\\]|\ + \\[Feature:.+\\]|\\[Driver:.+\\]|\ + PersistentVolumes.NFS|\\[Slow\\]"