README.md

   1 # Functest
   2
   3 Network virtualization has dramatically modified our architectures which asks
   4 for more automation and powerful testing tools like Functest, a collection of
   5 state-of-the-art virtual infrastructure test suites, including automatic VNF
   6 testing (cf.
   7 [[1]](https://www.linuxfoundation.org/press-release/2019/05/opnfv-hunter-delivers-test-tools-ci-cd-framework-to-enable-common-nfvi-for-verifying-vnfs/)).
   8
   9 In context of OPNFV, Functest verifies any kind of OpenStack and Kubernetes
  10 deployments including production environments. It conforms to upstream rules
  11 and integrates smoothly lots of the test cases available in the opensource
  12 market. It includes about 3000+ functional tests and 3 hours upstream API and
  13 dataplane benchmarks. It’s completed by Virtual Network Function deployments
  14 and testing (vIMS, vRouter and vEPC) to ensure that the platforms meet Network
  15 Functions Virtualization requirements. Raspberry PI is also supported to verify
  16 datacenters as the lowest cost (50 euros hardware and software included).
  17
  18 | Functest releases | Kubernetes releases       |
  19 |-------------------|---------------------------|
  20 | **Hunter**        | **v1.13**                 |
  21 | Iruya             | v1.15                     |
  22 | Jerma             | v1.17                     |
  23 | Kali              | v1.19                     |
  24 | Leguer            | v1.20                     |
  25 | Master            | v1.21.0-alpha.3 (rolling) |
  26
  27 ## Prepare your environment
  28
  29 cat env
  30 ```
  31 DEPLOY_SCENARIO=k8s-XXX
  32 ```
  33
  34 ## Run healthcheck suite
  35
  36 ```bash
  37 sudo docker run --env-file env \
  38     -v $(pwd)/config:/root/.kube/config \
  39     opnfv/functest-kubernetes-healthcheck:hunter
  40 ```
  41
  42 ```
  43 +-------------------+------------------+---------------------+------------------+----------------+
  44 |     TEST CASE     |     PROJECT      |         TIER        |     DURATION     |     RESULT     |
  45 +-------------------+------------------+---------------------+------------------+----------------+
  46 |     k8s_quick     |     functest     |     healthcheck     |      00:20       |      PASS      |
  47 |     k8s_smoke     |     functest     |     healthcheck     |      00:45       |      PASS      |
  48 +-------------------+------------------+---------------------+------------------+----------------+
  49 ```
  50
  51 ## Run smoke suite
  52
  53 ```bash
  54 sudo docker run --env-file env \
  55     -v $(pwd)/config:/root/.kube/config \
  56     opnfv/functest-kubernetes-smoke:hunter
  57 ```
  58
  59 ```
  60 +---------------------------+------------------+---------------+------------------+----------------+
  61 |         TEST CASE         |     PROJECT      |      TIER     |     DURATION     |     RESULT     |
  62 +---------------------------+------------------+---------------+------------------+----------------+
  63 |      k8s_conformance      |     functest     |     smoke     |      100:50      |      PASS      |
  64 |     xrally_kubernetes     |     functest     |     smoke     |      13:19       |      PASS      |
  65 +---------------------------+------------------+---------------+------------------+----------------+
  66 ```
  67
  68 ## Run security suite
  69
  70 ```bash
  71 sudo docker run --env-file env \
  72     -v $(pwd)/config:/root/.kube/config \
  73     opnfv/functest-kubernetes-security:hunter
  74 ```
  75
  76 ```
  77 +---------------------------+------------------+------------------+------------------+----------------+
  78 |         TEST CASE         |     PROJECT      |       TIER       |     DURATION     |     RESULT     |
  79 +---------------------------+------------------+------------------+------------------+----------------+
  80 |        kube_hunter        |     functest     |     security     |      00:19       |      PASS      |
  81 |     kube_bench_master     |     functest     |     security     |      00:01       |      PASS      |
  82 |      kube_bench_node      |     functest     |     security     |      00:01       |      PASS      |
  83 +---------------------------+------------------+------------------+------------------+----------------+
  84 ```
  85
  86 ## Run benchmarking suite
  87
  88 ```bash
  89 sudo docker run --env-file env \
  90     -v $(pwd)/config:/root/.kube/config \
  91     opnfv/functest-kubernetes-benchmarking:hunter
  92 ```
  93
  94 ```
  95 +--------------------------------+------------------+----------------------+------------------+----------------+
  96 |           TEST CASE            |     PROJECT      |         TIER         |     DURATION     |     RESULT     |
  97 +--------------------------------+------------------+----------------------+------------------+----------------+
  98 |     xrally_kubernetes_full     |     functest     |     benchmarking     |      37:48       |      PASS      |
  99 +--------------------------------+------------------+----------------------+------------------+----------------+
 100 ```
 101
 102 ## Run cnf suite
 103
 104 ```bash
 105 sudo docker run --env-file env \
 106     -v $(pwd)/config:/root/.kube/config \
 107     opnfv/functest-kubernetes-cnf:hunter
 108 ```
 109
 110 ```
 111 +-------------------------+------------------+--------------+------------------+----------------+
 112 |        TEST CASE        |     PROJECT      |     TIER     |     DURATION     |     RESULT     |
 113 +-------------------------+------------------+--------------+------------------+----------------+
 114 |         k8s_vims        |     functest     |     cnf      |      09:09       |      PASS      |
 115 |        helm_vims        |     functest     |     cnf      |      07:55       |      PASS      |
 116 |     cnf_conformance     |     functest     |     cnf      |      02:18       |      PASS      |
 117 +-------------------------+------------------+--------------+------------------+----------------+
 118 ```