README.md

   1 # Xtesting in a nutshell
   2
   3 Xtesting is a simple framework to assemble sparse test cases and to accelerate
   4 the adoption of continuous integration best practices. By managing all
   5 the interactions with the components (test scheduler, test results database,
   6 artifact repository), it allows the developer to work only on the test suites
   7 without diving into CI/CD.
   8
   9 It asks for a few low constraints
  10 [quickly achievable](https://www.sdxcentral.com/articles/news/opnfvs-6th-release-brings-testing-capabilities-that-orange-is-already-using/2018/05/)
  11 to verify multiple components in the same CI/CD toolchain. Even more, it brings
  12 the capability to run third-party test cases in our CI toolchains and then to
  13 also rate network functions by the coverage.
  14
  15 Please see
  16 [the Katacoda scenarios](https://www.katacoda.com/ollivier/courses/xtestingci)
  17 to try Xtesting. You will love them!
  18
  19 ## [Write your own Xtesting driver](https://www.katacoda.com/ollivier/courses/xtestingci/firstdriver)
  20
  21 ### dump all the following files in an empty dir
  22
  23 weather.py
  24
  25 ```python
  26 #!/usr/bin/env python
  27
  28 # pylint: disable=missing-docstring
  29
  30 import json
  31 import os
  32 import sys
  33 import time
  34
  35 import requests
  36
  37 from xtesting.core import testcase
  38
  39
  40 class Weather(testcase.TestCase):
  41
  42     url = "https://samples.openweathermap.org/data/2.5/weather"
  43     city_name = "London,uk"
  44     app_key = "439d4b804bc8187953eb36d2a8c26a02"
  45
  46     def run(self, **kwargs):
  47         try:
  48             self.start_time = time.time()
  49             req = requests.get("{}?q={}&&appid={}".format(
  50                 self.url, self.city_name, self.app_key))
  51             req.raise_for_status()
  52             data = req.json()
  53             os.makedirs(self.res_dir, exist_ok=True)
  54             with open('{}/dump.txt'.format(self.res_dir), 'w+') as report:
  55                 json.dump(data, report, indent=4, sort_keys=True)
  56             for key in kwargs:
  57                 if data["main"][key] > kwargs[key]:
  58                     self.result = self.result + 100/len(kwargs)
  59             self.stop_time = time.time()
  60         except Exception:  # pylint: disable=broad-except
  61             print("Unexpected error:", sys.exc_info()[0])
  62             self.result = 0
  63             self.stop_time = time.time()
  64 ```
  65
  66 setup.py
  67
  68 ```python
  69 #!/usr/bin/env python
  70
  71 # pylint: disable=missing-docstring
  72
  73 import setuptools
  74
  75 setuptools.setup(
  76     setup_requires=['pbr>=2.0.0'],
  77     pbr=True)
  78 ```
  79
  80 setup.cfg
  81
  82 ```
  83 [metadata]
  84 name = weather
  85 version = 1
  86
  87 [files]
  88 packages = .
  89
  90 [entry_points]
  91 xtesting.testcase =
  92     weather = weather:Weather
  93 ```
  94
  95 requirements.txt
  96
  97 ```
  98 xtesting
  99 requests!=2.20.0,!=2.24.0 # Apache-2.0
 100 ```
 101
 102 testcases.yaml
 103
 104 ```yaml
 105 ---
 106 tiers:
 107     -
 108         name: simple
 109         order: 0
 110         description: ''
 111         testcases:
 112             -
 113                 case_name: humidity
 114                 project_name: weather
 115                 criteria: 100
 116                 blocking: true
 117                 clean_flag: false
 118                 description: ''
 119                 run:
 120                     name: weather
 121                     args:
 122                         humidity: 80
 123             -
 124                 case_name: pressure
 125                 project_name: weather
 126                 criteria: 100
 127                 blocking: true
 128                 clean_flag: false
 129                 description: ''
 130                 run:
 131                     name: weather
 132                     args:
 133                         pressure: 1000
 134             -
 135                 case_name: temp
 136                 project_name: weather
 137                 criteria: 100
 138                 blocking: true
 139                 clean_flag: false
 140                 description: ''
 141                 run:
 142                     name: weather
 143                     args:
 144                         temp: 280
 145     -
 146         name: combined
 147         order: 1
 148         description: ''
 149         testcases:
 150             -
 151                 case_name: half
 152                 project_name: weather
 153                 criteria: 50
 154                 blocking: true
 155                 clean_flag: false
 156                 description: ''
 157                 run:
 158                     name: weather
 159                     args:
 160                         humidity: 90
 161                         pressure: 1000
 162                         temp: 280
 163 ```
 164
 165 Dockerfile
 166
 167 ```
 168 FROM alpine:3.12
 169
 170 ADD . /src/
 171 RUN apk --no-cache add --update python3 py3-pip py3-wheel git && \
 172     git init /src && pip3 install /src
 173 COPY testcases.yaml /usr/lib/python3.8/site-packages/xtesting/ci/testcases.yaml
 174 CMD ["run_tests", "-t", "all"]
 175 ```
 176
 177 ### make world
 178
 179 Deploy your own Xtesting toolchain
 180
 181 ```bash
 182 virtualenv xtesting
 183 . xtesting/bin/activate
 184 pip install ansible
 185 ansible-galaxy install collivier.xtesting
 186 ansible-galaxy collection install ansible.posix community.general community.grafana community.kubernetes
 187 ansible-playbook site.yml
 188 deactivate
 189 rm -r xtesting
 190 ```
 191
 192 Build your container
 193
 194 ```bash
 195 sudo docker build -t 127.0.0.1:5000/weather .
 196 ```
 197
 198 Publish your container on your local repository
 199
 200 ```bash
 201 sudo docker push 127.0.0.1:5000/weather
 202 ```
 203
 204 ### That's all folks!