README.md

   1 # Xtesting in a nutshell
   2
   3 Xtesting is a simple framework to assemble sparse test cases and to accelerate
   4 the adoption of continuous integration best practices. By managing all
   5 the interactions with the components (test scheduler, test results database,
   6 artifact repository), it allows the developer to work only on the test suites
   7 without diving into CI/CD.
   8
   9 It asks for a few low constraints
  10 [quickly achievable](https://www.sdxcentral.com/articles/news/opnfvs-6th-release-brings-testing-capabilities-that-orange-is-already-using/2018/05/)
  11 to verify multiple components in the same CI/CD toolchain. Even more, it brings
  12 the capability to run third-party test cases in our CI toolchains and then to
  13 also rate network functions by the coverage.
  14
  15 Please see
  16 [the Katacoda scenarios](https://www.katacoda.com/ollivier/courses/xtestingci)
  17 to try Xtesting. You will love them!
  18
  19 ## [Write your own Xtesting driver](https://www.katacoda.com/ollivier/courses/xtestingci/firstdriver)
  20
  21 Note that [running MongoDB 5.0+ requires _avx_ CPU instruction set](https://www.mongodb.com/docs/manual/administration/production-notes/#x86_64)
  22 that is usually shipped in all recent _x86_ hardware processors.
  23 Though, it may not be available in your virtualized environments.
  24 For example, Qemu _avx_ support is only available [since version 7.2](https://github.com/nodkz/mongodb-memory-server/issues/710#issuecomment-1297462935)
  25 and must be explicitly enabled (e.g. with the argument _-cpu max_).
  26
  27 You can check the presence of the _avx_ CPU instruction set on your processor
  28 with the following command.
  29 ```bash
  30 grep '^processor\|^flags.* avx' /proc/cpuinfo
  31 ```
  32
  33 ### dump all the following files in an empty dir
  34
  35 weather.py
  36
  37 ```python
  38 #!/usr/bin/env python
  39
  40 # pylint: disable=missing-docstring
  41
  42 import json
  43 import os
  44 import sys
  45 import time
  46
  47 import requests
  48
  49 from xtesting.core import testcase
  50
  51
  52 class Weather(testcase.TestCase):
  53
  54     url = "https://samples.openweathermap.org/data/2.5/weather"
  55     city_name = "London,uk"
  56     app_key = "439d4b804bc8187953eb36d2a8c26a02"
  57
  58     def run(self, **kwargs):
  59         try:
  60             self.start_time = time.time()
  61             req = requests.get("{}?q={}&&appid={}".format(
  62                 self.url, self.city_name, self.app_key))
  63             req.raise_for_status()
  64             data = req.json()
  65             os.makedirs(self.res_dir, exist_ok=True)
  66             with open('{}/dump.txt'.format(self.res_dir), 'w+') as report:
  67                 json.dump(data, report, indent=4, sort_keys=True)
  68             for key in kwargs:
  69                 if data["main"][key] > kwargs[key]:
  70                     self.result = self.result + 100/len(kwargs)
  71             self.stop_time = time.time()
  72         except Exception:  # pylint: disable=broad-except
  73             print("Unexpected error:", sys.exc_info()[0])
  74             self.result = 0
  75             self.stop_time = time.time()
  76 ```
  77
  78 setup.py
  79
  80 ```python
  81 #!/usr/bin/env python
  82
  83 # pylint: disable=missing-docstring
  84
  85 import setuptools
  86
  87 setuptools.setup(
  88     setup_requires=['pbr>=2.0.0'],
  89     pbr=True)
  90 ```
  91
  92 setup.cfg
  93
  94 ```
  95 [metadata]
  96 name = weather
  97 version = 1
  98
  99 [files]
 100 packages = .
 101
 102 [entry_points]
 103 xtesting.testcase =
 104     weather = weather:Weather
 105 ```
 106
 107 requirements.txt
 108
 109 ```
 110 xtesting
 111 requests!=2.20.0,!=2.24.0 # Apache-2.0
 112 ```
 113
 114 testcases.yaml
 115
 116 ```yaml
 117 ---
 118 tiers:
 119     -
 120         name: simple
 121         order: 0
 122         description: ''
 123         testcases:
 124             -
 125                 case_name: humidity
 126                 project_name: weather
 127                 criteria: 100
 128                 blocking: true
 129                 clean_flag: false
 130                 description: ''
 131                 run:
 132                     name: weather
 133                     args:
 134                         humidity: 80
 135             -
 136                 case_name: pressure
 137                 project_name: weather
 138                 criteria: 100
 139                 blocking: true
 140                 clean_flag: false
 141                 description: ''
 142                 run:
 143                     name: weather
 144                     args:
 145                         pressure: 1000
 146             -
 147                 case_name: temp
 148                 project_name: weather
 149                 criteria: 100
 150                 blocking: true
 151                 clean_flag: false
 152                 description: ''
 153                 run:
 154                     name: weather
 155                     args:
 156                         temp: 280
 157     -
 158         name: combined
 159         order: 1
 160         description: ''
 161         testcases:
 162             -
 163                 case_name: half
 164                 project_name: weather
 165                 criteria: 50
 166                 blocking: true
 167                 clean_flag: false
 168                 description: ''
 169                 run:
 170                     name: weather
 171                     args:
 172                         humidity: 90
 173                         pressure: 1000
 174                         temp: 280
 175 ```
 176
 177 Dockerfile
 178
 179 ```
 180 FROM alpine:3.12
 181
 182 ADD . /src/
 183 RUN apk --no-cache add --update python3 py3-pip py3-wheel git && \
 184     git init /src && pip3 install /src
 185 COPY testcases.yaml /usr/lib/python3.8/site-packages/xtesting/ci/testcases.yaml
 186 CMD ["run_tests", "-t", "all"]
 187 ```
 188
 189 ### make world
 190
 191 Deploy your own Xtesting toolchain
 192
 193 ```bash
 194 virtualenv xtesting
 195 . xtesting/bin/activate
 196 pip install ansible
 197 ansible-galaxy install collivier.xtesting
 198 ansible-galaxy collection install ansible.posix community.general community.grafana community.kubernetes
 199 ansible-playbook site.yml
 200 deactivate
 201 rm -r xtesting
 202 ```
 203
 204 Build your container
 205
 206 ```bash
 207 sudo docker build -t 127.0.0.1:5000/weather .
 208 ```
 209
 210 Publish your container on your local repository
 211
 212 ```bash
 213 sudo docker push 127.0.0.1:5000/weather
 214 ```
 215
 216 ### That's all folks!