2 ### Feedback on Euphrates evolutions
7 ### Main Framework evolutions
8 * Functest Framework refactoring
9 * Docker slicing with Alpine
10 * Building Functest dockers
11 * Requirement management...1st time...in OPNFV
15 ## Functest Framework refactoring
19 * Finalized in euphrates
20 * Goal: ease the integration of feature/vnf tests
22 ### See complete presentation on the framework
23 http://testresults.opnfv.org/functest/framework/
29 #### Short story of Functest Docker
32 ### Many rivers to cross
34 |-------------|-----------------------------------------------------|
35 | Arno | No docker, all tests initiated from the jumphost |
36 | Brahmaputra | Introduction of Docker |
37 | Colorado | Growth of Historical Docker |
38 | Danube | Growth of Historical Docker |
39 | Euphrates | Introduction of Alpine |
42 ### Focus on the Historical Docker
43 * Based on ubuntu 14.04
45 | Version | Compressed Size |
46 |--------------|-----------------|
47 | Brahmaputra | 354 MB |
48 | Colorado.1.0 | 627 MB |
49 | Danube.1.0 | 713 MB |
50 | cvp.0.2.0 | 710 MB |
52 * src: https://hub.docker.com/r/opnfv/functest/tags/
55 ### Introduction to Alpine
56 * Alpine Linux is a GNU/Linux distribution based on musl and Busybox
57 * Hardened kernel, compiles all user space binaries as position-independent executables with stack-smashing protection.
58 * Docker Alpine leverage Alpine Linux: https://docs.docker.com/samples/library/alpine/
62 * Lighten docker / save bandwidth / save CI resources
64 * Simplify Requirement management
65 * Isolate "exotic" test cases
68 ### From 1 to many dockers
69 | Docker | Size | Role |
70 |-------------|-------|-------------------------------------------------|
71 | core | 122MB | baseline (tools, env) |
72 | healthcheck | 122MB | OS connectivity, API, DHCP testing |
73 | smoke | 131MB | vpings, Tempest/Refstack, Rally, odl, Snaps |
74 | features | 214MB | doctor, domino, sdnvpn, sfc, promise, barometer |
77 ### From 1 to many dockers
78 | Docker | Size | Role |
79 |-------------|-------|-------------------------------------------------|
80 | components | 122MB | Full Tempest, Full Rally |
81 | vnf | 155MB | vIMS, vRouter |
82 | parser | 127MB | parser (feature needs pike clients) |
86 src: https://wiki.opnfv.org/display/functest/Run+Alpine+Functest+containers
87 * env: OPNFV env variables
88 * openstack.creds: OpenStack rc file
91 sudo docker run --env-file env \
92 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
93 -v $(pwd)/images:/home/opnfv/functest/images \
94 opnfv/functest-smoke:euphrates
101 INSTALLER_TYPE=Compass
103 EXTERNAL_NETWORK=ext-net
104 DEPLOY_SCENARIO=os-nosdn-nofeature-ha
111 export OS_AUTH_URL=XXX
112 export OS_USER_DOMAIN_NAME=XXX
113 export OS_PROJECT_DOMAIN_NAME=XXX
114 export OS_USERNAME=XXX
115 export OS_PROJECT_NAME=XXX
116 export OS_PASSWORD=XXX
117 export OS_IDENTITY_API_VERSION=3
121 #### Retrieve all the images
123 mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | sh -s -- images && ls -1 images/*
124 images/CentOS-7-aarch64-GenericCloud.qcow2
125 images/CentOS-7-aarch64-GenericCloud.qcow2.xz
126 images/CentOS-7-x86_64-GenericCloud.qcow2
127 images/cirros-0.3.5-x86_64-disk.img
128 images/cirros-0.3.5-x86_64-lxc.tar.gz
129 images/cirros-d161201-aarch64-disk.img
130 images/cirros-d161201-aarch64-initramfs
131 images/cirros-d161201-aarch64-kernel
132 images/cloudify-manager-premium-4.0.1.qcow2
134 images/trusty-server-cloudimg-amd64-disk1.img
135 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
136 images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
137 images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
138 images/vyos-1.1.7.img
140 * could be improved (retrieve only needed images Tier/config)
145 +----------------------+--------- + ------+----------+--------+
146 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
147 +----------------------+--------- + ------+----------+--------+
148 | vping_ssh | functest | smoke | 01:19 | PASS |
149 | vping_userdata | functest | smoke | 01:56 | PASS |
150 | tempest_smoke_serial | functest | smoke | 26:30 | PASS |
151 | rally_sanity | functest | smoke | 19:42 | PASS |
152 | refstack_defcore | functest | smoke | 22:00 | PASS |
153 | snaps_smoke | functest | smoke | 41:14 | PASS |
154 | odl | functest | smoke | 00:16 | PASS |
155 | odl_netvirt | functest | smoke | 00:00 | SKIP |
156 | fds | functest | smoke | 00:00 | SKIP |
157 +----------------------+--------- + ------+----------+--------+
162 #### Easy way to customize
163 * Change list of testcase -v your_config.yaml:/usr/lib/python2.7/site-packages/functest/ci/testcases.yaml
164 * Change logger param -v your_logger.ini:/usr/lib/python2.7/site-packages/functest/ci/logging.ini
167 #### Docker slicing technical presentation
168 http://testresults.opnfv.org/functest/dockerslicing/
172 ## Building Functest dockers
176 * the unique Functest docker was "produced" from Releng
179 #### For Euphrates, build was done on Docker hub
180 * Releng adaptations not ready in time
181 * more capabilities using Docker Hub
182 * Use of private ollivier then official opnfv Docker Hub
183 * Euphrates 5.1: come back to releng with at least same level of features than Docker Hub/Travis CI
187 ## Requirement management
188 Until now, requirements were managed as follow:
194 ### Requirement management
196 * Danube, Colorado, .. dockers may run but no garantee on mid/long term because
197 * No control of upstream or internal project dependencies
198 * the dependencies of the last feature project overwrite the previous ones
201 ### Requirement management
202 * Nothing done in any OPNFV project
203 * No dependency list, no reco (e.g. support Python 3)
204 * Only high level wiki declarative intentions for OpenStack (e.g. Euphrates => Ocata)
205 * Nothing equivalent to OpenStack https://releases.openstack.org/ocata/
208 ### Functest evolutions
209 * creation of requirements.txt, upper-constraints.text
210 https://git.opnfv.org/functest/tree/requirements.txt
211 https://git.opnfv.org/functest/tree/upper-constraints.txt
213 pbr>=1.8 # Apache-2.0
215 GitPython>=1.0.1 # BSD License (3 clause)
216 keystoneauth1>=2.18.0 # Apache-2.0
217 python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0
218 python-glanceclient>=2.5.0 # Apache-2.0
219 python-heatclient>=1.6.1 # Apache-2.0
222 * sync with ocata done manually by Cédric
225 ### Functest evolutions
226 * Code of the feature projects under project responsibility
227 * code moved to their own repo
228 * no more mix between Functest and Feature project code
229 * All python OPNFV projects imported properly as python modules
230 * all dependencies a priori under control...
238 * A Rest API has been introduced in Euphrates (Linda)
239 * Goal: allow third party to invoke Functest resources
240 * pseudo micro services approach
241 * avoid overlap (e.g. deployement of vIMS from other project to run perfo tests)
242 src: https://wiki.opnfv.org/display/functest/Functest+REST+API
245 ### Functest rest API
246 | resource | Methods | Description |
247 |-------------|----------|-------------------------------------------------|
248 | environment | GET,POST | show, prepare environment |
249 | openstack | GET,POST | show, check, clean, update credentials |
250 | testcases | GET,POST | list, show, run |
251 | tiers | GET,POST | list, show |
252 | tasks | GET | Get the result of the task id |
257 curl -X POST --header "Content-Type: application/json" \
258 --data '{"action":"run_test_case", "args": {"opts": {}, "testcase": "vping_ssh"}}' \
259 http://127.0.0.1:5000/api/v1/functest/testcases/action
261 "task_id": "1a9f3c5d-ce0b-4354-862e-dd08b26fc484",
262 "testcase":"vping_ssh"
269 * Framework heavily refactored for Euphrates
270 * Better code quality and rules (coverage, pylint, pep8, ..)
271 * Much more "trustable", light and evolutive
275 * Still lots of work for Fraser
276 * Adaptation to use it for XCI gating (xTesting)
277 * Generic dockerfile Functest customized docker on demand
278 * split framework and testcases in order to be able to reuse Functest for k8 or even beyond OPNFV
280 * better management of the images