--- /dev/null
+# Functest
+### Feedback on Euphrates evolutions
+
+2017/10/09
+
+
+### Main Framework evolutions
+ * Functest Framework refactoring
+ * Docker slicing with Alpine
+ * Building Functest dockers
+ * Requirement management...1st time...in OPNFV
+ * Functest API
+
+
+## Functest Framework refactoring
+
+### The framework
+ * Initiated in Danube
+ * Finalized in euphrates
+ * Goal: ease the integration of feature/vnf tests
+
+### See complete presentation on the framework
+http://testresults.opnfv.org/functest/framework/
+
+
+## Docker slicing
+
+
+#### Short story of Functest Docker
+
+
+### Many rivers to cross
+| Version | Comment |
+|-------------|-----------------------------------------------------|
+| Arno | No docker, all tests initiated from the jumphost |
+| Brahmaputra | Introduction of Docker |
+| Colorado | Growth of Historical Docker |
+| Danube | Growth of Historical Docker |
+| Euphrates | Introduction of Alpine |
+
+
+### Focus on the Historical Docker
+ * Based on ubuntu 14.04
+
+| Version | Compressed Size |
+|--------------|-----------------|
+| Brahmaputra | 354 MB |
+| Colorado.1.0 | 627 MB |
+| Danube.1.0 | 713 MB |
+| cvp.0.2.0 | 710 MB |
+
+* src: https://hub.docker.com/r/opnfv/functest/tags/
+
+
+### Introduction to Alpine
+* Alpine Linux is a GNU/Linux distribution based on musl and Busybox
+* Hardened kernel, compiles all user space binaries as position-independent executables with stack-smashing protection.
+* Docker Alpine leverage Alpine Linux: https://docs.docker.com/samples/library/alpine/
+
+
+### Our goals
+* Lighten docker / save bandwidth / save CI resources
+* Slice testing
+* Simplify Requirement management
+* Isolate "exotic" test cases
+
+
+### From 1 to many dockers
+| Docker | Size | Role |
+|-------------|-------|-------------------------------------------------|
+| core | 122MB | baseline (tools, env) |
+| healthcheck | 122MB | OS connectivity, API, DHCP testing |
+| smoke | 131MB | vpings, Tempest/Refstack, Rally, odl, Snaps |
+| features | 214MB | doctor, domino, sdnvpn, sfc, promise, barometer |
+
+
+### From 1 to many dockers
+| Docker | Size | Role |
+|-------------|-------|-------------------------------------------------|
+| components | 122MB | Full Tempest, Full Rally |
+| vnf | 155MB | vIMS, vRouter |
+| parser | 127MB | parser (feature needs pike clients) |
+
+
+### Easy way to run
+src: https://wiki.opnfv.org/display/functest/Run+Alpine+Functest+containers
+ * env: OPNFV env variables
+ * openstack.creds: OpenStack rc file
+
+```
+sudo docker run --env-file env \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
+ -v $(pwd)/images:/home/opnfv/functest/images \
+ opnfv/functest-smoke:euphrates
+```
+
+
+#### Env
+```
+cat env
+INSTALLER_TYPE=Compass
+INSTALLER_IP=XXX
+EXTERNAL_NETWORK=ext-net
+DEPLOY_SCENARIO=os-nosdn-nofeature-ha
+```
+
+
+#### openstack.creds
+```
+cat openstack.creds
+export OS_AUTH_URL=XXX
+export OS_USER_DOMAIN_NAME=XXX
+export OS_PROJECT_DOMAIN_NAME=XXX
+export OS_USERNAME=XXX
+export OS_PROJECT_NAME=XXX
+export OS_PASSWORD=XXX
+export OS_IDENTITY_API_VERSION=3
+```
+
+
+#### Retrieve all the images
+```
+mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/*
+images/CentOS-7-aarch64-GenericCloud.qcow2
+images/CentOS-7-aarch64-GenericCloud.qcow2.xz
+images/CentOS-7-x86_64-GenericCloud.qcow2
+images/cirros-0.3.5-x86_64-disk.img
+images/cirros-0.3.5-x86_64-lxc.tar.gz
+images/cirros-d161201-aarch64-disk.img
+images/cirros-d161201-aarch64-initramfs
+images/cirros-d161201-aarch64-kernel
+images/cloudify-manager-premium-4.0.1.qcow2
+images/img
+images/trusty-server-cloudimg-amd64-disk1.img
+images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
+images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
+images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
+images/vyos-1.1.7.img
+```
+* could be improved (retrieve only needed images Tier/config)
+
+
+#### Results
+```
++----------------------+--------- + ------+----------+--------+
+| TEST CASE | PROJECT | TIER | DURATION | RESULT |
++----------------------+--------- + ------+----------+--------+
+| vping_ssh | functest | smoke | 01:19 | PASS |
+| vping_userdata | functest | smoke | 01:56 | PASS |
+| tempest_smoke_serial | functest | smoke | 26:30 | PASS |
+| rally_sanity | functest | smoke | 19:42 | PASS |
+| refstack_defcore | functest | smoke | 22:00 | PASS |
+| snaps_smoke | functest | smoke | 41:14 | PASS |
+| odl | functest | smoke | 00:16 | PASS |
+| odl_netvirt | functest | smoke | 00:00 | SKIP |
+| fds | functest | smoke | 00:00 | SKIP |
++----------------------+--------- + ------+----------+--------+
+
+```
+
+
+#### Easy way to customize
+ * Change list of testcase -v your_config.yaml:/usr/lib/python2.7/site-packages/functest/ci/testcases.yaml
+ * Change logger param -v your_logger.ini:/usr/lib/python2.7/site-packages/functest/ci/logging.ini
+
+
+#### Docker slicing technical presentation
+http://testresults.opnfv.org/functest/dockerslicing/
+
+
+
+## Building Functest dockers
+
+
+#### Until Danube
+ * the unique Functest docker was "produced" from Releng
+
+
+#### For Euphrates, build was done on Docker hub
+ * Releng adaptations not ready in time
+ * more capabilities using Docker Hub
+ * Use of private ollivier then official opnfv Docker Hub
+ * Euphrates 5.1: come back to releng with at least same level of features than Docker Hub/Travis CI
+
+
+
+## Requirement management
+Until now, requirements were managed as follow:
+
+
+# ?
+
+
+### Requirement management
+ * Nothing was done
+ * Danube, Colorado, .. dockers may run but no garantee on mid/long term because
+ * No control of upstream or internal project dependencies
+ * the dependencies of the last feature project overwrite the previous ones
+
+
+### Requirement management
+ * Nothing done in any OPNFV project
+ * No dependency list, no reco (e.g. support Python 3)
+ * Only high level wiki declarative intentions for OpenStack (e.g. Euphrates => Ocata)
+ * Nothing equivalent to OpenStack https://releases.openstack.org/ocata/
+
+
+### Functest evolutions
+ * creation of requirements.txt, upper-constraints.text
+ https://git.opnfv.org/functest/tree/requirements.txt
+ https://git.opnfv.org/functest/tree/upper-constraints.txt
+ ```
+ pbr>=1.8 # Apache-2.0
+ PyYAML>=3.10.0 # MIT
+ GitPython>=1.0.1 # BSD License (3 clause)
+ keystoneauth1>=2.18.0 # Apache-2.0
+ python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0
+ python-glanceclient>=2.5.0 # Apache-2.0
+ python-heatclient>=1.6.1 # Apache-2.0
+ ...
+ ```
+ * sync with ocata done manually by Cédric
+
+
+### Functest evolutions
+ * Code of the feature projects under project responsibility
+ * code moved to their own repo
+ * no more mix between Functest and Feature project code
+ * All python OPNFV projects imported properly as python modules
+ * all dependencies a priori under control...
+
+
+
+## Functest Rest API
+
+
+### Introduction
+* A Rest API has been introduced in Euphrates (Linda)
+* Goal: allow third party to invoke Functest resources
+ * pseudo micro services approach
+ * avoid overlap (e.g. deployement of vIMS from other project to run perfo tests)
+src: https://wiki.opnfv.org/display/functest/Functest+REST+API
+
+
+### Functest rest API
+| resource | Methods | Description |
+|-------------|----------|-------------------------------------------------|
+| environment | GET,POST | show, prepare environment |
+| openstack | GET,POST | show, check, clean, update credentials |
+| testcases | GET,POST | list, show, run |
+| tiers | GET,POST | list, show |
+| tasks | GET | Get the result of the task id |
+
+
+### Examples
+```
+curl -X POST --header "Content-Type: application/json" \
+ --data '{"action":"run_test_case", "args": {"opts": {}, "testcase": "vping_ssh"}}' \
+ http://127.0.0.1:5000/api/v1/functest/testcases/action
+{
+ "task_id": "1a9f3c5d-ce0b-4354-862e-dd08b26fc484",
+ "testcase":"vping_ssh"
+}
+```
+
+
+
+## Conclusions
+ * Framework heavily refactored for Euphrates
+ * Better code quality and rules (coverage, pylint, pep8, ..)
+ * Much more "trustable", light and evolutive
+
+
+## Next steps
+ * Still lots of work for Fraser
+ * Adaptation to use it for XCI gating (xTesting)
+ * Generic dockerfile Functest customized docker on demand
+ * split framework and testcases in order to be able to reuse Functest for k8 or even beyond OPNFV
+ * integrate k8 tests
+ * better management of the images
+ * ...
Code Review / functest.git / blobdiff