From: Cédric Ollivier <cedric.ollivier@orange.com>
Date: Sun, 6 Jan 2019 10:28:52 +0000 (+0100)
Subject: Publish "Functest on steroids" (Plugfest-Gambia)
X-Git-Tag: opnfv-8.1.0~94^2
X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=commitdiff_plain;h=refs%2Fchanges%2F39%2F66239%2F2;p=functest.git

Publish "Functest on steroids" (Plugfest-Gambia)

It will be completed by demos:
  - deploy your own Functest CI/CD toolchains (Functest bot)
  - deploy your own CI/CD toolchains for your test case
  - run Functest on Raspberry PI

The presentation is avalaible in [1]

[1] http://testresults.opnfv.org/functest/gambia/

Change-Id: If69f11a3f7dde070e3af0f8a5b082b5ec1443bbb
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 6a31d033b8aef014e62cf6f94f2f98358ed99b90)
---

diff --git a/docs/com/pres/gambia/gambia.md b/docs/com/pres/gambia/gambia.md
new file mode 100644
index 000000000..cf5f0f1bf
--- /dev/null
+++ b/docs/com/pres/gambia/gambia.md
@@ -0,0 +1,178 @@
+# Functest on steroids
+
+[Cédric Ollivier](mailto:cedric.ollivier@orange.com)
+
+2019/01/08
+
+
+
+## OPNFV
+
+![OPNFV](https://docs.opnfv.org/en/stable-fraser/_images/OPNFV_testing_working_group1.png)
+<!-- .element: style="border: 0; width: 90%" -->
+
+
+
+## Verify OpenStack and Kubernetes
+
+
+### Functest in a nutshell
+
+- verify any kind of OpenStack and Kubernetes deployments
+- conform with upstream rules (OpenStack gate jobs and Kubernetes conformance
+  tests)
+- ensure that the platforms meet Network Functions Virtualization requirements
+
+
+### Functest suites
+
+- as many upstream functional tests as possible (e.g. Tempest,
+  neutron-tempest-api, Barbican, Patrole...)
+- upstream API and dataplane benchmarking tools (Rally, Vmtp and Shaker)
+- additional VNF deployments and testing (vIMS, vRouter and vEPC)
+
+**Dovetail only runs few Functest functional tests and then we have to verify
+any OPNFV-certified scenarios via Functest anyway**
+
+
+### What's new in Functest?
+
+- new testcases were quickly integrated in Functest: Patrole, Barbican, Shaker,
+  ...
+- all testcases can run in parallel to decrease the overall duration
+- the resources cleaning has been improved
+- **our testcases may be run vs VIM in production**
+- **it includes most of the OpenStack gate jobs**
+
+
+### Support of OS and K8s master
+
+| Functest | OpenStack   | Kubernetes |
+| :------: | :---------: | :--------: |
+| master   | master      | master     |
+| hunter   | rocky       | v1.11.3    |
+| gambia   | queens      | v1.11.3    |
+
+
+### < 50 Euros
+
+![Raspberry PI](raspberrypi.jpg)
+<!-- .element: style="border: 0; width: 70%" -->
+
+
+
+## Reuse of OPNFV
+
+
+### Xtesting in a nutshell
+
+- allow the developer to work only on the test suites without diving into CI/CD
+  integration
+- simplify test integration in a complete LFN-based CI/CD toolchain (e.g.
+  Jenkins, Testing Containers, Test API and dashboard)
+- allow a proper design and verify multiple components in the same CI/CD
+  toolchain (OpenStack, Kubernetes, ONAP, etc.)
+
+**Easy to use and very useful for any CI/CD toochain (unlinked to Infrastrure)**
+
+
+### A user story ONAP
+
+- all tests are run by a specialized Docker container(**<100 MB**) instead of
+the classical ONAP testing virtual machine (**> 1GB**).
+- the container mainly inherits from opnfv/xtesting and is completed by:
+  - Python dependencies
+  - all ONAP Robot Framework files retrieved from the original repositories
+  - testcases.yaml describing the testcases
+
+[Orange-OpenSource/xtesting-onap-robot](https://github.com/Orange-OpenSource/xtesting-onap-robot/)  
+
+
+###  What's new in Xtesting?
+
+- new ansible roles and playbooks have been developed to allow **deploying your
+  full CI/CD toolchains in few minutes** (Jenkins, Minio, TestAPI, MongoDB and
+  Docker registry)
+
+```bash
+virtualenv xtesting
+. xtesting/bin/activate
+pip install ansible docker
+ansible-galaxy install collivier.xtesting
+git clone https://gerrit.opnfv.org/gerrit/functest-xtesting functest-xtesting-src
+ansible-playbook functest-xtesting-src/ansible/site.yml
+deactivate
+```
+
+**They are already reused in Functest and by Orange out of the
+Infrastrure domain**
+
+
+
+## Collect results
+
+
+### OPNFV Test Database in a nutshell
+
+- it's a fair comparison of Neutron implementations (Agents vs SDN
+  controller)
+- it stores all verification results and all performance data from different
+  hardware over the world which could be easily postprocessed
+- it could be very useful to select the adequate opensource solutions
+  regarding metrics and capabilities
+
+
+### which Neutron backend?
+
+- most Neutron standalone and OVN scenarios pass Functest decently
+- no ODL scenarios pass the advanced testcases (benchmarking tools and
+  VNFs). It's still unclear if it's due to the Installers, ODL or POD
+  misconfigurations.
+- no Tungsten Fabric is released in Gambia
+
+**We expected that ODL results would have improved before the first Gambia
+  corrective**
+
+
+### Contrail testing (out of OPNFV)
+
+- **3.X 4.X**: mostly verified except some functional tests about
+  visibility which fail due to the falsy admin role (they can be easily
+  blacklisted)
+- **5.X**: a limited set of bugs in Contrail mostly forbid running few
+  functional tests and benchmarking tools:
+  - wrong external network listing
+  - Contrail doesn't allow booting a VM without network (and elects the wrong
+    network)
+
+
+### And performance?
+
+- OVS DPDK is not fully integrated by an OPNFV installer (Fuel is in a good
+  shape to support it)
+- the only scenario including VPP is not part of Gambia
+
+**From the time being, we can't evaluate the benefits of OVS DPDK or VPP thanks
+  to OPNFV**
+
+
+
+## Conclusion
+
+
+### Gambia
+
+- Functest and Xtesting are powerful and easy to (re)use (containers, jenkins
+  jobs, ansible playbooks, Raspberry PI, etc.).
+- the number of installers and scenarios decreased in Gambia (it's still
+  unclear regarding the overall quality). **What about OPNFV Test Database if
+  it decreases again in 2019?**
+
+
+### And beyond
+
+- test frameworks are now considered as crucial for OPNFV (see [Last OPNFV Marketing update](https://wiki.opnfv.org/download/attachments/2925933/OPNFV%20Marketing%20Update%20091818.pptx?version=1&modificationDate=1537228648000&api=v2)) and Functest and Xtesting could be
+already widely reused out of OPNFV
+- the new test-driven approach as proposed by the [OPNFV Strategic Plan](https://wiki.opnfv.org/download/attachments/2925933/OPNFV%20Strategy%20and%20Plan%20v0.5.pptx?version=1&modificationDate=1540961098000&api=v2) could increase **the quality of all
+scenarios**. But we are also suggesting to let the installers decide their
+test cases.
diff --git a/docs/com/pres/gambia/index.html b/docs/com/pres/gambia/index.html
new file mode 100644
index 000000000..db3a6aaa2
--- /dev/null
+++ b/docs/com/pres/gambia/index.html
@@ -0,0 +1,52 @@
+<html>
+<head>
+<title>Functest on steroids</title>
+<meta name="author" content="Cédric Ollivier">
+<meta name="viewport"
+    content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
+<link rel="stylesheet" href="../reveal.js/css/reveal.css">
+<link rel="stylesheet" href="../reveal.js/css/theme/white.css">
+<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css">
+<script>
+var link = document.createElement( 'link' );
+link.rel = 'stylesheet';
+link.type = 'text/css';
+link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css';
+document.getElementsByTagName( 'head' )[0].appendChild( link );
+</script>
+</head>
+<body>
+    <div class="reveal">
+        <div class="slides">
+            <section data-markdown="gambia.md" data-separator="^\n\n\n"
+                data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section>
+        </div>
+    </div>
+    <script src="../reveal.js/lib/js/head.min.js"></script>
+    <script src="../reveal.js/js/reveal.js"></script>
+    <script>
+        Reveal.initialize({
+            dependencies : [ {
+                src : '../reveal.js/plugin/markdown/marked.js',
+                condition : function() {
+                    return !!document.querySelector('[data-markdown]');
+                }
+            }, {
+                src : '../reveal.js/plugin/markdown/markdown.js',
+                condition : function() {
+                    return !!document.querySelector('[data-markdown]');
+                }
+            }, {
+                src: '../reveal.js/plugin/highlight/highlight.js',
+                async: true,
+                callback: function() {
+                    hljs.initHighlightingOnLoad();
+                }
+            }, {
+            src: '../reveal.js/plugin/notes/notes.js',
+        async: true
+            } ]
+        });
+    </script>
+</body>
+</html>
diff --git a/docs/com/pres/gambia/raspberrypi.jpg b/docs/com/pres/gambia/raspberrypi.jpg
new file mode 100644
index 000000000..60a2638f3
Binary files /dev/null and b/docs/com/pres/gambia/raspberrypi.jpg differ