7 <title>OPNFV presentation</title>
9 <meta name="description" content="Test API">
10 <meta name="author" content="Serena Feng">
12 <meta name="apple-mobile-web-app-capable" content="yes" />
13 <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
15 <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, minimal-ui">
17 <link rel="stylesheet" href="../css/reveal.css">
18 <link rel="stylesheet" href="../css/theme/OPNFV-Berlin.css" id="theme">
20 <!-- Code syntax highlighting -->
21 <link rel="stylesheet" href="../lib/css/zenburn.css">
23 <!-- Printing and PDF exports -->
25 var link = document.createElement( 'link' );
26 link.rel = 'stylesheet';
27 link.type = 'text/css';
28 link.href = window.location.search.match( /print-pdf/gi ) ? '../css/print/pdf.css' : '../css/print/paper.css';
29 document.getElementsByTagName( 'head' )[0].appendChild( link );
33 <script src="lib/js/html5shiv.js"></script>
40 <!-- Any section element inside of this container is displayed as a slide -->
43 <section data-background="../img/title-bg-berlin.png" data-background-transition="none">
46 <h3>Clean, Easy, Complete</h3>
48 <h4>OPNFV testing community</h4>
49 <h5>OPNFV Design Summit, 20/6/2016, Berlin</h5>
53 <section data-markdown>
62 <section data-markdown>
65 <section data-markdown>
66 ## Consistant view for
73 <section data-markdown>
76 * Many formats (log, html, json, ..)
77 * Many locations (in VM, Jumphost, external DB,...)
79 <section data-markdown>
80 <script type='text/template'>
81 ## Achieve using testAPI
82 * Uniform API: testAPI
83 * Uniform format: JSON
84 * Universal Location: http://testresults.opnfv.org
86 By using testAPI, we hope to provide a uniform way of collection and saving test results to a universal location
91 <section data-markdown>
99 <section data-markdown>
102 <section data-markdown>
104 ![testapi](https://wiki.opnfv.org/download/attachments/2926452/results_collection_structure.png?version=1&modificationDate=1459196347000&api=v2 "OPNFV API page")
106 <section data-markdown>
107 ## API in Brahmaputra
108 ![testapi](../img/testapi0.png)
109 https://wiki.opnfv.org/display/functest/Collection+Of+Test+Results
112 <section data-markdown>
113 ## API status in Brahmaputra
114 * Simple data models aggreed with the testing group
116 * Wiki and rst documentation
119 <section data-markdown>
120 ## Lessons learned in B.
121 * Painful wiki documentation
122 * Simple result model
123 * Misuse version as scenario
124 * No version actually
125 * No overall criteria (PASS/FAIL)
126 * Difficult refactoring
129 <h2>Colorado refactoring</h2>
132 <li>Update of the data model</li>
133 <li>Tornado_swagger for the documentation</li>
134 <li>URL and MongoDB collection renaming</li>
135 <li>Scripts to support databse backup/restore/update</li>
136 <li>Support installation</li>
137 <aside class='notes'>
138 So you can discover and understand the capabilities of the service without
139 access to source code, documentation, or through network traffic inspection,
140 and also you can interact with the testAPI directly through swagger website.
143 <p class="fragment fade-up"><b>All done in Colorado!</b></p>
148 <section data-markdown>
153 <h2> URI changes...</h2>
154 <div style="text-align:left"">
155 <p> testresults.opnfv.org/<span style="color:lightblue">testapi</span> => <br>testresults.opnfv.org/<span style="color:yellow">test/api/v1</span> </p>
157 <p> /test/api/v1/<b>pods</b></p>
158 <p> /test/api/v1/<b>projects</b></p>
159 <p> /test/api/v1/projects/qtip/<b>cases</b></p>
160 <p> /test/api/v1/<b>results</b></p>
163 <section data-markdown>
164 ## One new field for pods
167 | role | community_pod, ci_pod, .. |
169 <section data-markdown>
170 ## Changed fields for results
171 | Item | Brahmaputra | Colorado |
172 | :--- | :--- | :--- |
173 | date | creation_date | start_date and stop_date |
174 | version | used as scenario | opnfv release version |
176 <section data-markdown>
177 <script type='text/template'>
178 ## New fields for results
181 | scenario | used for scenario |
182 | criteria | PASS/FAILED |
183 | trust_indicator | between 0 and 1|
184 <aside class='notes'>
185 to detect if the test case is stable over time and further more to avoid
186 running it systematically and saving time for other long duration test cases
190 <section data-markdown>
191 ## One new filter for results
194 | last | get last N results|
196 <section data-markdown>
197 http://testresults.opnfv.org/test/swagger/spec.html
198 ![alt text](../img/testapi1.png "Test API swagger interface")
202 <pre><code class="hljs" data-trim contenteditable>
203 umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh
205 WARNING:tornado.general:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1): Project name missing
206 WARNING:tornado.access:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1) 2.30ms
207 WARNING:tornado.general:400 POST /api/v1/projects (127.0.0.1): name missing
209 WARNING:tornado.access:400 POST /api/v1/projects (127.0.0.1) 1.13ms
210 WARNING:tornado.access:403 PUT /api/v1/projects/functest/cases/vping_1 (127.0.0.1) 2.95ms
211 WARNING:tornado.general:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1): {'project_name': u'functest', 'name': u'notFound'} could not be found in table [testcases]
212 WARNING:tornado.access:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1) 2.85ms
214 Ran 74 tests in 1.848s
221 <section data-markdown>
224 <section data-markdown>
227 * Automatic update of testAPI
228 * Command Line Interface
229 * Automatic update for pods/projects/testcases
231 <section data-markdown>
236 <img width="600" data-src="../img/colorado.png" alt="tests">
241 <img src="../img/logo-OPNFV-Berlin.png" alt="OPNFV logo">
245 <script src="../lib/js/head.min.js"></script>
246 <script src="../js/reveal.js"></script>
250 // Full list of configuration options available at:
251 // https://github.com/hakimel/reveal.js#configuration
258 transition: 'slide', // none/fade/slide/convex/concave/zoom
260 // Optional reveal.js plugins
262 { src: '../lib/js/classList.js', condition: function() { return !document.body.classList; } },
263 { src: '../plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
264 { src: '../plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
265 { src: '../plugin/highlight/highlight.js', async: true, condition: function() { return !!document.querySelector( 'pre code' ); }, callback: function() { hljs.initHighlightingOnLoad(); } },
266 { src: '../plugin/zoom-js/zoom.js', async: true },
267 { src: '../plugin/notes/notes.js', async: true }