1 How to setup the workflow of automatic documentation build for your project
2 ----------------------------------------------------------------------------
4 **Setup you repository and then clone locally**::
8 git clone ssh://<username>@gerrit.opnfv.org:29418/<project>
13 **Inside the repository create the following structure:**::
15 gerrit.opnfv.org/<project>
17 | |-- some-project-description.rst
22 | |-- some-release-doc.rst
26 | |-- requirements.rst
30 | |-- some-design-doc.rst
33 |-- some_project_file.py
34 |-- some_shell_script.sh
39 More details about the default structure you can find `here <https://wiki.opnfv.org/documentation>`_ at paragraph "How and where to store the document content files in your repository".
41 **In order to obtain a nice .html & .pdf at then end you must write you documentation using reSt markup**
45 * http://docutils.sourceforge.net/docs/user/rst/quickref.html
46 * http://rest-sphinx-memo.readthedocs.org/en/latest/ReST.html
47 * http://www.math.uiuc.edu/~gfrancis/illimath/windows/aszgard_mini/movpy-2.0.0-py2.4.4/manuals/docutils/ref/rst/directives.html
49 An `nice online editor <http://rst.ninjs.org/>`_ that will help you write reSt and see your changes live. After done editing you can copy the source document in the repository and follow the workflow.
52 **Clone the releng repository so you can created jobs for JJB**::
54 git clone ssh://<username>@gerrit.opnfv.org:29418/releng
57 Enter the project settings::
59 cd releng/jjb/<project>/
62 **Create the verify & build scripts**
64 The scripts are the same for most projects and if you need customizations copy them
65 under your project in releng/jjb/<project>/::
67 cp releng/jjb/opnfvdocs/build-docu.sh releng/jjb/<your-project>/
69 and change according to you needs.
71 If standard will suffice for you skip this step and jump to **Edit <your-project>.yml**, **Variant 1 - standard**
81 project="$(git remote -v | head -n1 | awk '{{print $2}}' | sed -e 's,.*:\(.*/\)\?,,' -e 's/\.git$//')"
82 export PATH=$PATH:/usr/local/bin/
84 git_sha1="$(git rev-parse HEAD)"
85 docu_build_date="$(date)"
88 while read -r -d ''; do
90 done < <(find * -type f -iname '*.rst' -print0)
92 for file in "${{files[@]}}"; do
94 file_cut="${{file%.*}}"
95 gs_cp_folder="${{file_cut}}"
98 # add one '_' at the end of each trigger variable; ex: _sha1 +'_' & _date + '_' on both of the lines below
99 # they were added here without the '_'suffix to avoid sed replacement
100 sed -i "s/_sha1/$git_sha1/g" $file
101 sed -i "s/_date/$docu_build_date/g" $file
104 echo "rst2html $file"
105 rst2html --halt=2 $file | gsutil cp -L gsoutput.txt - \
106 gs://artifacts.opnfv.org/"$project"/"$gs_cp_folder".html
107 gsutil setmeta -h "Content-Type:text/html" \
108 -h "Cache-Control:private, max-age=0, no-transform" \
109 gs://artifacts.opnfv.org/"$project"/"$gs_cp_folder".html
114 rst2pdf $file -o - | gsutil cp -L gsoutput.txt - \
115 gs://artifacts.opnfv.org/"$project"/"$gs_cp_folder".pdf
116 gsutil setmeta -h "Content-Type:application/pdf" \
117 -h "Cache-Control:private, max-age=0, no-transform" \
118 gs://artifacts.opnfv.org/"$project"/"$gs_cp_folder".pdf
125 while read -r -d ''; do
127 done < <(find * -type f \( -iname \*.jpg -o -iname \*.png \) -print0)
129 for img in "${{images[@]}}"; do
131 # uploading found images
132 echo "uploading $img"
133 cat "$img" | gsutil cp -L gsoutput.txt - \
134 gs://artifacts.opnfv.org/"$project"/"$img"
135 gsutil setmeta -h "Content-Type:image/jpeg" \
136 -h "Cache-Control:private, max-age=0, no-transform" \
137 gs://artifacts.opnfv.org/"$project"/"$img"
143 #the double {{ in file_cut="${{file%.*}}" is to escape jjb's yaml
154 project="$(git remote -v | head -n1 | awk '{{print $2}}' | sed -e 's,.*:\(.*/\)\?,,' -e 's/\.git$//')"
155 export PATH=$PATH:/usr/local/bin/
157 git_sha1="$(git rev-parse HEAD)"
158 docu_build_date="$(date)"
161 while read -r -d ''; do
163 done < <(find * -type f -iname '*.rst' -print0)
165 for file in "${{files[@]}}"; do
167 file_cut="${{file%.*}}"
168 gs_cp_folder="${{file_cut}}"
171 # add one '_' at the end of each trigger variable; ex: _sha1 +'_' & _date + '_' on both of the lines below
172 # they were added here without the '_'suffix to avoid sed replacement
173 sed -i "s/_sha1/$git_sha1/g" $file
174 sed -i "s/_date/$docu_build_date/g" $file
177 echo "rst2html $file"
178 rst2html --exit-status=2 $file > $file_cut".html"
181 rst2pdf $file -o $file_cut".pdf"
185 #the double {{ in file_cut="${{file%.*}}" is to escape jjb's yaml
188 **Edit <your-project>.yml**::
190 vi releng/jjb/<your-project>/<your-project>.yml
193 Make sure you have the job-templates set correctly as below.
196 vi releng/jjb/opnfvdocs/opnfvdocs.yml
197 # make sure you are using one of the variants below and that !include-raw directive is present
200 ---------------------
202 By chosing **Variant 1** you will use the scripts from opnfvdocs project.
204 **<your-project>.yml**::
207 name: 'opnfvdocs-daily-{stream}'
213 !include-raw ../opnfvdocs/docu-build.sh
216 name: 'opnfvdocs-verify'
222 !include-raw ../opnfvdocs/docu-verify.sh
225 name: 'opnfvdocs-merge'
231 !include-raw ../opnfvdocs/docu-build.sh
237 **<your-project>.yml**::
240 name: 'opnfvdocs-daily-{stream}'
246 !include-raw docu-build.sh
249 name: 'opnfvdocs-verify'
255 !include-raw docu-verify.sh
258 name: 'opnfvdocs-merge'
264 !include-raw docu-build.sh
267 "node: master" is important here as all documentations are built on Jenkins master node for now.
269 Please reffer to the releng repository for the correct indentation as JJB is very picky
270 with those and also for the rest of the code that is missing in the example code and replaced by "...".
271 Also you must have your documentation under docs/ in the repository or gsutil will fail to copy them;
272 for customizations you might need to addapt build-docu.sh as we did for genesis project
273 as different documents need to go into different places.
276 Stage files example::
278 git add docu-build.sh docu-verify.sh <project>.yml
281 Commit change with --signoff::
286 Send code for review in Gerrit::
291 Create the documentation using the recommended structure in your repository and submit to gerrit for review
294 **Jenkins will take over and produce artifacts in the form of .html & .pdf**
296 Jenkins has the proper packages installed in order to produce the artifacts.
299 **Artifacts are stored on Google Storage (still to decide where, structure and how to present them)**
301 http://artifacts.opnfv.org/
304 `Here you can download the PDF version <http://artifacts.opnfv.org/opnfvdocs/docs/enable_docu_gen.pdf>`_ of this guide.
307 **Scrape content from html artifacts on wiki**
309 This section describes how the html build artifacts can be made visible on Wiki using he scrape method.
310 DokuWiki speeds up browsing through the wiki by caching parsed files1).
311 If a currently cached version of a document exists, this cached copy is delivered
312 instead of parsing the data again. On editing and previewing no cache is used.
314 To prevent a page from ever being cached, use the NOCACHE tag anywhere in the document.
315 This is useful if the page contains dynamic content, e.g. PHP code that pulls in outside information,
316 where the caching would prevent the most recent information from being displayed.
317 Same applies if documentation artifacts are rebuilt the cached version is shown
318 if the NOCACHE tag is not used.
320 https://www.dokuwiki.org/caching
322 In order to have you documentation on Wiki you need to create a wiki page and include an adaption of the code below:
328 {{scrape>http://artifacts.opnfv.org/opnfvdocs/docs/enable_docu_gen.html}}
331 Please try to write documentation as accurate and clear as possible as once reviewed and
332 merged it will be automatically built and displayed on Wiki and
333 everyone would apreciate a good written/nice looking guide.
335 If you want to see on wiki what code is scraped from the built artifacts click "Show pagesource" in the right
336 (it will appear if you hover over the magnifier icon);
337 this way you know what is written straight on wiki and what is embedded with "scrape".
338 By knowing these details you will be able to prevent damages by manually updating wiki.
341 **Wiki update - how it works**
343 Edit Wiki page https://wiki.opnfv.org/<page> and look for {{scrape>http://artifacts.opnfv.org/<project>/<folder>/<doc-file>.html}}
344 Click "Preview" and see if the change you submitted to Git is present;
345 add a short description in "Edit summary" field, then click "Save" to update the page.
346 This extra step is needed as Wiki does not auto update content for now.
349 **How to track documentation**
351 You must include at the bottom of every document that you want to track the following::
353 **Documentation tracking**
359 **Image inclusion for artifacts**
361 Create a folder called images in the same folder where you documentation resides and copy .jpg or .png files there, according to the guide here: https://wiki.opnfv.org/documentation
363 Here is an example of what you need to include in the .rst files to include an image::
365 .. image:: images/smiley.png
368 :alt: Just a smiley face!
371 The image will be shown in both .html and .pdf resulting artifacts.
377 In order to generate html & pdf documentation the needed packages are rst2pdf & python-docutils
378 if the Jenkins is CentOS/RHEL; many variants have been tested but this is the cleanest solution found.
379 For html generation it also supports css styles if needed.
382 **Documentation tracking**