4 Yardstick is a project dealing with performance testing. Yardstick produces its own test cases but can also be considered as a framework to support feature project testing.
6 Yardstick developed a test API that can be used by any OPNFV project. Therefore there are many ways to contribute to Yardstick.
10 * Develop new test cases
12 * Develop Yardstick API / framework
13 * Develop Yardstick grafana dashboards and Yardstick reporting page
14 * Write Yardstick documentation
16 This developer guide describes how to interact with the Yardstick project.
17 The first section details the main working areas of the project. The Second
18 part is a list of “How to” to help you to join the Yardstick family whatever
19 your field of interest is.
21 Where can I find some help to start?
22 --------------------------------------
24 .. _`user guide`: http://artifacts.opnfv.org/yardstick/danube/1.0/docs/stesting_user_userguide/index.html
25 .. _`wiki page`: https://wiki.opnfv.org/display/yardstick/
27 This guide is made for you. You can have a look at the `user guide`_.
28 There are also references on documentation, video tutorials, tips in the
29 project `wiki page`_. You can also directly contact us by mail with [Yardstick] prefix in the title at opnfv-tech-discuss@lists.opnfv.org or on the IRC chan #opnfv-yardstick.
32 Yardstick developer areas
33 ==========================
38 Yardstick can be considered as a framework. Yardstick is release as a docker
39 file, including tools, scripts and a CLI to prepare the environement and run
40 tests. It simplifies the integration of external test suites in CI pipeline
41 and provide commodity tools to collect and display results.
43 Since Danube, test categories also known as tiers have been created to group
44 similar tests, provide consistant sub-lists and at the end optimize test
45 duration for CI (see How To section).
47 The definition of the tiers has been agreed by the testing working group.
64 The installation and configuration of the Yardstick is described in the `user guide`_.
66 How can I contribute to Yardstick?
67 -----------------------------------
69 If you are already a contributor of any OPNFV project, you can contribute to
70 Yardstick. If you are totally new to OPNFV, you must first create your Linux
71 Foundation account, then contact us in order to declare you in the repository
74 We distinguish 2 levels of contributors:
76 * the standard contributor can push patch and vote +1/0/-1 on any Yardstick patch
77 * The commitor can vote -2/-1/0/+1/+2 and merge
79 Yardstick commitors are promoted by the Yardstick contributors.
81 Gerrit & JIRA introduction
82 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
84 .. _Gerrit: https://www.gerritcodereview.com/
85 .. _`OPNFV Gerrit`: http://gerrit.opnfv.org/
86 .. _link: https://identity.linuxfoundation.org/
87 .. _JIRA: https://jira.opnfv.org/secure/Dashboard.jspa
89 OPNFV uses Gerrit_ for web based code review and repository management for the
90 Git Version Control System. You can access `OPNFV Gerrit`_. Please note that
91 you need to have Linux Foundation ID in order to use OPNFV Gerrit. You can get one from this link_.
93 OPNFV uses JIRA_ for issue management. An important principle of change
94 management is to have two-way trace-ability between issue management
95 (i.e. JIRA_) and the code repository (via Gerrit_). In this way, individual
96 commits can be traced to JIRA issues and we also know which commits were used
97 to resolve a JIRA issue.
99 If you want to contribute to Yardstick, you can pick a issue from Yardstick's
100 JIRA dashboard or you can create you own issue and submit it to JIRA.
102 Install Git and Git-reviews
103 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
105 Installing and configuring Git and Git-Review is necessary in order to submit
106 code to Gerrit. The `Getting to the code <https://wiki.opnfv.org/display/DEV/Developer+Getting+Started>`_ page will provide you with some help for that.
109 Verify your patch locally before submitting
110 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
112 Once you finish a patch, you can submit it to Gerrit for code review. A
113 developer sends a new patch to Gerrit will trigger patch verify job on Jenkins
114 CI. The yardstick patch verify job includes python flake8 check, unit test and
115 code coverage test. Before you submit your patch, it is recommended to run the
116 patch verification in your local environment first.
118 Open a terminal window and set the project's directory to the working
119 directory using the ``cd`` command. Assume that ``YARDSTICK_REPO_DIR`` is the path to the Yardstick project folder on your computer::
121 cd $YARDSTICK_REPO_DIR
127 It is used in CI but also by the CLI.
129 Submit the code with Git
130 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
132 Tell Git which files you would like to take into account for the next commit.
133 This is called 'staging' the files, by placing them into the staging area,
134 using the ``git add`` command (or the synonym ``git stage`` command)::
136 git add $YARDSTICK_REPO_DIR/samples/sample.yaml
138 Alternatively, you can choose to stage all files that have been modified (that
139 is the files you have worked on) since the last time you generated a commit,
140 by using the `-a` argument::
144 Git won't let you push (upload) any code to Gerrit if you haven't pulled the
145 latest changes first. So the next step is to pull (download) the latest
146 changes made to the project by other collaborators using the ``pull`` command::
150 Now that you have the latest version of the project and you have staged the
151 files you wish to push, it is time to actually commit your work to your local
154 git commit --signoff -m "Title of change"
156 Test of change that describes in high level what was done. There is a lot of
157 documentation in code so you do not need to repeat it here.
161 .. _`this document`: http://chris.beams.io/posts/git-commit/
163 The message that is required for the commit should follow a specific set of
164 rules. This practice allows to standardize the description messages attached
165 to the commits, and eventually navigate among the latter more easily.
167 `This document`_ happened to be very clear and useful to get started with that.
169 Push the code to Gerrit for review
170 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
172 Now that the code has been comitted into your local Git repository the
173 following step is to push it online to Gerrit for it to be reviewed. The
174 command we will use is ``git review``::
178 This will automatically push your local commit into Gerrit. You can add
179 Yardstick committers and contributors to review your codes.
181 .. image:: images/review.PNG
183 :alt: Gerrit for code review
185 You can find Yardstick people info `here <https://wiki.opnfv.org/display/yardstick/People>`_.
187 Modify the code under review in Gerrit
188 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
190 At the same time the code is being reviewed in Gerrit, you may need to edit it
191 to make some changes and then send it back for review. The following steps go
192 through the procedure.
194 Once you have modified/edited your code files under your IDE, you will have to
195 stage them. The 'status' command is very helpful at this point as it provides
196 an overview of Git's current state::
200 The output of the command provides us with the files that have been modified
201 after the latest commit.
203 You can now stage the files that have been modified as part of the Gerrit code
204 review edition/modification/improvement using ``git add`` command. It is now
205 time to commit the newly modified files, but the objective here is not to
206 create a new commit, we simply want to inject the new changes into the
207 previous commit. You can achieve that with the '--amend' option on the
208 ``git commit`` command::
212 If the commit was successful, the ``git status`` command should not return the
213 updated files as about to be commited.
215 The final step consists in pushing the newly modified commit to Gerrit::
223 For information about Yardstick plugins, refer to the chapter **Installing a plug-in into Yardstick** in the `user guide`_.