--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel, Ericsson AB, Huawei Technologies Co. Ltd and others.
+
+..
+ Convention for heading levels in Yardstick:
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ^^^^^^^ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+ Avoid deeper levels because they do not render well.
+
+===============
+Yardstick Usage
+===============
+
+Once you have yardstick installed, you can start using it to run testcases
+immediately, through the CLI. You can also define and run new testcases and
+test suites. This chapter details basic usage (running testcases), as well as
+more advanced usage (creating your own testcases).
+
+Yardstick common CLI
+--------------------
+
+List test cases
+^^^^^^^^^^^^^^^
+
+``yardstick testcase list``: This command line would list all test cases in
+Yardstick. It would show like below::
+
+ +---------------------------------------------------------------------------------------
+ | Testcase Name | Description
+ +---------------------------------------------------------------------------------------
+ | opnfv_yardstick_tc001 | Measure network throughput using pktgen
+ | opnfv_yardstick_tc002 | measure network latency using ping
+ | opnfv_yardstick_tc005 | Measure Storage IOPS, throughput and latency using fio.
+ ...
+ +---------------------------------------------------------------------------------------
+
+
+Show a test case config file
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Take opnfv_yardstick_tc002 for an example. This test case measure network
+latency. You just need to type in ``yardstick testcase show
+opnfv_yardstick_tc002``, and the console would show the config yaml of this
+test case::
+
+ ---
+
+ schema: "yardstick:task:0.1"
+ description: >
+ Yardstick TC002 config file;
+ measure network latency using ping;
+
+ {% set image = image or "cirros-0.3.5" %}
+
+ {% set provider = provider or none %}
+ {% set physical_network = physical_network or 'physnet1' %}
+ {% set segmentation_id = segmentation_id or none %}
+ {% set packetsize = packetsize or 100 %}
+
+ scenarios:
+ {% for i in range(2) %}
+ -
+ type: Ping
+ options:
+ packetsize: {{packetsize}}
+ host: athena.demo
+ target: ares.demo
+
+ runner:
+ type: Duration
+ duration: 60
+ interval: 10
+
+ sla:
+ max_rtt: 10
+ action: monitor
+ {% endfor %}
+
+ context:
+ name: demo
+ image: {{image}}
+ flavor: yardstick-flavor
+ user: cirros
+
+ placement_groups:
+ pgrp1:
+ policy: "availability"
+
+ servers:
+ athena:
+ floating_ip: true
+ placement: "pgrp1"
+ ares:
+ placement: "pgrp1"
+
+ networks:
+ test:
+ cidr: '10.0.1.0/24'
+ {% if provider == "vlan" %}
+ provider: {{provider}}
+ physical_network: {{physical_network}}
+ {% if segmentation_id %}
+ segmentation_id: {{segmentation_id}}
+ {% endif %}
+ {% endif %}
+
+
+Run a Yardstick test case
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you want run a test case, then you need to use ``yardstick task start
+<test_case_path>`` this command support some parameters as below:
+
+ +---------------------+--------------------------------------------------+
+ | Parameters | Detail |
+ +=====================+==================================================+
+ | -d | show debug log of yardstick running |
+ | | |
+ +---------------------+--------------------------------------------------+
+ | --task-args | If you want to customize test case parameters, |
+ | | use "--task-args" to pass the value. The format |
+ | | is a json string with parameter key-value pair. |
+ | | |
+ +---------------------+--------------------------------------------------+
+ | --task-args-file | If you want to use yardstick |
+ | | env prepare command(or |
+ | | related API) to load the |
+ +---------------------+--------------------------------------------------+
+ | --parse-only | |
+ | | |
+ | | |
+ +---------------------+--------------------------------------------------+
+ | --output-file \ | Specify where to output the log. if not pass, |
+ | OUTPUT_FILE_PATH | the default value is |
+ | | "/tmp/yardstick/yardstick.log" |
+ | | |
+ +---------------------+--------------------------------------------------+
+ | --suite \ | run a test suite, TEST_SUITE_PATH specify where |
+ | TEST_SUITE_PATH | the test suite locates |
+ | | |
+ +---------------------+--------------------------------------------------+
+
+
+Run Yardstick in a local environment
+------------------------------------
+
+We also have a guide about `How to run Yardstick in a local environment`_.
+This work is contributed by Tapio Tallgren.
+
+Create a new testcase for Yardstick
+-----------------------------------
+
+As a user, you may want to define a new testcase in addition to the ones
+already available in Yardstick. This section will show you how to do this.
+
+Each testcase consists of two sections:
+
+* ``scenarios`` describes what will be done by the test
+* ``context`` describes the environment in which the test will be run.
+
+Defining the testcase scenarios
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+TODO
+
+Defining the testcase context(s)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Each testcase consists of one or more contexts, which describe the environment
+in which the testcase will be run.
+Current available contexts are:
+
+* ``Dummy``: this is a no-op context, and is used when there is no environment
+ to set up e.g. when testing whether OpenStack services are available
+* ``Node``: this context is used to perform operations on baremetal servers
+* ``Heat``: uses OpenStack to provision the required hosts, networks, etc.
+* ``Kubernetes``: uses Kubernetes to provision the resources required for the
+ test.
+
+Regardless of the context type, the ``context`` section of the testcase will
+consist of the following::
+
+ context:
+ name: demo
+ type: Dummy|Node|Heat|Kubernetes
+
+The content of the ``context`` section will vary based on the context type.
+
+Dummy Context
++++++++++++++
+
+No additional information is required for the Dummy context::
+
+ context:
+ name: my_context
+ type: Dummy
+
+Node Context
+++++++++++++
+
+TODO
+
+Heat Context
+++++++++++++
+
+In addition to ``name`` and ``type``, a Heat context requires the following
+arguments:
+
+* ``image``: the image to be used to boot VMs
+* ``flavor``: the flavor to be used for VMs in the context
+* ``user``: the username for connecting into the VMs
+* ``networks``: The networks to be created, networks are identified by name
+
+ * ``name``: network name (required)
+ * (TODO) Any optional attributes
+
+* ``servers``: The servers to be created
+
+ * ``name``: server name
+ * (TODO) Any optional attributes
+
+In addition to the required arguments, the following optional arguments can be
+passed to the Heat context:
+
+* ``placement_groups``:
+
+ * ``name``: the name of the placement group to be created
+ * ``policy``: either ``affinity`` or ``availability``
+* ``server_groups``:
+
+ * ``name``: the name of the server group
+ * ``policy``: either ``affinity`` or ``anti-affinity``
+
+Combining these elements together, a sample Heat context config looks like:
+
+.. literalinclude::
+ ../../../../yardstick/tests/integration/dummy-scenario-heat-context.yaml
+ :start-after: ---
+
+Using exisiting HOT Templates
+'''''''''''''''''''''''''''''
+
+TODO
+
+Kubernetes Context
+++++++++++++++++++
+
+TODO
+
+Using multiple contexts in a testcase
++++++++++++++++++++++++++++++++++++++
+
+When using multiple contexts in a testcase, the ``context`` section is replaced
+by a ``contexts`` section, and each context is separated with a ``-`` line::
+
+ contexts:
+ -
+ name: context1
+ type: Heat
+ ...
+ -
+ name: context2
+ type: Node
+ ...
+
+
+Reusing a context
++++++++++++++++++
+
+Typically, a context is torn down after a testcase is run, however, the user
+may wish to keep an context intact after a testcase is complete.
+
+.. note::
+ This feature has been implemented for the Heat context only
+
+To keep or reuse a context, the ``flags`` option must be specified:
+
+* ``no_setup``: skip the deploy stage, and fetch the details of a deployed
+ context/Heat stack.
+* ``no_teardown``: skip the undeploy stage, thus keeping the stack intact for
+ the next test
+
+If either of these ``flags`` are ``True``, the context information must still
+be given. By default, these flags are disabled::
+
+ context:
+ name: mycontext
+ type: Heat
+ flags:
+ no_setup: True
+ no_teardown: True
+ ...
+
+Create a test suite for Yardstick
+---------------------------------
+
+A test suite in Yardstick is a .yaml file which includes one or more test
+cases. Yardstick is able to support running test suite task, so you can
+customize your own test suite and run it in one task.
+
+``tests/opnfv/test_suites`` is the folder where Yardstick puts CI test suite.
+A typical test suite is like below (the ``fuel_test_suite.yaml`` example)::
+
+ ---
+ # Fuel integration test task suite
+
+ schema: "yardstick:suite:0.1"
+
+ name: "fuel_test_suite"
+ test_cases_dir: "samples/"
+ test_cases:
+ -
+ file_name: ping.yaml
+ -
+ file_name: iperf3.yaml
+
+As you can see, there are two test cases in the ``fuel_test_suite.yaml``. The
+``schema`` and the ``name`` must be specified. The test cases should be listed
+via the tag ``test_cases`` and their relative path is also marked via the tag
+``test_cases_dir``.
+
+Yardstick test suite also supports constraints and task args for each test
+case. Here is another sample (the ``os-nosdn-nofeature-ha.yaml`` example) to
+show this, which is digested from one big test suite::
+
+ ---
+
+ schema: "yardstick:suite:0.1"
+
+ name: "os-nosdn-nofeature-ha"
+ test_cases_dir: "tests/opnfv/test_cases/"
+ test_cases:
+ -
+ file_name: opnfv_yardstick_tc002.yaml
+ -
+ file_name: opnfv_yardstick_tc005.yaml
+ -
+ file_name: opnfv_yardstick_tc043.yaml
+ constraint:
+ installer: compass
+ pod: huawei-pod1
+ task_args:
+ huawei-pod1: '{"pod_info": "etc/yardstick/.../pod.yaml",
+ "host": "node4.LF","target": "node5.LF"}'
+
+As you can see in test case ``opnfv_yardstick_tc043.yaml``, there are two
+tags, ``constraint`` and ``task_args``. ``constraint`` is to specify which
+installer or pod it can be run in the CI environment. ``task_args`` is to
+specify the task arguments for each pod.
+
+All in all, to create a test suite in Yardstick, you just need to create a
+yaml file and add test cases, constraint or task arguments if necessary.
+
+References
+----------
+
+.. _`How to run Yardstick in a local environment`: https://wiki.opnfv.org/display/yardstick/How+to+run+Yardstick+in+a+local+environment
Code Review / yardstick.git / commitdiff