Adds Yardstick test case TC008, Network Performance Extended Tests

New test case using pktgen tool. Based on TC001, but with additional
packet sizes and more repetitions of each test combination,
respectively. Test case takes approx.3 hours to complete.

Change-Id: Iba2de5efa33d6e2c960772fa5345595f6d7ab4e6
JIRA: YARDSTICK-80

diff --git a/docs/yardstick/opnfv_yardstick_tc008.rst b/docs/yardstick/opnfv_yardstick_tc008.rst
new file mode 100644 (file)
index 0000000..f4971fb
--- /dev/null
+++ b/docs/yardstick/opnfv_yardstick_tc008.rst
@@ -0,0 +1,71 @@
+*************************************
+Yardstick Test Case Description TC008
+*************************************
++-----------------------------------------------------------------------------+
+|Network Performance                                                          |
++==============+==============================================================+
+|test case id  | OPNFV_YARDSTICK_TC008_NW PERF, Packet loss Extended Test     |
++--------------+--------------------------------------------------------------+
+|metric        | Number of flows, packet size and throughput                  |
++--------------+--------------------------------------------------------------+
+|test purpose  | To evaluate the IaaS network performance with regards to     |
+|              | flows and throughput, such as if and how different amounts   |
+|              | of packet sizes and flows matter for the throughput between  |
+|              | VMs on different compute blades. Typically e.g. the          |
+|              | performance of a vSwitch                                     |
+|              | depends on the number of flows running through it. Also      |
+|              | performance of other equipment or entities can depend        |
+|              | on the number of flows or the packet sizes used.             |
+|              | The purpose is also to be able to spot trends. Test results, |
+|              | graphs ans similar shall be stored for comparison reasons and|
+|              | product evolution understanding between different OPNFV      |
+|              | versions and/or configurations.                              |
++--------------+--------------------------------------------------------------+
+|configuration | file: opnfv_yardstick_tc008.yaml                             |
+|              |                                                              |
+|              | Packet size: 64, 128, 256, 512, 1024, 1280 and 1518 bytes.   |
+|              |                                                              |
+|              | Number of ports: 1, 10, 50, 100, 500 and 1000. The amount of |
+|              | configured ports map from 2 up to 1001000 flows,             |
+|              | respectively. Each packet_size/port_amount combination is run|
+|              | ten times, for 20 seconds each. Then the next                |
+|              | packet_size/port_amount combination is run, and so on.       |
+|              |                                                              |
+|              | The client and server are distributed on different HW.       |
+|              |                                                              |
+|              | For SLA max_ppm is set to 1000.                              |
++--------------+--------------------------------------------------------------+
+|test tool     | pktgen                                                       |
+|              |                                                              |
+|              | (Pktgen is not always part of a Linux distribution, hence it |
+|              | needs to be installed. It is part of the Yardstick Docker    |
+|              | image.                                                       |
+|              | As an example see the /yardstick/tools/ directory for how    |
+|              | to generate a Linux image with pktgen included.)             |
++--------------+--------------------------------------------------------------+
+|references    |https://www.kernel.org/doc/Documentation/networking/pktgen.txt|
+|              |                                                              |
+|              |ETSI-NFV-TST001                                               |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different packet sizes, amount   |
+|              | of flows and test duration. Default values exist.            |
+|              |                                                              |
+|              |SLA (optional):                                               |
+|              |    max_ppm: The number of packets per million packets sent   |
+|              |             that are acceptable to lose, i.e. not received.  |
++--------------+--------------------------------------------------------------+
+|pre-test      | The test case image needs to be installed into Glance        |
+|conditions    | with pktgen included in it.                                  |
+|              |                                                              |
+|              | No POD specific requirements have been identified.           |
++--------------+------+----------------------------------+--------------------+
+|test sequence | step | description                      | result             |
+|              +------+----------------------------------+--------------------+
+|              |  1   | The hosts are installed, as      | Logs are stored    |
+|              |      | server and client. pktgen is     |                    |
+|              |      | invoked and logs are produced    |                    |
+|              |      | and stored.                      |                    |
++--------------+------+----------------------------------+--------------------+
+|test verdict  | Fails only if SLA is not passed, or if there is a test case  |
+|              | execution problem.                                           |
++--------------+--------------------------------------------------------------+

diff --git a/tests/opnfv/test_cases/opnfv_yardstick_tc008.yaml b/tests/opnfv/test_cases/opnfv_yardstick_tc008.yaml
new file mode 100644 (file)
index 0000000..385e530
--- /dev/null
+++ b/tests/opnfv/test_cases/opnfv_yardstick_tc008.yaml
@@ -0,0 +1,58 @@
+---
+# Yardstick TC008 config file
+# Measure network throughput and packet loss using Pktgen.
+# Different amount of flows, from 2 up to 1001000, in combination
+# with different packet sizes are run in each test.
+# Each combination of packet size and flow amount is run 10 times.
+# First 10 times with the smallest packet size, starting with the
+# least amount of ports/flows, then next amount of ports with same
+# packet size, and so on. The test sequence continues with the next
+# packet size, with same ports/flows sequence as before.
+
+schema: "yardstick:task:0.1"
+
+scenarios:
+{% for pkt_size in [64, 128, 256, 512, 1024, 1280, 1518] %}
+  {% for num_ports in [1, 10, 50, 100, 500, 1000] %}
+-
+  type: Pktgen
+  options:
+    packetsize: {{pkt_size}}
+    number_of_ports: {{num_ports}}
+    duration: 20
+
+  host: demeter.yardstick-TC008
+  target: poseidon.yardstick-TC008
+
+  runner:
+    type: Iteration
+    iterations: 10
+    interval: 1
+
+  sla:
+    max_ppm: 1000
+    action: monitor
+  {% endfor %}
+{% endfor %}
+
+context:
+  name: yardstick-TC008
+  image: yardstick-trusty-server
+  flavor: yardstick-flavor
+  user: ec2-user
+
+  placement_groups:
+    pgrp1:
+      policy: "availability"
+
+  servers:
+    demeter:
+      floating_ip: true
+      placement: "pgrp1"
+    poseidon:
+      floating_ip: true
+      placement: "pgrp1"
+
+  networks:
+    test:
+      cidr: '10.0.1.0/24'