Add support in task file for the template syntax based on Jinja2.
JIRA:YARDSTICK-101
Change-Id: I24be133ba590510612d97a1fce6c024e6edb57e4
Signed-off-by: kubi <jean.gaoliang@huawei.com>
--- /dev/null
+Task Template Syntax
+====================
+
+Basic template syntax
+---------------------
+A nice feature of the input task format used in Yardstick is that it supports the template syntax based on Jinja2.
+This turns out to be extremely useful when, say, you have a fixed structure of your task but you want to
+parameterize this task in some way.
+For example, imagine your input task file (task.yaml) runs a set of Ping scenarios:
+
+::
+
+ # Sample benchmark task config file
+ # measure network latency using ping
+ schema: "yardstick:task:0.1"
+
+ scenarios:
+ -
+ type: Ping
+ options:
+ packetsize: 200
+ host: athena.demo
+ target: ares.demo
+
+ runner:
+ type: Duration
+ duration: 60
+ interval: 1
+
+ sla:
+ max_rtt: 10
+ action: monitor
+
+ context:
+ ...
+
+Let's say you want to run the same set of scenarios with the same runner/context/sla,
+but you want to try another packetsize to compare the performance.
+The most elegant solution is then to turn the packetsize name into a template variable:
+
+::
+
+ # Sample benchmark task config file
+ # measure network latency using ping
+
+ schema: "yardstick:task:0.1"
+ scenarios:
+ -
+ type: Ping
+ options:
+ packetsize: {{packetsize}}
+ host: athena.demo
+ target: ares.demo
+
+ runner:
+ type: Duration
+ duration: 60
+ interval: 1
+
+ sla:
+ max_rtt: 10
+ action: monitor
+
+ context:
+ ...
+
+and then pass the argument value for {{packetsize}} when starting a task with this configuration file.
+Yardstick provides you with different ways to do that:
+
+1.Pass the argument values directly in the command-line interface (with either a JSON or YAML dictionary):
+
+::
+
+ yardstick task start samples/ping-template.yaml --task-args '{"packetsize": "200"}'
+
+2.Refer to a file that specifies the argument values (JSON/YAML):
+
+::
+
+ yardstick task start samples/ping-template.yaml --task-args-file args.yaml
+
+Using the default values
+------------------------
+Note that the Jinja2 template syntax allows you to set the default values for your parameters.
+With default values set, your task file will work even if you don't parameterize it explicitly while starting a task.
+The default values should be set using the {% set ... %} clause (task.yaml).For example:
+
+::
+
+ # Sample benchmark task config file
+ # measure network latency using ping
+ schema: "yardstick:task:0.1"
+ {% set packetsize = packetsize or "100" %}
+ scenarios:
+ -
+ type: Ping
+ options:
+ packetsize: {{packetsize}}
+ host: athena.demo
+ target: ares.demo
+
+ runner:
+ type: Duration
+ duration: 60
+ interval: 1
+ ...
+
+If you don't pass the value for {{packetsize}} while starting a task, the default one will be used.
+
+Advanced templates
+------------------
+Yardstick makes it possible to use all the power of Jinja2 template syntax, including the mechanism of built-in functions.
+As an example, let us make up a task file that will do a block storage performance test.
+The input task file (fio-template.yaml) below uses the Jinja2 for-endfor construct to accomplish that:
+
+::
+
+ #Test block sizes of 4KB, 8KB, 64KB, 1MB
+ #Test 5 workloads: read, write, randwrite, randread, rw
+ schema: "yardstick:task:0.1"
+
+ scenarios:
+ {% for bs in ['4k', '8k', '64k', '1024k' ] %}
+ {% for rw in ['read', 'write', 'randwrite', 'randread', 'rw' ] %}
+ -
+ type: Fio
+ options:
+ filename: /home/ec2-user/data.raw
+ bs: {{bs}}
+ rw: {{rw}}
+ ramp_time: 10
+ host: fio.demo
+ runner:
+ type: Duration
+ duration: 60
+ interval: 60
+
+ {% endfor %}
+ {% endfor %}
+ context
+ ...
--- /dev/null
+# Sample benchmark task config file
+# measure storage performance using fio
+# Jinja2 Syntax is supported
+# using built-in functions ( Jinja2 for-endfor construct ) to test complex tasks
+# Test block sizes of 4KB, 8KB, 64KB, 1MB
+# Test 5 workloads: 4 corners and 1 mixed :read, write, randwrite, randread, rw
+schema: "yardstick:task:0.1"
+
+scenarios:
+{% for rw in ['read', 'write', 'randwrite', 'randread', 'rw'] %}
+ {% for bs in ['4k', '8k', '64k', '1024k'] %}
+-
+ type: Fio
+ options:
+ filename: /home/ec2-user/data.raw
+ bs: {{bs}}
+ rw: {{rw}}
+ ramp_time: 10
+ duration: 20
+ host: fio.demo
+ runner:
+ type: Iteration
+ iterations: 2
+ interval: 1
+ {% endfor %}
+{% endfor %}
+
+context:
+ name: demo
+ image: yardstick-trusty-server
+ flavor: yardstick-flavor
+ user: ec2-user
+ servers:
+ fio:
+ floating_ip: true
+ networks:
+ test:
+ cidr: "10.0.1.0/24"
+ external_network: "net04_ext"
+
--- /dev/null
+# Sample benchmark task config file
+# measure network latency using ping
+# Jinja2 Syntax is supported
+# parameterize this task, {{packetsize}} is passed to the scenario as an argument
+# If you don't pass the value for {{packetsize}} while starting a task,
+# the default one will be used.
+
+
+schema: "yardstick:task:0.1"
+{% set packetsize = packetsize or "100" %}
+scenarios:
+-
+ type: Ping
+ options:
+ packetsize: {{packetsize}}
+ host: athena.demo
+ target: ares.demo
+
+ runner:
+ type: Duration
+ duration: 60
+ interval: 1
+
+ sla:
+ max_rtt: 10
+ action: monitor
+
+context:
+ name: demo
+ image: cirros-0.3.3
+ flavor: m1.tiny
+ user: cirros
+
+ placement_groups:
+ pgrp1:
+ policy: "availability"
+
+ servers:
+ athena:
+ floating_ip: true
+ placement: "pgrp1"
+ ares:
+ placement: "pgrp1"
+
+ networks:
+ test:
+ cidr: '10.0.1.0/24'
+ external_network: "net04_ext"
+
url="https://www.opnfv.org",
install_requires=["backport_ipaddress", # remove with python3
"flake8",
+ "Jinja2>=2.6",
"PyYAML>=3.10",
"pbr<2.0,>=1.3",
"python-glanceclient>=0.12.0",
from yardstick.benchmark.context.model import Context
from yardstick.benchmark.runners import base as base_runner
-
+from yardstick.common.task_template import TaskTemplate
from yardstick.common.utils import cliargs
output_file_default = "/tmp/yardstick.out"
'''
@cliargs("taskfile", type=str, help="path to taskfile", nargs=1)
+ @cliargs("--task-args", dest="task_args",
+ help="Input task args (dict in json). These args are used"
+ "to render input task that is jinja2 template.")
+ @cliargs("--task-args-file", dest="task_args_file",
+ help="Path to the file with input task args (dict in "
+ "json/yaml). These args are used to render input"
+ "task that is jinja2 template.")
@cliargs("--keep-deploy", help="keep context deployed in cloud",
action="store_true")
@cliargs("--parse-only", help="parse the benchmark config file and exit",
atexit.register(atexit_handler)
parser = TaskParser(args.taskfile[0])
- scenarios, run_in_parallel = parser.parse()
+ scenarios, run_in_parallel = parser.parse(args.task_args,
+ args.task_args_file)
if args.parse_only:
sys.exit(0)
print "Done, exiting"
-
# TODO: Move stuff below into TaskCommands class !?
+
class TaskParser(object):
'''Parser for task config files in yaml format'''
def __init__(self, path):
self.path = path
- def parse(self):
+ def parse(self, task_args=None, task_args_file=None):
'''parses the task file and return an context and scenario instances'''
print "Parsing task config:", self.path
+
+ try:
+ kw = {}
+ if task_args_file:
+ with open(task_args_file) as f:
+ kw.update(parse_task_args("task_args_file", f.read()))
+ kw.update(parse_task_args("task_args", task_args))
+ except TypeError:
+ raise TypeError()
+
try:
- with open(self.path) as stream:
- cfg = yaml.load(stream)
+ with open(self.path) as f:
+ try:
+ input_task = f.read()
+ rendered_task = TaskTemplate.render(input_task, **kw)
+ except Exception as e:
+ print(("Failed to render template:\n%(task)s\n%(err)s\n")
+ % {"task": input_task, "err": e})
+ raise e
+ print(("Input task is:\n%s\n") % rendered_task)
+
+ cfg = yaml.load(rendered_task)
except IOError as ioerror:
sys.exit(ioerror)
base_runner.Runner.release(runner)
if status != 0:
sys.exit("Runner failed")
+
+
+def print_invalid_header(source_name, args):
+ print(("Invalid %(source)s passed:\n\n %(args)s\n")
+ % {"source": source_name, "args": args})
+
+
+def parse_task_args(src_name, args):
+ try:
+ kw = args and yaml.safe_load(args)
+ kw = {} if kw is None else kw
+ except yaml.parser.ParserError as e:
+ print_invalid_header(src_name, args)
+ print(("%(source)s has to be YAML. Details:\n\n%(err)s\n")
+ % {"source": src_name, "err": e})
+ raise TypeError()
+
+ if not isinstance(kw, dict):
+ print_invalid_header(src_name, args)
+ print(("%(src)s had to be dict, actually %(src_type)s\n")
+ % {"src": src_name, "src_type": type(kw)})
+ raise TypeError()
+ return kw
--- /dev/null
+##############################################################################
+# Copyright (c) 2015 Huawei Technologies Co.,Ltd and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+# yardstick: this file is copied from rally and slightly modified
+##############################################################################
+import re
+import jinja2
+import jinja2.meta
+
+
+class TaskTemplate(object):
+ @classmethod
+ def render(cls, task_template, **kwargs):
+ """Render jinja2 task template to Yardstick input task.
+
+ :param task_template: string that contains template
+ :param kwargs: Dict with template arguments
+ :returns:rendered template str
+ """
+
+ from six.moves import builtins
+
+ ast = jinja2.Environment().parse(task_template)
+ required_kwargs = jinja2.meta.find_undeclared_variables(ast)
+
+ missing = set(required_kwargs) - set(kwargs) - set(dir(builtins))
+ real_missing = [mis for mis in missing
+ if is_really_missing(mis, task_template)]
+
+ if real_missing:
+ multi_msg = ("Please specify next template task arguments:%s")
+ single_msg = ("Please specify template task argument:%s")
+ raise TypeError((len(real_missing) > 1 and multi_msg or single_msg)
+ % ", ".join(real_missing))
+ return jinja2.Template(task_template).render(**kwargs)
+
+
+def is_really_missing(mis, task_template):
+ # Removing variables that have default values from
+ # missing. Construction that won't be properly
+ # check is {% set x = x or 1}
+ if re.search(mis.join(["{%\s*set\s+", "\s*=\s*", "[^\w]+"]),
+ task_template):
+ return False
+ # Also check for a default filter which can show up as
+ # a missing variable
+ if re.search(mis + "\s*\|\s*default\(", task_template):
+ return False
+ return True
Code Review / yardstick.git / commitdiff