pharos.git
6 years agoIDF and PDF files for auto pod in unh lab 41/57741/11
pava [Tue, 22 May 2018 14:58:51 +0000 (16:58 +0200)]
IDF and PDF files for auto pod in unh lab

Change-Id: Iad65abda66b772e8ed44c2257ebdb7cb48ecc6c1
Signed-off-by: joekidder <Joe.Kidder@enea.com>
Signed-off-by: pava <Paul.Vaduva@enea.com>
6 years ago[PDF] Preparations for vPDF support 15/50315/9
Alexandru Avadanii [Thu, 28 Dec 2017 22:24:41 +0000 (23:24 +0100)]
[PDF] Preparations for vPDF support

- extend check-{jinja2,schema}.sh to scan vPDFs and construct
  proper IDF path for vPDFs;
- add support for '/' in PDF fields, by default filtered by j2
  template rendering;

SPEC changes:
- extend remote_management 'type' enum with 'libvirt';

SCHEMA changes:
- add new enum value 'libvirt' according to SPEC update;
- add separate handling of 'remote_management' for baremetal/virtual;
- remote_management 'versions' and 'mac_address' are mandatory for
  all but 'libvirt' (used to be mandatory for all);

Change-Id: Ib3f711cae807c3ab27920391402ba74844f155c3
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "Okinawa Open Labratory(OOL) POD1 PDF/IDF"
Julien [Mon, 30 Apr 2018 15:23:12 +0000 (15:23 +0000)]
Merge "Okinawa Open Labratory(OOL) POD1 PDF/IDF"

6 years agoupdate idf file for zte-pod1 37/56337/1
zhihui wu [Tue, 24 Apr 2018 09:48:41 +0000 (17:48 +0800)]
update idf file for zte-pod1

Change-Id: Iffed25c5e01aa9bf2a7b4a7103b07fb1c097d811
Signed-off-by: zhihui wu <wu.zhihui1@zte.com.cn>
6 years agoOkinawa Open Labratory(OOL) POD1 PDF/IDF 91/55791/13
hhayashiool [Mon, 16 Apr 2018 06:32:08 +0000 (15:32 +0900)]
Okinawa Open Labratory(OOL) POD1 PDF/IDF

Change-Id: I0980de9c2ff37654dbfd2d58002b9e61226cf749
Signed-off-by: Hideyasu Hayashi <hideyasu.hayashi@okinawaopenlabs.org>
6 years ago[fuel] Parameterize tenant VLAN ranges 01/56001/3
Alexandru Avadanii [Wed, 18 Apr 2018 16:37:29 +0000 (18:37 +0200)]
[fuel] Parameterize tenant VLAN ranges

- IDF: convert all tenant VLANs to ranges;
- IA: convert 'M-N' to 'M:N' for `opnfv_net_tenant_vlan`;

Change-Id: Ic9211235281c81ba082d10806651c35116ba4c35
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoJOID PDF-IDF support for intel-pod18 69/54569/12
agardner [Wed, 28 Mar 2018 20:06:23 +0000 (16:06 -0400)]
JOID PDF-IDF support for intel-pod18

More info in gerrit comments.

Remove joid from all other idfs other than intel/pod18
add installer: ['apex', 'compass4nfv', 'daisy', 'fuel'] where missing

Change-Id: I2afc415599070e210736c5fc304f7cb150f05029
Signed-off-by: agardner <agardner@linuxfoundation.org>
6 years ago[IDF] arm-pod6: Adjust CPU pinning variables 25/55425/1
Guillermo Herrero [Wed, 11 Apr 2018 15:42:50 +0000 (17:42 +0200)]
[IDF] arm-pod6: Adjust CPU pinning variables

For non-dpdk scenarios we define as cpu config:
  0-7   for OS
  8-47  for nova

For dpdk scenarios we define as cpu config:
  0-7   for OS
  8-11  for DPDK drivers and processes
  12-47 for nova

Change-Id: Ia9d7cfaee76f996993b94a52e1484752157a23cd
Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
6 years ago[IDF] zte-pod3: Remove vlan for tenant/external 97/54597/1
Alex Yang [Thu, 29 Mar 2018 07:19:25 +0000 (15:19 +0800)]
[IDF] zte-pod3: Remove vlan for tenant/external

To deploy ovs_dpdk scenario by daisy.
In this scenario, vlan configuration is not supported on
tenant/external interface for now.

Change-Id: I6e91bea92bde085079f4435fa72f20b5cef12ad3
Signed-off-by: Alex Yang <yangyang1@zte.com.cn>
6 years agoMerge "[IDF] arm-pod7: Configure hugepages and cpupinning variables for armpod7"
Alexandru Avadanii [Fri, 23 Mar 2018 20:28:54 +0000 (20:28 +0000)]
Merge "[IDF] arm-pod7: Configure hugepages and cpupinning variables for armpod7"

6 years ago[IDF] arm-pod7: Configure hugepages and cpupinning variables for armpod7 91/53791/7
Charalampos Kominos [Wed, 14 Mar 2018 12:56:17 +0000 (13:56 +0100)]
[IDF] arm-pod7: Configure hugepages and cpupinning variables for armpod7

For non-dpdk scenarios we define as cpu config:
        0,1 for OS
        2-7 for nova
For dpdk scenarios we define as cpu config:
        0,1 for OS,
        2,3 for DPDK drivers and processes
        4-7 for nova

Dpdk config is also provided for the 3 thunderx nodes which currently
serve as KVM nodes. The config is similar to the config above with
cores 4-47 being used for nova.

JIRA: ARMBAND-364

Change-Id: I761aa8e73935b7180f4f377841cc93c9d81cf4d7
Signed-off-by: Charalampos Kominos <charalampos.kominos@enea.com>
Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
6 years ago[IDF] arm-pod5: Configure hugepages and cpupinning variables for armpod5 01/53801/3
Charalampos Kominos [Wed, 14 Mar 2018 14:43:34 +0000 (15:43 +0100)]
[IDF] arm-pod5: Configure hugepages and cpupinning variables for armpod5

For non-dpdk scenarios we define as cpu config:
        0,1 for OS
        2-7 for nova
For dpdk scenarios we define as cpu config:
        0,1 for OS,
        2,3 for DPDK drivers and processes
        4-7 for nova

JIRA: ARMBAND-364

Change-Id: Icc976e8511fed221b71a348afda3cbe4e0556b1a
Signed-off-by: Charalampos Kominos <charalampos.kominos@enea.com>
6 years ago[IDF] arm, lf, ericsson: Align IP ranges start 31/53431/3
Alexandru Avadanii [Fri, 9 Mar 2018 19:34:55 +0000 (20:34 +0100)]
[IDF] arm, lf, ericsson: Align IP ranges start

- Enea lab has infra IPs in public networks, similar to ericsson-pod1
  so start assigning public IPs from .100 onwards;
- since most Fuel PODs have public (and mgmt) IPs starting at .100,
  align this across all PODs for uniformity;
- align all mgmt networks start to .10;

JIRA: ARMBAND-365

Change-Id: I9df4083c9e2e4c15da5ce6a2cb24df560808161a
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[IDF, fuel] Add idf.net_config.*.ip-range support 51/53251/2
Alexandru Avadanii [Wed, 7 Mar 2018 19:17:39 +0000 (20:17 +0100)]
[IDF, fuel] Add idf.net_config.*.ip-range support

- labs: ericsson-pod1: Restrict management and public network IP
  ranges to exclude pre-existing infrastructe addresses;
- fuel: Consume new ip-range network property in installer adapter;
  NOTE: For now, we only look at range start (and ignore end addr).

JIRA: FUEL-351

Change-Id: If81d0214f896fd42dfd0855ae4cecb06335d2e00
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoLF-POD4 PDF/IDF 59/51659/10
Blaisonneau David [Mon, 5 Feb 2018 10:56:51 +0000 (11:56 +0100)]
LF-POD4 PDF/IDF

PDF+IDF for LF POD4 inspired from securelab PDF and LF-pod2 XDFs.

Change-Id: If7ee14efe5f279104a18d2a3281aa7b23bdf704a

6 years agoMerge "config/installers: Adopt j2 'do' extension"
Alexandru Avadanii [Mon, 5 Mar 2018 17:50:07 +0000 (17:50 +0000)]
Merge "config/installers: Adopt j2 'do' extension"

6 years agogenerate_config: minor code formatting updates 05/53105/1
Alexandru Avadanii [Mon, 5 Mar 2018 16:58:01 +0000 (17:58 +0100)]
generate_config: minor code formatting updates

Change-Id: If0ee12287f4257f2de315bd6ad82edb709e24492
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "Add SDF files in Octopus"
Julien [Mon, 5 Mar 2018 03:27:39 +0000 (03:27 +0000)]
Merge "Add SDF files in Octopus"

6 years ago[IDF] zte-pod1: Remove br-mgmt 39/53039/1
Alexandru Avadanii [Sat, 3 Mar 2018 18:32:58 +0000 (19:32 +0100)]
[IDF] zte-pod1: Remove br-mgmt

All recent Fuel deploy attempts on zte-pod1 failed due to br-mgmt
not being present (or up), although it is defined in IDF.
Since this bridge is optional in the Fuel deployment process and
mostly useful for manual debugging, let's remove it from idf.fuel.

Change-Id: I130ec61427237f979f6a6212ca8c291ba203ef39
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoconfig/installers: Adopt j2 'do' extension 15/53015/2
Alexandru Avadanii [Fri, 2 Mar 2018 19:11:47 +0000 (20:11 +0100)]
config/installers: Adopt j2 'do' extension

Change-Id: Iffc473a4c90ff4ee8cfbcc304a3462559199fbe0
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[fuel] Fix missing space between yaml key:val 19/53019/2
Alexandru Avadanii [Fri, 2 Mar 2018 21:55:02 +0000 (22:55 +0100)]
[fuel] Fix missing space between yaml key:val

Change-Id: Icf0e5d386f593bb691f777704ac799c8729221ce
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agogenerate_config: New `-b` arg for batch processing 13/53013/3
Alexandru Avadanii [Fri, 2 Mar 2018 17:29:53 +0000 (18:29 +0100)]
generate_config: New `-b` arg for batch processing

Sometimes, parsing multiple j2 templates based on the same PDF+IDF
is needed, in which case we'd have to call generate_config once for
each template.

Add a new argument, `-b`, which allows batch processing of multiple
templates in one go:
- files ending in '.j2' (e.g. 'template.yml.j2') will be expanded to
  a file without the '.j2' suffix in the same directory as the
  source template (e.g. 'template.yml');
- templates not ending in '.j2' are skipped in batch mode;
- in order to pass multiple templates, `-j` argument can now be passed
  multiple times, similar to `-i`;
- although relative filepaths still work, it is highly recommended to
  use the full path (abspath) for j2 templates;
- to avoid filename collisions, j2 parser will open '/' (as well as
  any directories specified via `-i`) and lookup each j2 template
  by its full path;

Add a new argument, `-v` for verbose logging.

While at it, import only needed functions from 'os.path' to make our
code more compact and easier to read.

Change-Id: I2291935d42e4196813cd75154d5b674bcc330183
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "config/utils: YAML: Use C bindings if available"
Alexandru Avadanii [Fri, 2 Mar 2018 16:36:58 +0000 (16:36 +0000)]
Merge "config/utils: YAML: Use C bindings if available"

6 years agogenerate_config: New `-i` arg for j2 includes dir 61/52961/1
Alexandru Avadanii [Thu, 1 Mar 2018 22:47:31 +0000 (23:47 +0100)]
generate_config: New `-i` arg for j2 includes dir

jinja2 does not support imports from parent directories, so in order
for j2 to be able to use '{% from <some>.j2 import ... %}' constructs
with included j2 residing on a separate dirpath than current j2, add
support for explicitly adding directories to the templater search
path.

This also allows calling generate_config with only file basename as
'-j' arg value, as long as the dirname is passed via '-i' (can be
extended later to parsing multiple j2 files at once).

Change-Id: I943fda9262a6f7de5ee6fcc3e7bc9828c339651a
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agogenerate_config: Enable 'do' j2 extension 57/52957/1
Alexandru Avadanii [Thu, 1 Mar 2018 19:57:23 +0000 (20:57 +0100)]
generate_config: Enable 'do' j2 extension

'do' is useful to avoid hacks like {% set _ = x.update(...) %}.
Instead: {% do x.update(...) %}.

[1] http://jinja.pocoo.org/docs/2.10/extensions/

Change-Id: I886beb97e0383ba11b81aa21cf12b11ebc4b05b2
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoconfig/utils: YAML: Use C bindings if available 51/52951/3
Alexandru Avadanii [Thu, 1 Mar 2018 18:57:34 +0000 (19:57 +0100)]
config/utils: YAML: Use C bindings if available

Time diff before/after this change for the same template via
generate_config expansion:
-user    0m0.144s
+user    0m0.096s

Change-Id: Id574afcd8a74a530d791ebed8b72ccae53703611
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "[fuel] Improved IP handling on config template"
Alexandru Avadanii [Tue, 27 Feb 2018 13:42:47 +0000 (13:42 +0000)]
Merge "[fuel] Improved IP handling on config template"

6 years ago[IDF] arm-pod6: Use 2M hugepages, 12 cores for nova 93/52693/1
Alexandru Avadanii [Tue, 27 Feb 2018 02:52:19 +0000 (03:52 +0100)]
[IDF] arm-pod6: Use 2M hugepages, 12 cores for nova

Change-Id: I03098ca3d4a90f6420cee64985c2853931a6a870
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[IDF] arm-pod6: Fix nova_cpu_pinning param expr 79/52679/1
Alexandru Avadanii [Mon, 26 Feb 2018 18:07:09 +0000 (19:07 +0100)]
[IDF] arm-pod6: Fix nova_cpu_pinning param expr

Constructs based on reclass interpolation (e.g. '${_param:x}') do
not work when parameters are passed via reclass.storage templating,
so change reclass interpolation syntax with classic YAML anchors.

Change-Id: Ia974d45fc6a5869f4b30b31b947df088ea3aaf0c
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[fuel] Improved IP handling on config template 99/51499/22
Guillermo Herrero [Fri, 2 Feb 2018 15:35:21 +0000 (16:35 +0100)]
[fuel] Improved IP handling on config template

-New IP addressing.
Starts on lower available addresses for
better compatibilty for small networks.
IPs assigned secuentially and independently
on each network.

-Switched to advanced IP generation filters.
Defined integers are no longer end of the IP address part
but host order on the IP network.

-Dynamic generation of compute nodes

JIRA:FUEL-335

Change-Id: Ie95b83af64cb21ba27749d5f5d1506bfbc9ddf15
Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
6 years ago[fuel] IDF: Add DPDK parameters support 53/52453/6
Cristina Pauna [Tue, 20 Feb 2018 16:29:42 +0000 (18:29 +0200)]
[fuel] IDF: Add DPDK parameters support

In order to enable DPDK for OVS scenarios in Armband, we need DPDK
specific configuration to be parameterizable.
The default DPDK configuration values will remain in Fuel repo and
will be overriden with the values defined in IDF (only if present).
Since hugepage config and CPU pinning also apply to non-DPDK
scenarios, handle them together for 'common', respectively 'dpdk'.

To keep things simple, we will reuse the format expected by Fuel's
reclass model, so we can pass the whole config block as-is.

- IDF schema: fuel: Add strict checking of new block's structure,
  while allowing the rvalues to be more or less freestyle;
- arm-pod6 IDF: Add initial config block implementation;

JIRA: ARMBAND-342

Change-Id: I9224f06bb6b9fb2387414eb2a1be35461c27fb99
Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
6 years agogen_config_lib: Add storage_size_num filter 35/52435/1
Alexandru Avadanii [Thu, 22 Feb 2018 00:31:50 +0000 (01:31 +0100)]
gen_config_lib: Add storage_size_num filter

New filter for converting PDF disk_capacity strings into a numeric
only string (to bypass scientific notation), parsable as float.

Change-Id: I779bec7db03ef1b3ce1bfd6ca239a3e7b4021eb2
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agogenerate_config: Move filters to external library 03/51403/12
Guillermo Herrero [Wed, 31 Jan 2018 15:39:02 +0000 (16:39 +0100)]
generate_config: Move filters to external library

Clean and isolate main script logic from custom filters by moving
all them to a external library file.
Library has a function to load all the existing filters at once.

New enhanced IP handling custom filters:
 - ipnet_hostaddr
 - ipnet_hostmin
 - ipnet_hostmax
 - ipnet_broadcast
 - ipnet_netmask
 - ipnet_contains_ip
 - ipnet_contains_iprange
 - ipnet_range_size

ipnet filters work with proper IP network with prefixlen mask,
ensuring consistent IP calculations and error handling.

Previous IP handling filters should be deprecated.

Change-Id: I83c41d7ad3c6bd1d9df1deca6cc5b9d2481ecf52
Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
6 years agoConfigure stacklight IP addresses 29/52229/2
Delia Popescu [Fri, 16 Feb 2018 11:56:24 +0000 (13:56 +0200)]
Configure stacklight IP addresses

Related to: https://gerrit.opnfv.org/gerrit/#/c/52043/

Change-Id: I8378070f9efedc24229efc3c3d5542b62808e1e6
Signed-off-by: Delia Popescu <delia.popescu@enea.com>
6 years agoMerge "[PDF/IDF] LF-POD4"
Trevor Bramwell [Thu, 15 Feb 2018 00:54:39 +0000 (00:54 +0000)]
Merge "[PDF/IDF] LF-POD4"

6 years ago[IDF] LF-POD5 69/52169/1
Trevor Bramwell [Wed, 14 Feb 2018 20:28:52 +0000 (12:28 -0800)]
[IDF] LF-POD5

Update networks for LF-POD5 since discovering there is a bug using
non-/24 networks with Fuel, and to ensure there is no overlap with
LF-POD4.

Change-Id: Ie6eb6f12e8fd2831355e8bcec064e152cdc02f27
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
6 years ago[PDF/IDF] LF-POD4 71/52171/1
Trevor Bramwell [Wed, 14 Feb 2018 20:27:26 +0000 (12:27 -0800)]
[PDF/IDF] LF-POD4

Add PDF and IDF for LF-POD4

Change-Id: Ie3e682e40f751b8f91340373fbcabd522d3686a8
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
6 years agoMerge "Update PTL to Julien Zhang in INFO file"
Alexandru Avadanii [Tue, 13 Feb 2018 15:07:37 +0000 (15:07 +0000)]
Merge "Update PTL to Julien Zhang in INFO file"

6 years agoMerge "Copy scenario-lifecycle docs into pharos"
Julien [Mon, 12 Feb 2018 10:46:39 +0000 (10:46 +0000)]
Merge "Copy scenario-lifecycle docs into pharos"

6 years agoUpdate PTL to Julien Zhang in INFO file 15/51915/1
Trevor Bramwell [Thu, 8 Feb 2018 18:10:44 +0000 (10:10 -0800)]
Update PTL to Julien Zhang in INFO file

Includes a link to the TSC notice of PTL change.

Change-Id: Ib5ea9abdc9bcfec29c3cb1ab02906221436969f1
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
6 years ago[IDF] Schema: Add OSA installer stubs 93/51693/1 6.0.0
Alexandru Avadanii [Mon, 5 Feb 2018 18:38:16 +0000 (19:38 +0100)]
[IDF] Schema: Add OSA installer stubs

While at it, order installers by name in IDF schema.

Change-Id: I4bf1dd047a3f7a64b8bc928951fc18c4728a9b68
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoCopy scenario-lifecycle docs into pharos 65/51665/1
Julien [Mon, 5 Feb 2018 12:12:43 +0000 (20:12 +0800)]
Copy scenario-lifecycle docs into pharos

Copy project scenario-lifecycle from Octopus and keep the original
format.

Change-Id: I312b81b88fa7e69cf4b8c23b50f941aab8fba9bd
Signed-off-by: Julien <zhang.jun3g@zte.com.cn>
6 years agoAdd SDF files in Octopus 49/51649/1
Julien [Mon, 5 Feb 2018 07:31:45 +0000 (15:31 +0800)]
Add SDF files in Octopus

Add SDF files in Octopus to scenarios sub-directories. In the future,
we will add schema files for SDF just like PDF/IDF.

Change-Id: I248834bc7fe91bfbd8afe4827905f6ebd4f7a5ab
Signed-off-by: Julien <zhang.jun3g@zte.com.cn>
6 years ago[fuel] pod_config: Consume newly added MaaS props 69/51469/2
Alexandru Avadanii [Thu, 1 Feb 2018 17:45:39 +0000 (18:45 +0100)]
[fuel] pod_config: Consume newly added MaaS props

Change-Id: I23446e82dd148afd14d56717dd35fe0ee271d971
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[fuel] Allow MaaS timeout config via IDF 67/51467/1
Alexandru Avadanii [Thu, 1 Feb 2018 16:27:19 +0000 (17:27 +0100)]
[fuel] Allow MaaS timeout config via IDF

MaaS timeouts should be configurable on per-POD basis, since nodes
in some labs take longer to comission/deploy than our default values.

IDF SCHEMA:
- add new properties to Fuel IDF section;

Change-Id: Ife5ff85faeae683a8c1781cae8b29f09c1f07b82
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[IDF] lf-pod2: Add missing bridge, eth info 63/51463/1
Alexandru Avadanii [Thu, 1 Feb 2018 15:39:29 +0000 (16:39 +0100)]
[IDF] lf-pod2: Add missing bridge, eth info

Some information was lost while syncing from securedlab to pharos
git repo, bring it back.

Change-Id: I938f4c573234c7a9d55eb140ea02f3328373ae1d
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge changes from topics 'fuel-netmap', 'fix-check-jinja'
Alexandru Avadanii [Tue, 30 Jan 2018 22:32:46 +0000 (22:32 +0000)]
Merge changes from topics 'fuel-netmap', 'fix-check-jinja'

* changes:
  [fuel] Add net_map.j2 for net config abstraction
  [fuel] Explicitly blacklist incompatible labs
  [IDF] Add optional 'idf.installer' array

6 years agoMerge "[fuel] Update IA for vPDF support"
Alexandru Avadanii [Mon, 29 Jan 2018 16:17:01 +0000 (16:17 +0000)]
Merge "[fuel] Update IA for vPDF support"

6 years ago[fuel] Add net_map.j2 for net config abstraction 67/51267/1
Alexandru Avadanii [Sun, 28 Jan 2018 16:36:50 +0000 (17:36 +0100)]
[fuel] Add net_map.j2 for net config abstraction

This will be leveraged by other j2 templates in Fuel to construct
scenario-specific network config files.

Change-Id: Iba1fd99b23936fe42abe16af3cd80dce2a63d997
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[fuel] Explicitly blacklist incompatible labs 61/51261/1
Alexandru Avadanii [Sun, 28 Jan 2018 18:34:31 +0000 (19:34 +0100)]
[fuel] Explicitly blacklist incompatible labs

Previously, we used to silently refuse to construct 'pod_config'
for PODs missing mandatory PDF/IDF configuration; and instead
generated an empty output YAML.

Retire safety checks in Fuel IA and explicitly blacklist Fuel via
new 'idf.installer' where appropiate.

The following PODs do not yet support Fuel due to missing 'idf.fuel':
- ericsson-pod2
- huawei-pod1
- intel-pod18
- nokia-pod1
- zte-pod2
- zte-pod3

Change-Id: Ief62d62f99a79187303e3f15b83030289e5efdcb
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[IDF] Add optional 'idf.installer' array 75/51075/2
Alexandru Avadanii [Wed, 24 Jan 2018 20:02:40 +0000 (21:02 +0100)]
[IDF] Add optional 'idf.installer' array

There is no easy way to determine automatically if a PDF/IDF set is
compatible with a specific installer adapter.
Most common incompatibilities are caused by:
- installer adapters hardcode more network ports than nodes have
  defined in PDF;
- installer adapters hardcode more nodes than PDF defines;

Previous attempt at handling this dynamically (cd54edf) fails for
recently added PODs, so let's handle this explicitly per-pod via a
new optional IDF property, 'idf.installer'.

If 'idf.installer' is defined, it will be used by `check-jinja2.sh`
as a filter-in list of installer adapters to check against.
The default (no 'idf.installer' present) is to check all IAs.

Leverage the new property for:
- arm-pod5: not enough network ports on computes for Compass, Joid;
- arm-pod7: same as arm-pod5;
- bii-pod1: no 'storage' network defined in IDF for Daisy;
- zte-pod9: 3 cluster nodes are only supported by Daisy for now;

SPEC changes:
- net_config (previously mandatory for IDF version '0.1') is now
  mandatory only if 'idf.fuel' or 'idf.daisy' is defined;

Fixes: cd54edf

Change-Id: I706c05519e5491ad631069d7cc090e9c8bfd3011
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[fuel] Update IA for vPDF support 47/51247/2
Alexandru Avadanii [Sat, 27 Jan 2018 18:55:31 +0000 (19:55 +0100)]
[fuel] Update IA for vPDF support

- move gtw tenant addrs from {6,7,9} to {124,125,126}
  (e.g. 10.0.1.6 -> 10.0.1.124 for ericsson-virtual* PODs);
- add gtw external (public) addresses, e.g. 10.16.0.124;
- add ctl external (public) addresses, e.g. 10.16.0.11;

JIRA: FUEL-322

Change-Id: Ice73aa3985831dca7427e496850d0865db71477e
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[daisy] network template: Fix set inside loop 77/51077/3
Alexandru Avadanii [Wed, 24 Jan 2018 21:47:52 +0000 (22:47 +0100)]
[daisy] network template: Fix set inside loop

Previous implementation fails in older versions of py-jinja2.
See related bug reports [1, 2].
Since `networks.update` on its own is enough, stop assigning its
result back to 'networks'.

[1] https://github.com/pallets/jinja/issues/641
[2] https://github.com/pallets/jinja/issues/164

Change-Id: I303388223bdec9ae43e49d988aee41b5b47c1562
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "[PDF] Update LF-POD5 IDF bridges, interfaces"
Alexandru Avadanii [Sun, 28 Jan 2018 17:53:25 +0000 (17:53 +0000)]
Merge "[PDF] Update LF-POD5 IDF bridges, interfaces"

6 years agoMerge "[joid] Refresh installer adapter for Pike"
Alexandru Avadanii [Sun, 28 Jan 2018 17:53:11 +0000 (17:53 +0000)]
Merge "[joid] Refresh installer adapter for Pike"

6 years ago[PDF] Update LF-POD5 IDF bridges, interfaces 65/50265/3
Trevor Bramwell [Mon, 8 Jan 2018 22:11:53 +0000 (14:11 -0800)]
[PDF] Update LF-POD5 IDF bridges, interfaces

Change-Id: Ia761d6cecc360dde729965e499a899ba4fbc6a32
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
6 years agoMerge "[IDF] zte-pod1: Fix admin bridge"
Alexandru Avadanii [Sun, 28 Jan 2018 17:46:17 +0000 (17:46 +0000)]
Merge "[IDF] zte-pod1: Fix admin bridge"

6 years agoMerge "[daisy] Use jump remote_management params"
Alexandru Avadanii [Sun, 28 Jan 2018 17:44:50 +0000 (17:44 +0000)]
Merge "[daisy] Use jump remote_management params"

6 years agoMerge "[compass4nfv] Use jump remote_management params"
Alexandru Avadanii [Sun, 28 Jan 2018 17:44:26 +0000 (17:44 +0000)]
Merge "[compass4nfv] Use jump remote_management params"

6 years agoMerge "[joid] Use node remote_management params"
Alexandru Avadanii [Sun, 28 Jan 2018 17:43:32 +0000 (17:43 +0000)]
Merge "[joid] Use node remote_management params"

6 years agogenerate_config: Check yaml arg is really a file 21/51221/1
Alexandru Avadanii [Fri, 26 Jan 2018 20:32:07 +0000 (21:32 +0100)]
generate_config: Check yaml arg is really a file

In order to support bash process substition, e.g.
$ ./generate_config.py -j /path/to/template.j2 -y <(echo stuff);
do not consume the contents of ARGS.yaml unless it is a file.

Change-Id: I89053cfaadc158f6a15f85836892b21980081772
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[IDF] zte-pod1: Fix admin bridge 59/51159/1
Alexandru Avadanii [Fri, 26 Jan 2018 01:24:41 +0000 (02:24 +0100)]
[IDF] zte-pod1: Fix admin bridge

Align PXE/admin bridge name on jumpserver with value in releng
slave param BRIDGE for zte-pod1.

Change-Id: I57e1079bc047046cb85afe6d7f7abcc340217235
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[compass4nfv] Use jump remote_management params 85/51085/2
Alexandru Avadanii [Wed, 24 Jan 2018 22:44:36 +0000 (23:44 +0100)]
[compass4nfv] Use jump remote_management params

Stop relying on 'remote_params' optional YAML anchor.

Change-Id: I8d967b767cf66b298a651306c9bcc91cc9acb0a6
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[joid] Refresh installer adapter for Pike 43/51143/2
Alexandru Avadanii [Thu, 25 Jan 2018 18:45:02 +0000 (19:45 +0100)]
[joid] Refresh installer adapter for Pike

Change-Id: I4e4960a772fcd5ae32e34c9bf375ed78e3a30311
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[joid] Use node remote_management params 83/51083/2
Alexandru Avadanii [Wed, 24 Jan 2018 22:39:38 +0000 (23:39 +0100)]
[joid] Use node remote_management params

Previously, we relied on jumpserver IPMI user/pass/type to be the
same for all nodes (including jumpserver).
Instead, read node-specific params and stop relying on
'remote_params' optional YAML anchor.

Change-Id: Ia8925261bc8242cbfd195f1d09bb61121b3fdacf
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[daisy] Use jump remote_management params 87/51087/2
Alexandru Avadanii [Wed, 24 Jan 2018 22:50:33 +0000 (23:50 +0100)]
[daisy] Use jump remote_management params

Stop relying on 'remote_params' optional YAML anchor.

Change-Id: Ie5c563834cee19d676cc635c622ebbe8ffc0cc83
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "[IDF] Add IDF schema validation based on version"
Aric Gardner [Wed, 24 Jan 2018 16:36:42 +0000 (16:36 +0000)]
Merge "[IDF] Add IDF schema validation based on version"

6 years agoMerge "[fuel] Get public pool parameters from idf.fuel"
Alexandru Avadanii [Wed, 24 Jan 2018 16:26:05 +0000 (16:26 +0000)]
Merge "[fuel] Get public pool parameters from idf.fuel"

6 years agoMerge "update user/passwd"
Aric Gardner [Wed, 24 Jan 2018 16:11:35 +0000 (16:11 +0000)]
Merge "update user/passwd"

6 years agoMerge "nokia-pod1: Add PDF version"
Alexandru Avadanii [Wed, 24 Jan 2018 15:24:08 +0000 (15:24 +0000)]
Merge "nokia-pod1: Add PDF version"

6 years ago[fuel] Get public pool parameters from idf.fuel 29/50929/5
Guillermo Herrero [Mon, 22 Jan 2018 13:19:20 +0000 (14:19 +0100)]
[fuel] Get public pool parameters from idf.fuel

Reads parameters from idf.fuel.config.public_pool
start_ip and end_ip

 JIRA: FUEL-315

Change-Id: I4456b22bc21b20906f18953d80a57483fd6700bc
Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
6 years agoupdate user/passwd 41/51041/1
mting [Wed, 24 Jan 2018 10:27:04 +0000 (11:27 +0100)]
update user/passwd

Change-Id: I9675e419bdda676cb2d4fa6ae49d0c4198006d33

6 years ago[IDF] Add IDF schema validation based on version 83/50583/7
Alexandru Avadanii [Sun, 14 Jan 2018 01:26:05 +0000 (02:26 +0100)]
[IDF] Add IDF schema validation based on version

- add basic structure for IDF schema validation;
- fill in Fuel section;
- add stub for Daisy section;
- add stub for Compass section;
- add stub for net_config;
- enable IDF schema validation via `check-schema.sh`;
- prepare for new IDF versions with different schemas,
  while allowing subsection schema reuse from IDF v0.1;
- extend validate_schema.py for IDF;

Change-Id: I59b1c9d857e9dee9e6ff9956e81b2ae6dc27a767
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agonokia-pod1: Add PDF version 15/51015/1
Alexandru Avadanii [Tue, 23 Jan 2018 21:10:21 +0000 (22:10 +0100)]
nokia-pod1: Add PDF version

Fix PDF schema validation by adding mandatory version param.

Change-Id: Ic86d182ce8e4a290e9bd3a0ae3dc1823ab27c8e8
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agointel-pod18: Fix typo in MAC address 13/51013/1
Alexandru Avadanii [Tue, 23 Jan 2018 21:05:02 +0000 (22:05 +0100)]
intel-pod18: Fix typo in MAC address

While at it, add dummy user/pass IPMI credentials, so we can pass
schema validation.

Change-Id: I3538f5aee21731e46fe0d831d164961c33118e2e
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "[PDF] [SPEC] Add 'version: 1.0'"
julien zhang [Tue, 23 Jan 2018 16:03:37 +0000 (16:03 +0000)]
Merge "[PDF] [SPEC] Add 'version: 1.0'"

6 years ago[fuel] Fix missing EOL after yaml doc start 49/50949/1
Alexandru Avadanii [Mon, 22 Jan 2018 17:41:02 +0000 (18:41 +0100)]
[fuel] Fix missing EOL after yaml doc start

Fixes: 4cef6b2

Change-Id: I628ce8da7e0959ff916082c1fc30934dca0d93dc
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[PDF] [SPEC] Add 'version: 1.0' 61/50661/3
Alexandru Avadanii [Mon, 15 Jan 2018 03:32:31 +0000 (04:32 +0100)]
[PDF] [SPEC] Add 'version: 1.0'

- add 'version: 1.0' to PDF spec;
- add 'version: 1.0' to all existing lab PDFs;
- extend schema with new property;
- add workaround for value-based decision-making in schema version
  selection via `validate-template.py`;
- add support for multiple schema versions;
- add versions for all schema blocks defined so far;
- fix PDF schema pattern for disk size decimals (e.g. '1.8T');

Change-Id: Ie8f768803ec19f1f9a7982fe5ca59df80764fc4a
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "[PDF] check-jinja: Filter-out incompatible PDF/IA"
Alexandru Avadanii [Mon, 22 Jan 2018 16:50:10 +0000 (16:50 +0000)]
Merge "[PDF] check-jinja: Filter-out incompatible PDF/IA"

6 years agoMerge "[fuel] Add support to use public net netmask from IDF"
Alexandru Avadanii [Mon, 22 Jan 2018 16:49:57 +0000 (16:49 +0000)]
Merge "[fuel] Add support to use public net netmask from IDF"

6 years agoMerge "generate_config: Add netmask jinja filter"
Alexandru Avadanii [Mon, 22 Jan 2018 16:49:34 +0000 (16:49 +0000)]
Merge "generate_config: Add netmask jinja filter"

6 years agoMerge "update BII pod1 PDF for Euphrates install"
Alexandru Avadanii [Mon, 22 Jan 2018 16:49:07 +0000 (16:49 +0000)]
Merge "update BII pod1 PDF for Euphrates install"

6 years agoMerge "[apex] Use node remote_management params"
Aric Gardner [Mon, 22 Jan 2018 16:47:14 +0000 (16:47 +0000)]
Merge "[apex] Use node remote_management params"

6 years agoMerge "check-jinja: Add optional filtering of test matrix"
Alexandru Avadanii [Mon, 22 Jan 2018 16:32:40 +0000 (16:32 +0000)]
Merge "check-jinja: Add optional filtering of test matrix"

6 years agoMerge "Nokia public lab Pod Descriptor File"
Alexandru Avadanii [Mon, 22 Jan 2018 16:28:38 +0000 (16:28 +0000)]
Merge "Nokia public lab Pod Descriptor File"

6 years agoMerge "[PDF] config: Add/update license headers"
Aric Gardner [Mon, 22 Jan 2018 16:13:11 +0000 (16:13 +0000)]
Merge "[PDF] config: Add/update license headers"

6 years agoMerge "arm-pod7: add new PDF and IDF files"
Alexandru Avadanii [Mon, 22 Jan 2018 16:10:29 +0000 (16:10 +0000)]
Merge "arm-pod7: add new PDF and IDF files"

6 years agoupdate BII pod1 PDF for Euphrates install 49/50749/2
louie.long [Thu, 18 Jan 2018 03:46:21 +0000 (11:46 +0800)]
update BII pod1 PDF for Euphrates install

  - add licenses in all PDF file
  - add jumphost bridge in idf-pod1.yaml
  - add network info in idf-pod1.yaml
  - use eyaml decrypt IPMI user/passwd
  - update node network&node name in pod1.yaml

Change-Id: Ia239d5d1ac261cc8b6e35dee6af8292b60884f0b
Signed-off-by: louie.long <longyu805@163.com>
6 years ago[fuel] Add proxy node management network VIP 67/50867/1
Alexandru Avadanii [Sat, 20 Jan 2018 20:52:56 +0000 (21:52 +0100)]
[fuel] Add proxy node management network VIP

Change-Id: I5fd74163cab8f6ee5f575ed72edfcc6919e60161
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[fuel] Update MaaS PXE config 65/50865/1
Alexandru Avadanii [Sat, 20 Jan 2018 20:50:48 +0000 (21:50 +0100)]
[fuel] Update MaaS PXE config

- reduce DHCP range to silence a dummy MaaS warning about address exhaustion;
- define PXE/admin address for Salt Master node;
- drop obsolete opnfv_infra_maas_pxe_address;

JIRA: FUEL-316

Change-Id: Ic946960aac3c09dbcfce005098a5fc97396da01e
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agocheck-jinja: Add optional filtering of test matrix 55/50855/1
Alexandru Avadanii [Fri, 19 Jan 2018 18:54:42 +0000 (19:54 +0100)]
check-jinja: Add optional filtering of test matrix

When adding new PDFs, it is sometimes useful to check only the newly
added PDF (or all the PDFs for a specific lab) against all or only
some installer templates.

Change-Id: I1d02943527597df1e1908a3ff4398bc398c887e7
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoarm-pod7: add new PDF and IDF files 69/50769/4
ting wu [Thu, 18 Jan 2018 12:27:28 +0000 (13:27 +0100)]
arm-pod7: add new PDF and IDF files

- net_config is moved to IDF file;
- rename 'disk' to 'disks';
- add 'name: nicN' to interface;
- add 'disk_rotation: 0' for ssd drivers;

Change-Id: I1ea91a65abeb9db4f2f73f25ce51cd059b4b4753
Signed-off-by: ting wu <ting.wu@enea.com>
6 years ago[fuel] Add support to use public net netmask from IDF 97/50697/6
Guillermo Herrero [Tue, 16 Jan 2018 11:59:23 +0000 (12:59 +0100)]
[fuel] Add support to use public net netmask from IDF

- New reclass params for public network and public network mask
- Network mask is converted from prefix format to reclass compatible IP format
- New reclass params to set public pool ip range

JIRA: FUEL-315

Change-Id: I30c5332b958796af5a10cd39c56ed5cf5800e029
Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
6 years ago[apex] Use node remote_management params 07/50807/1
Alexandru Avadanii [Fri, 19 Jan 2018 01:58:04 +0000 (02:58 +0100)]
[apex] Use node remote_management params

Previously, we relied on jumpserver IPMI user/pass/type to be the
same for all nodes (including jumpserver).
Instead, read node-specific params and stop relying on
'remote_params' optional YAML anchor.

Change-Id: Id64a0c4350481a6c81212a4559115bb9b1a0155e
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agogenerate_config: Add netmask jinja filter 89/50689/5
Guillermo Herrero [Tue, 16 Jan 2018 11:03:19 +0000 (12:03 +0100)]
generate_config: Add netmask jinja filter

- Add custom filter to convert integer prefix network
  mask to IP format network mask string

Change-Id: I13d6a2725b93132f858b64d7f93de3a4e0dc5961
Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
6 years agoNokia public lab Pod Descriptor File 73/50673/6
Mika Rautakumpu [Tue, 16 Jan 2018 08:47:20 +0000 (10:47 +0200)]
Nokia public lab Pod Descriptor File

Comment lines removed, typos fixed and missing fields added.

Change-Id: Ic027544b2e7e70fb7c02452945fb2a1252417ce3
Signed-off-by: Mika Rautakumpu <mika.rautakumpu@nokia.com>
6 years agoMerge "Committers standing down"
Jack Morgan [Tue, 16 Jan 2018 23:43:36 +0000 (23:43 +0000)]
Merge "Committers standing down"

6 years agoFix daisy gateway generated by PDF 79/50679/1
Alex Yang [Tue, 16 Jan 2018 10:03:59 +0000 (18:03 +0800)]
Fix daisy gateway generated by PDF

Fixed_ips were removed from the PDF. So we need get gateway address
from the 'interfaces'.

Change-Id: Ie6adbf4ef3cbde05bb2fb114a58395dda8ae50fb
Signed-off-by: Alex Yang <yangyang1@zte.com.cn>
6 years ago[PDF] check-jinja: Filter-out incompatible PDF/IA 07/50607/3
Alexandru Avadanii [Mon, 15 Jan 2018 00:33:02 +0000 (01:33 +0100)]
[PDF] check-jinja: Filter-out incompatible PDF/IA

While checking all PDFs against all installers, some mandatory
requirements are sometimes not met:
- minimum cluster nodes (e.g. Apex requires 5 nodes, 1 PDF has 3);
- minimum interface count (e.g. Compass requires min 3 NICs, some
  PDFs only have two interfaces or less);

The added filter is far from ideal:
- node requirements are based on the latest (not highest) index
  used by the installer adapter to accomodate special handling of
  3-node cluster in Daisy templates;
- interface req is based on the average interfaces/node, with
  quite some margin - this is a very loose mechanism;

With this in, `check-jinja` has a return code of 0 (note that
yamllint failures of output files are not affecting the return code).

Change-Id: I43ade3567bf3026069ff93eca17abc212fab211c
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge changes from topics 'pdf-spec-update', 'pdf-schema'
Aric Gardner [Mon, 15 Jan 2018 22:03:37 +0000 (22:03 +0000)]
Merge changes from topics 'pdf-spec-update', 'pdf-schema'

* changes:
  [PDF] [SPEC] disk_interface: Add 'scsi', 'iscsi'
  [PDF] [SPEC] Align interface keys for NIC names
  [PDF] [SPEC] Extend disk_rotation enum with '0'
  [PDF] Add schema validation