fuel.git
6 years ago[odl] Configure Odl memory requirements 55/54555/3
Charalampos Kominos [Wed, 28 Mar 2018 12:05:09 +0000 (14:05 +0200)]
[odl] Configure Odl memory requirements

Odl scripts present in /opt/opendaylight crash with
out-of-memory errors and we cannot use them so we add more memory
to the ODL virtual machine and to the JVM.

Change-Id: Ia29a324ca34b5d33e947ca6d177ce3567316dcb9
Signed-off-by: Charalampos Kominos <charalampos.kominos@enea.com>
6 years agoMerge "[doc] installation guide update: PDF section"
Alexandru Avadanii [Tue, 27 Mar 2018 15:39:59 +0000 (15:39 +0000)]
Merge "[doc] installation guide update: PDF section"

6 years ago[doc] installation guide update: PDF section 87/53787/6
ting wu [Wed, 14 Mar 2018 11:54:47 +0000 (12:54 +0100)]
[doc] installation guide update: PDF section

Update PDF section to:
1. describe "net_config" section in IDF.
2. describe compute parameterizaion required in IDF for
   both DPDK and non-DPDK scenario.

JIRA: ARMBAND-361

Change-Id: I63fbe79c5cc3a4ee5244e6a0efad57eb89aad7d2
Signed-off-by: ting wu <ting.wu@enea.com>
6 years agoApply opendaylight state after ovs host config 97/54397/1
Michael Polenchuk [Fri, 23 Mar 2018 12:47:46 +0000 (16:47 +0400)]
Apply opendaylight state after ovs host config

* employ GA kernel for baremetal computes as well
* setup/start opendaylight server after ovs host config

Change-Id: Ic772aed544b17be02e6ca9ccd175f2288b2128a8
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years ago[ovs/dpdk] Turn on non-blocking ovs calls 23/54323/1
Michael Polenchuk [Thu, 22 Mar 2018 14:03:06 +0000 (18:03 +0400)]
[ovs/dpdk] Turn on non-blocking ovs calls

Change-Id: I8206707cdb2ebc267b6f4971de35477e3dc73c5f
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agoMerge "lib.sh: Skip modprobing nbd, loop if inbuilt"
Alexandru Avadanii [Thu, 22 Mar 2018 12:02:16 +0000 (12:02 +0000)]
Merge "lib.sh: Skip modprobing nbd, loop if inbuilt"

6 years ago[patch] Drop keystoneclient project ID checking 41/54241/1
Alexandru Avadanii [Wed, 21 Mar 2018 05:15:49 +0000 (06:15 +0100)]
[patch] Drop keystoneclient project ID checking

Since we brought back NAT over MaaS PXE/admin, drop out-of-tree patch
that was only needed for full network isolation (no cross-talk
between public and admin).

Change-Id: I577b1116bbdcc19647b3c01fe6f2acb7bb9ce627
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[virtual/odl] Use default gw offered from dhcp 01/54201/1
Michael Polenchuk [Tue, 20 Mar 2018 13:41:35 +0000 (17:41 +0400)]
[virtual/odl] Use default gw offered from dhcp

Remove default route to external gateway on compute nodes
instead of placing wrapper script as a workaround.

Change-Id: I8e1d5d9920a7aa2ed2d1092ebce692176a3bf41a
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agolib.sh: Skip modprobing nbd, loop if inbuilt 59/54059/5
Alexandru Avadanii [Sun, 18 Mar 2018 21:27:11 +0000 (22:27 +0100)]
lib.sh: Skip modprobing nbd, loop if inbuilt

While at it, skip waiting for Salt master when deleting UEFI stale
entries if it doesn't respond to ping.

Also, use https for fetching Armband GPG key to bypass yet another
hks issue behind proxies/firewalls that block hks port.

Change-Id: I400cbe3257094b62c96b302a3c81c5ffd1ba4755
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[ovs/dpdk] Set bridge datapath_type to netdev 83/53983/1
Michael Polenchuk [Fri, 16 Mar 2018 11:57:52 +0000 (15:57 +0400)]
[ovs/dpdk] Set bridge datapath_type to netdev

This prevents neutron ovs agent to re-create existing bridge which
might cause unpredictable/faulty state of network interfaces.

Change-Id: I289365e1dea7d178b5b72eb506f5c711f6d60199
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agoMerge "salt.sh: Copy mcp.rsa to cfg01"
Alexandru Avadanii [Thu, 15 Mar 2018 00:00:04 +0000 (00:00 +0000)]
Merge "salt.sh: Copy mcp.rsa to cfg01"

6 years ago[virtual/odl] Simplify network scheme on computes 83/53783/1
Michael Polenchuk [Wed, 14 Mar 2018 11:45:23 +0000 (15:45 +0400)]
[virtual/odl] Simplify network scheme on computes

Change-Id: Ifb874aba38b2fa36fde05184bcdd74870257bec1
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agosalt.sh: Copy mcp.rsa to cfg01 25/53725/1
Alexandru Avadanii [Tue, 13 Mar 2018 20:31:53 +0000 (21:31 +0100)]
salt.sh: Copy mcp.rsa to cfg01

- let rsync transfer mcp.rsa* to cfg01;
- symlink mcp.rsa in /root;

Change-Id: I18cb70306faf9d166e1110e374a9fe16ac4449de
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[odl] Add SDN_CONTROLLER_IP to openrc 09/53709/1
Alexandru Avadanii [Tue, 13 Mar 2018 18:25:41 +0000 (19:25 +0100)]
[odl] Add SDN_CONTROLLER_IP to openrc

JIRA: FUEL-352

Change-Id: I73aa5d41e93f3ebd5c8e44577868e8d21e0106a9
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[vcp] Catch 'no response' for salt state 05/53705/1
Alexandru Avadanii [Tue, 13 Mar 2018 17:09:26 +0000 (18:09 +0100)]
[vcp] Catch 'no response' for salt state

JIRA: FUEL-358

Change-Id: I8dc89676aa777068d1a13168bf7b7d7156903c03
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[virtual/odl] Turn off websocket for pseudo agent 79/53679/1
Michael Polenchuk [Tue, 13 Mar 2018 14:04:24 +0000 (18:04 +0400)]
[virtual/odl] Turn off websocket for pseudo agent

To prevent error in packet_handler callback: AttributeError:
'PseudoAgentDBBindingWebSocket' object has no attribute
'update_agents_db_row'.

Change-Id: I1a6a4b95ce3290ce108bdb91ae8eb5e85a13ef51
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agoMerge "[virtual/odl] Apply missing neutron.compute state"
Alexandru Avadanii [Mon, 12 Mar 2018 19:42:34 +0000 (19:42 +0000)]
Merge "[virtual/odl] Apply missing neutron.compute state"

6 years agoMerge "Set volume_device_name variable"
Alexandru Avadanii [Mon, 12 Mar 2018 19:41:01 +0000 (19:41 +0000)]
Merge "Set volume_device_name variable"

6 years agoUpdate patch with ovs bridges as L3 ifaces 75/53575/1
Michael Polenchuk [Mon, 12 Mar 2018 14:50:07 +0000 (18:50 +0400)]
Update patch with ovs bridges as L3 ifaces

Apply this patch if protocol is set to static to be
intended for the ip address settings action only.

Change-Id: I758340ff22376c01edd2a9a3555fe0fd9db3f4a9
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agoSet volume_device_name variable 63/53563/2
Michael Polenchuk [Mon, 12 Mar 2018 13:51:08 +0000 (17:51 +0400)]
Set volume_device_name variable

This evironment variable is required by test suite to refer
to an attached volume (vdb is reserved for config drive).

JIRA: FUEL-353
Change-Id: I4f7c96b2344575fcd9b785e3c74b27ef4c4d64f8
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years ago[virtual/odl] Apply missing neutron.compute state 33/53533/1
Michael Polenchuk [Mon, 12 Mar 2018 12:39:41 +0000 (16:39 +0400)]
[virtual/odl] Apply missing neutron.compute state

Change-Id: I078e11219fb8dea4505c46e7f75c295c5a72c59b
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agolib.sh: Add missing `sudo` for `losetup` 81/53481/1
Alexandru Avadanii [Sun, 11 Mar 2018 22:46:21 +0000 (23:46 +0100)]
lib.sh: Add missing `sudo` for `losetup`

Change-Id: I1c5e3d7a0dbac14bf242730d6ac8d2b1d0817907
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[IDF] Move ip-range start higher for mgmt, public 67/53467/1
Alexandru Avadanii [Sun, 11 Mar 2018 16:28:42 +0000 (17:28 +0100)]
[IDF] Move ip-range start higher for mgmt, public

- move mgmt start to .10;
- move public start to .100;
- sync local idf-pod1;

JIRA: ARMBAND-365

Change-Id: I5d93c4f486daf280d8bdbe92f5aa7edd8d020db0
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoFix missing root_helper_daemon for cmp, gtw 43/53343/1
Alexandru Avadanii [Thu, 8 Mar 2018 21:07:19 +0000 (22:07 +0100)]
Fix missing root_helper_daemon for cmp, gtw

While at it, drop patch now upstream.

JIRA: FUEL-348

Fixes: cf6cd9cd

Change-Id: Ieb4c93e9dc79a4e4dec140433574e042a1a6f2dc
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoRevert "[baremetal] Retire mas01 NAT" 67/53267/1
Alexandru Avadanii [Wed, 7 Mar 2018 23:49:03 +0000 (00:49 +0100)]
Revert "[baremetal] Retire mas01 NAT"

Bring back public internet access to all cluster nodes via NAT
on mas01 node, required for NTP syncing.

NOTE: Both mcpcontrol and PXE/admin networks are currently
hard wired to using /24 netmask, so we leverage that in pxe_nat.sls.

JIRA: FUEL-348

This reverts commit 9a6e655e0b851ff6e449027c01ac1a66188b0064.

Change-Id: I7bab385f95f8c6d92cadc4e2149c2cd56e10c506
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[IDF] net_config: Add support for custom ip-range 61/53261/3
Alexandru Avadanii [Wed, 7 Mar 2018 21:00:29 +0000 (22:00 +0100)]
[IDF] net_config: Add support for custom ip-range

- submodule: bump Pharos to pick up installer adapter support, as well
  as IDF updates for Ericsson baremetal pod1;
- labs/local/virtual: Bump mgmt, public networks start addresses from
  .1 to .10, similar to ericsson-pod1;

While at it, drop patch now upstream and instead adopt the new param
'neutron:server:root_helper_daemon'.

JIRA: FUEL-351

Change-Id: I9bc244a7fd8698861a390ed2b6a27804be46c285
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "[novcp] Add deploy argument `-N` (experimental)"
Alexandru Avadanii [Wed, 7 Mar 2018 16:36:30 +0000 (16:36 +0000)]
Merge "[novcp] Add deploy argument `-N` (experimental)"

6 years agoMerge "[vnode parsing] Move core logic to j2"
Alexandru Avadanii [Wed, 7 Mar 2018 16:36:11 +0000 (16:36 +0000)]
Merge "[vnode parsing] Move core logic to j2"

6 years agoMerge "Clean up aodh formula symlink"
Alexandru Avadanii [Wed, 7 Mar 2018 16:35:36 +0000 (16:35 +0000)]
Merge "Clean up aodh formula symlink"

6 years agoMerge "[docs] Openstack endpoints description"
Alexandru Avadanii [Wed, 7 Mar 2018 16:35:26 +0000 (16:35 +0000)]
Merge "[docs] Openstack endpoints description"

6 years agoClean up aodh formula symlink 31/53231/1
Michael Polenchuk [Wed, 7 Mar 2018 11:34:22 +0000 (15:34 +0400)]
Clean up aodh formula symlink

Change-Id: I1d84b5d4e3eb0de1c3ebf6fc5a538ee6714d53b2
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years ago[docs] Openstack endpoints description 21/53121/3
Guillermo Herrero [Mon, 5 Mar 2018 19:54:22 +0000 (20:54 +0100)]
[docs] Openstack endpoints description

- Describe SSL certificate usage for public endpoints

- Fixed all code-blocks formatting on the file

JIRA:FUEL-328

Change-Id: Ifecab459ee0d633b4d8a254dcb01c92f76b66d4f
Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
6 years ago[novcp] Add deploy argument `-N` (experimental) 39/52339/25
Alexandru Avadanii [Mon, 19 Feb 2018 18:01:56 +0000 (19:01 +0100)]
[novcp] Add deploy argument `-N` (experimental)

Instead of duplicating scenarios for NOVCP, allow it to be specified
using a new deploy argument, `-N`.

Things are getting convoluted, so instead of creating dedicated
'*_pdf.yml.j2' files for each group of similar features, apply the
templating in-place and rename all affected files to ".yml.j2".

This breaks .gitignore assumption of hiding only "*_pdf.yml" files,
so extend (manually) the <mcp/reclass/classes/cluster/.gitignore>
with `git ls-files --exclude-standard -o` after an expansion.

- ha: move nfv.cpu_pinning to j2, conditioned by 'baremetal';
- ha: add cmp00* vnode definitions (hugepages need more RAM);
- labs/local: enable hugepage params for non-dpdk noha;
- salt.sh: add route_wrapper to all non-infra VMs;

This change extends novcp support to all HA scenarios.

Change-Id: I7a80415ac33367ab227ececb4ffb1bc026546d36
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[vnode parsing] Move core logic to j2 65/53065/5
Alexandru Avadanii [Sat, 3 Mar 2018 20:39:59 +0000 (21:39 +0100)]
[vnode parsing] Move core logic to j2

j2/python is easier to read and manipulate strings, although it does
need some special care about undefined dict keys.

With this in place, deploy.sh only contains the higher level logic for
the deployment process.

- merge arch-specific default configuration files into a singular
  file with arch name as main dict key of old config (also avoids
  creating duplicate 'virtual' YAML keys in $LOCAL_PDF);
- move template handling to separate <lib_template.sh>;
- decouple tight bash ordering of scenario expansion -> parse_yaml ->
  variable export (e.g. CLUSTER_DOMAIN) -> re-use in cluster j2s;
  however we can't parse *all* j2s in one go, as scenario j2s might
  expand to YAMLs needed while expanding cluster j2;
- split `do_templates` into separate functions for each stage, with
  no coupling between them other then call order;

Change-Id: I4b5e804094c00e5e918caf769fd85fa52181ad76
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[ha] Add route_wrapper to prx, kvm 93/53193/2
Alexandru Avadanii [Wed, 7 Mar 2018 01:50:47 +0000 (02:50 +0100)]
[ha] Add route_wrapper to prx, kvm

Similar to cmp, when route already exists, networking service fails
to start on 'nginx:server' slaves ('kvm' in novcp case).

JIRA: FUEL-349

Change-Id: I2dc83ea78528533e92c9b9125e78b6e4387bdfe2
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[patch] Fixup ceilometer server publisher default 85/53185/1
Alexandru Avadanii [Tue, 6 Mar 2018 19:03:53 +0000 (20:03 +0100)]
[patch] Fixup ceilometer server publisher default

- backport proposed fixup for ceilometer;
- remove 'ignore: all' from .gitmodules to make it easier to bump the
  submodule;
- bump system submodule to upstream latest;
- drop patch merged upstream;

JIRA: FUEL-350

Change-Id: I68f232e0da5af368a1669a7379eedde5c4168b84
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "[keepalived] Check proccess id of nginx on proxy"
Alexandru Avadanii [Tue, 6 Mar 2018 16:29:20 +0000 (16:29 +0000)]
Merge "[keepalived] Check proccess id of nginx on proxy"

6 years ago[keepalived] Check proccess id of nginx on proxy 63/53163/1
Michael Polenchuk [Tue, 6 Mar 2018 12:32:43 +0000 (16:32 +0400)]
[keepalived] Check proccess id of nginx on proxy

By default vrrp script seeks out haproxy process id, so change
check_pidof script arguments to "nginx" on proxy nodes.

Change-Id: I616bf90ba49fa8257d28d236dcf3b7c5598039f1
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agoMend opendaylight settings 61/53161/1
Michael Polenchuk [Tue, 6 Mar 2018 12:19:22 +0000 (16:19 +0400)]
Mend opendaylight settings

* class including order (default was used)
* disable root_helper_daemon (incompatible with UCA)
* turn off websocket for pseudo agent updates

Change-Id: I4d7971d393da184bdd55f65b4d3fd8d9e898543f
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years ago[ovn] Fix cluster name in dummy baremetal scenario 35/53135/1
Alexandru Avadanii [Tue, 6 Mar 2018 02:31:50 +0000 (03:31 +0100)]
[ovn] Fix cluster name in dummy baremetal scenario

JIRA: FUEL-329

Change-Id: I56c8e027dc797a86848b73af30adbf40de119f85
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[ha] Add ovn scenario placeholder 27/53127/1
Alexandru Avadanii [Tue, 6 Mar 2018 01:43:23 +0000 (02:43 +0100)]
[ha] Add ovn scenario placeholder

- dummy copy of os-nosdn-nofeature-ha masquerading as
  os-ovn-nofeature-ha scenario placeholder;

JIRA: FUEL-329

Change-Id: Ic5e6d65cf057289bb13ce2fa157b6ba53d7dab80
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[reclass] Add missing neutron keyword in class uri 19/53119/1
Alexandru Avadanii [Mon, 5 Mar 2018 19:12:05 +0000 (20:12 +0100)]
[reclass] Add missing neutron keyword in class uri

Change-Id: I71fdc7ccd84a0700333734b5144a4281b53cf25b
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoAlign opendaylight settings with upstream 07/53107/2
Michael Polenchuk [Mon, 5 Mar 2018 16:47:05 +0000 (20:47 +0400)]
Align opendaylight settings with upstream

Change-Id: If7d51555bc13dbcaa63f93ab1993f3655e2ce643
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years ago[jump req] Move pkg installation to lib.sh 63/53063/1
Alexandru Avadanii [Sun, 4 Mar 2018 02:50:01 +0000 (03:50 +0100)]
[jump req] Move pkg installation to lib.sh

Change-Id: I687b73b256aca78c9d41d4bcd49bfbde51278b51
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[deploy.sh] Factor out jump bridge parsing to j2 61/53061/1
Alexandru Avadanii [Sat, 3 Mar 2018 19:16:43 +0000 (20:16 +0100)]
[deploy.sh] Factor out jump bridge parsing to j2

Change-Id: Ie4d8e70866d533d195a6e80cdfecbdb00a3027ce
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[submodule] Bump Pharos for IDF updates 41/53041/1
Alexandru Avadanii [Sat, 3 Mar 2018 18:41:56 +0000 (19:41 +0100)]
[submodule] Bump Pharos for IDF updates

Change-Id: I9e71d9006bad53cc79805ed9255b53b6c5705725
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[j2 parsing] Adopt new generate_config features 17/53017/5
Alexandru Avadanii [Thu, 8 Feb 2018 19:35:44 +0000 (20:35 +0100)]
[j2 parsing] Adopt new generate_config features

- bump Pharos git submodule for j2 'do' extension + batch mode;
- adopt j2 'do' in our templates;
- use int filter for 'native' vlan check;
- lib.sh: adopt `-i` to remove `ln` hack for net_map.j2;
- lib.sh: adopt `-b` to speedup template parsing;

NOTE: Bumping Pharos will also bring in the latest changes in
pod_config.yml.j2, which include massive IP shifts and updates.

JIRA: FUEL-335

Change-Id: I7d3a997b3d8659d5f09f867870fb3a148c1ec6df
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "Conform network scheme to tagged public interface"
Alexandru Avadanii [Fri, 2 Mar 2018 16:37:51 +0000 (16:37 +0000)]
Merge "Conform network scheme to tagged public interface"

6 years agoMerge "[ovs/dpdk] [noha] Do not add private nic to ifaces"
Alexandru Avadanii [Fri, 2 Mar 2018 16:37:22 +0000 (16:37 +0000)]
Merge "[ovs/dpdk] [noha] Do not add private nic to ifaces"

6 years agoConform network scheme to tagged public interface 57/52457/3
Michael Polenchuk [Thu, 22 Feb 2018 14:47:28 +0000 (18:47 +0400)]
Conform network scheme to tagged public interface

Change-Id: I503c8ad32900406e1fa375cec9a91b454889d8bf
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agoInstall missing python module of jsonschema 67/52967/1
Michael Polenchuk [Fri, 2 Mar 2018 07:42:12 +0000 (11:42 +0400)]
Install missing python module of jsonschema

Change-Id: Ib903863a7a46008c93ee795083712a50a022a41d
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agoMerge "lib.sh: Validate pdf/idf with pharos schema"
Alexandru Avadanii [Thu, 1 Mar 2018 16:57:19 +0000 (16:57 +0000)]
Merge "lib.sh: Validate pdf/idf with pharos schema"

6 years ago[ovs/dpdk] [noha] Do not add private nic to ifaces 95/52795/2
Alexandru Avadanii [Wed, 28 Feb 2018 03:12:10 +0000 (04:12 +0100)]
[ovs/dpdk] [noha] Do not add private nic to ifaces

All settings specific to private nic are handled by linux.network.dpdk
but since the private nic's `interface.type` is currently 'eth', it is
not filtered out from /etc/network/interfaces.

To prevent this, define `interface.type` as a dummy value containing the
'dpdk' substring.

Change-Id: I565153ee80698572209e12b4eec5f44f9416d945
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agolib.sh: Validate pdf/idf with pharos schema 43/52843/4
Guillermo Herrero [Wed, 28 Feb 2018 15:30:19 +0000 (16:30 +0100)]
lib.sh: Validate pdf/idf with pharos schema

Run the pharos yaml schema validation for configuration files
before expanding them

JIRA: FUEL-341

Change-Id: Ia1d69f53265876683a1b6674665a9594ba7dae16
Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
6 years agoClean up upstreamed patches 29/52829/1
Michael Polenchuk [Wed, 28 Feb 2018 14:10:17 +0000 (18:10 +0400)]
Clean up upstreamed patches

Change-Id: Icee56da3720f0926e42390965581639f6a344b77
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years ago[submodule] Bump Pharos git repo for IDF updates 95/52695/1
Alexandru Avadanii [Tue, 27 Feb 2018 03:02:37 +0000 (04:02 +0100)]
[submodule] Bump Pharos git repo for IDF updates

Change-Id: I8d526d51a3e18652c59bad9d7c3c384aa6583566
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[patch] cleanup: Drop LVM filter disable 77/52677/2
Alexandru Avadanii [Mon, 26 Feb 2018 17:40:01 +0000 (18:40 +0100)]
[patch] cleanup: Drop LVM filter disable

Now we explicitly add a LVM PV on /dev/sda{1,2} for Cinder storage,
so we can safely drop the patch disabling LVM volume filtering.
If we later move the PV to a different disk, we can just add the VG
and LV definitions to linux:storage via reclass, and Salt will skip
setting them up (as they're already created by MaaS), yet keep the
filtering sane.

While at it, fix 'nova_cpu_pinning' param expr; constructs based on
reclass interpolation (e.g. '${_param:x}') do not work when
parameters are passed via reclass.storage templating, so change
reclass interpolation syntax with classic YAML anchors.

Fixes: 672ae12

Change-Id: Ieb41635ddeb630543d7e4d1079f45d636d9a43af
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[ovs/dpdk] Parameterize node-specific compute args 91/52491/7
Cristina Pauna [Wed, 21 Feb 2018 14:50:12 +0000 (16:50 +0200)]
[ovs/dpdk] Parameterize node-specific compute args

- node-specific parameters (nova pinning, hugepages, dpdk) should be
  configurable via IDF, on a per-node basis;
- keep default settings for lf-pod2, with and without DPDK,
  override them for virtual deploys via local-virtual1 IDF;
- leave neutron_tenant_* vars hardcoded for now, as they are required
  on both ctl and cmp nodes - this way we'll deal stricly with
  cmp params, so we can nicely pass them via config.yml to reclass
  per-node (and not per-role), allowing mixed computes later;
- add compute params for ovs/odl-noha, preparing them for
  deployment on baremetal later.

JIRA: ARMBAND-343

Change-Id: I89a58b9565679ab3882d85f07ae817690ae85c67
Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
6 years ago[ovs/dpdk] Add opnfv.route_wrapper sls 57/52557/3
Alexandru Avadanii [Fri, 23 Feb 2018 21:10:33 +0000 (22:10 +0100)]
[ovs/dpdk] Add opnfv.route_wrapper sls

- fix `route-br-ex` if-up.d script failing when route already exists
  by adding a wrapper around distro's '/sbin/route' binary in
  '/usr/local/sbin/route', exploiting default order in Ubuntu PATH;
- fix 'br-prv' duplicate entry in 'interfaces.d/ifcfg-br-prv' and
  'interfaces' caused by upstream bug [1];
- add barrier waiting for all baremetal nodes online before attempting
  reboot, trying to catch rare failures which are undetectable in logs
  as both a succesful reboot and a disconneted minion report 'n/c';

With the above in place, networking service should no longer fail
to start on cmp nodes w/ DPDK.

[1] https://github.com/saltstack/salt/issues/40262

Change-Id: I6d4895376ce323c14c997e6c9af2ea3eeeee0184
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[salt.sh] Add explicit apt-get update 99/52599/1
Alexandru Avadanii [Sun, 25 Feb 2018 16:47:42 +0000 (17:47 +0100)]
[salt.sh] Add explicit apt-get update

Change-Id: I17e66457f90818bad4d746300bb49e9d5179c239
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[MaaS] Force Maas to use a fixed ip packet size 97/51397/10
Charalampos Kominos [Wed, 31 Jan 2018 12:59:48 +0000 (13:59 +0100)]
[MaaS] Force Maas to use a fixed ip packet size

Maas uses a tftp library during commisioning which in turn uses another
library which tries to find the best packet size to use. For some reason
during commisioning the library uses smaller sizes than the
infrastructure can handle and therefore takes a lot of time to deploy.

JIRA: ARMBAND-351

Change-Id: I9b3083a059c04b118e8b7f0f2723af67d96d2aad
Signed-off-by: Charalampos Kominos <charalampos.kominos@enea.com>
6 years ago[noHA] Add dedicated cinder volume for virtual cmp 41/52441/4
Alexandru Avadanii [Thu, 22 Feb 2018 02:02:05 +0000 (03:02 +0100)]
[noHA] Add dedicated cinder volume for virtual cmp

Replace loop device LVM-backed cinder volume with a dedicated
/dev/vdb drive.
This is another small step towards bringing noHA to baremetal.

Change-Id: I80f9c2bee42e933a36ab7a8f9b4c5247d1652b42
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[HA] [cinder] Fix rootfs part name for sda > 2TB 37/52437/2
Alexandru Avadanii [Thu, 22 Feb 2018 01:02:45 +0000 (02:02 +0100)]
[HA] [cinder] Fix rootfs part name for sda > 2TB

On systems with more than 2TB on /dev/sda, MaaS automatically creates
a 1MB partition with bios_grub label as /dev/sda1.
Account for this and expect rootfs to reside on /dev/sda2 (assume
all compute nodes have the same behavior wrt this issue).

Ideally, later we will switch to explicit partitioning via MaaS, or to
a new baremetal provisioner like Ironic.

Requires bumping Pharos git submodule to pick up the new j2 filter.

Fixes: f6f8a56

JIRA: FUEL-330

Change-Id: I12f5046cf69bf2f52c8b8dc331b90048df643646
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[HA] [cinder] Switch loop to free space on sda 83/52283/6
Alexandru Avadanii [Sat, 17 Feb 2018 23:30:20 +0000 (00:30 +0100)]
[HA] [cinder] Switch loop to free space on sda

On cmp nodes, we assing 30G (fixed) to rootfs, use the rest for
cinder.

Note: AArch64 gets one extra partition (the EFI System Partition or
ESP) on /dev/sda1 via MaaS.

JIRA: FUEL-330

Change-Id: I2a36107d074532b627bd2349cafc0c8ee61f500f
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[salt.sh] Armband extra repo for forked reclass 09/52309/4
Alexandru Avadanii [Mon, 19 Feb 2018 01:44:12 +0000 (02:44 +0100)]
[salt.sh] Armband extra repo for forked reclass

- add Armband nightly/extra DEB repository;
- install forked and updated reclass 1.5.2 which includes:
  * better error reporting;
  * support for deleting existing keys during list interpolation;
  * various other improvements and optimizations;

While at it, update copyright year for patches.

JIRA: FUEL-345

Change-Id: I00d8b625fe191648e7ea34b3dd4c8375691384e6
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoReset kernel version to xenial image default one 57/52357/2
Michael Polenchuk [Tue, 20 Feb 2018 14:00:55 +0000 (18:00 +0400)]
Reset kernel version to xenial image default one

Virtual compute nodes goes to kernel panic during nova instances ops
under hwe kernel, so use the default one from xenial image.

Change-Id: Iae100b68208cc1fb9e43e45f385e762cdbd6573a
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years ago[reclass] dbs, msg: Inherit only infra classes 25/52325/2
Alexandru Avadanii [Mon, 19 Feb 2018 19:19:03 +0000 (20:19 +0100)]
[reclass] dbs, msg: Inherit only infra classes

All other VCP nodes (mdb, prx, ctl etc.) inherit only the infra
subdir of each scenario, so apply the same to dbs, msg.

This should cut down some operations during reclass interpolation
for affected nodes, without any changes in output pillar data.

Change-Id: I77ae6e1d5658cba87fcd8c45a5f8f5e177e0eda9
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[reclass] Update copyright year 35/52335/1
Alexandru Avadanii [Mon, 19 Feb 2018 23:03:55 +0000 (00:03 +0100)]
[reclass] Update copyright year

Change-Id: Id1ca66938531e1d24ec0d44194f8b09643062944
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "[Horizon] Fix 'mcp' version check pattern"
Alexandru Avadanii [Mon, 19 Feb 2018 12:23:11 +0000 (12:23 +0000)]
Merge "[Horizon] Fix 'mcp' version check pattern"

6 years agoMerge "[MaaS] Add maas.machines.set_storage_layout sls"
Alexandru Avadanii [Mon, 19 Feb 2018 12:22:58 +0000 (12:22 +0000)]
Merge "[MaaS] Add maas.machines.set_storage_layout sls"

6 years agoMerge changes from topics 'lab-proxy-keyserver', 'maas-override-failed-testing'
Alexandru Avadanii [Mon, 19 Feb 2018 12:22:35 +0000 (12:22 +0000)]
Merge changes from topics 'lab-proxy-keyserver', 'maas-override-failed-testing'

* changes:
  [baremetal] Use upstream lab proxy for keyservers
  [MaaS] Override failed testing by default

6 years agoMerge "[patch] MaaS: mcp.rsa.pub to auth keys via pillar"
Alexandru Avadanii [Mon, 19 Feb 2018 12:22:19 +0000 (12:22 +0000)]
Merge "[patch] MaaS: mcp.rsa.pub to auth keys via pillar"

6 years ago[Horizon] Fix 'mcp' version check pattern 85/52285/1
Alexandru Avadanii [Sun, 18 Feb 2018 19:59:56 +0000 (20:59 +0100)]
[Horizon] Fix 'mcp' version check pattern

Previous commit used a pattern that is too generic and always matches
the substring 'mcp' vs the node hostname, not only pkg version.

Fixes: 4658acf

Change-Id: Ia4dcbbf7cdfa68574c86459217101d83d61add01
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[baremetal] Use upstream lab proxy for keyservers 75/51375/2
Alexandru Avadanii [Wed, 31 Jan 2018 01:12:51 +0000 (02:12 +0100)]
[baremetal] Use upstream lab proxy for keyservers

Proxy chainloading does not always work for keyserver requests, so:
- mv common.infra.lab_proxy_pdf common.include.lab_proxy_pdf;
- mv common.include.proxy common.include.maas_proxy;
- include lab_proxy_pdf after each maas_proxy, so if upstream_proxy
  is defined, it will override the MaaS proxy;

JIRA: FUEL-317

Change-Id: I63749be0d350ea73ea2cf6e629710766b14ecb73
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[MaaS] Add maas.machines.set_storage_layout sls 75/51875/3
Alexandru Avadanii [Thu, 8 Feb 2018 00:54:55 +0000 (01:54 +0100)]
[MaaS] Add maas.machines.set_storage_layout sls

On cmp nodes, allocate only 30GB (fixed for now) for / partition.
The rest of the disk(s) can later be allocated via salt-formula-linux.

JIRA: FUEL-330

Change-Id: Ie11c78791e60801719cd33475ff91fc003df5ffa
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[MaaS] Override failed testing by default 79/52279/3
Alexandru Avadanii [Sat, 17 Feb 2018 17:01:57 +0000 (18:01 +0100)]
[MaaS] Override failed testing by default

Some nodes fail automatic testing done by MaaS during commissioning,
although running the testing suites one more time manually works.
For now, just override all 'failed testing' nodes unconditionally.

JIRA: FUEL-333

Change-Id: I13d3ee3d82550524480aa53aa8752ab90aa940cd
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "reclass: maas: Dynamic machine definitions"
Alexandru Avadanii [Sat, 17 Feb 2018 19:18:56 +0000 (19:18 +0000)]
Merge "reclass: maas: Dynamic machine definitions"

6 years agoSet nova disk cachemodes to file directsync 31/52231/1
Michael Polenchuk [Fri, 16 Feb 2018 09:20:41 +0000 (13:20 +0400)]
Set nova disk cachemodes to file directsync

Set nova/libvirt disk cache modes to file directsync to bypass the
host page cache & prevent instance's kernel panic due to absent
/dev partition. Caching mode directsync effectively turns all guest
I/O operations into direct I/O operations on the host, which is the
NFS client (virtuals) or GlusterFS (baremetals).
Also return back to hwe kernel as a golden mean between GA & Edge ones.

Change-Id: I51ab7d0ee71c214ff16b756cfee16e918738b6fd
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agoFix check if node type is baremetal for novcp 99/52199/2
Delia Popescu [Thu, 15 Feb 2018 16:06:08 +0000 (18:06 +0200)]
Fix check if node type is baremetal for novcp

Fixes: 86c8109

Change-Id: I0a947bd7db5dd7cb830cc932f5a0d16831551923
Signed-off-by: Delia Popescu <delia.popescu@enea.com>
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMask opendaylight service 83/52183/1
Michael Polenchuk [Thu, 15 Feb 2018 09:19:33 +0000 (13:19 +0400)]
Mask opendaylight service

In order to avoid using cache data with initial/outdated
configuration, mask opendaylight service before package
installation.

JIRA: FUEL-344
Change-Id: I71eb0b0a5af93d6d21698e76587b32098aba96b4
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agoMerge "[docs] Temp dir permission requirements"
Alexandru Avadanii [Wed, 14 Feb 2018 12:46:56 +0000 (12:46 +0000)]
Merge "[docs] Temp dir permission requirements"

6 years ago[patch] MaaS: mcp.rsa.pub to auth keys via pillar 21/52121/1
Alexandru Avadanii [Tue, 13 Feb 2018 02:34:10 +0000 (03:34 +0100)]
[patch] MaaS: mcp.rsa.pub to auth keys via pillar

Drop one questionable patch responsible for MaaS node authorized
keys to include mcp.rsa.pub by reading the contents of authorized
keys on mas01, assuming mcp.rsa.pub will be on the first line.

Instead, export the contents of the public key using a shell env
var during deploy, which gets expanded via maas_pdf j2 template
into a reclass param, leveraging existing salt-formula-maas sshprefs
mechanism for delivering the key to MaaS.

Since we require the public key to exist before expanding templates,
move `generate_ssh_key` call outside the current infrastructure
handling block, allowing it to execute during all `deploy.sh` calls,
even for dry-runs.

Change-Id: I0f53b0f764a2fafd292e0ffd399c284acf61bd30
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoreclass: maas: Dynamic machine definitions 19/52119/1
Alexandru Avadanii [Sun, 4 Feb 2018 03:42:58 +0000 (04:42 +0100)]
reclass: maas: Dynamic machine definitions

Refactor maas machine definition flow from going through variables
in pod_config.yaml to directly handling it in Fuel using a new j2
template.
This prepares for future improvements allowing a dynamic compute
node count, as well as parameterizing new values via j2.

For now, node roles and count are still statically mapped to PDF
nodes.

JIRA: FUEL-319

Change-Id: I770d82987fcb99792f9d5bc0857ab513f5cd3731
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[docs] Temp dir permission requirements 57/51957/3
Cristina Pauna [Fri, 9 Feb 2018 12:50:25 +0000 (14:50 +0200)]
[docs] Temp dir permission requirements

JIRA: FUEL-342

Change-Id: I35efc993647c63779107d5f5604e58b124cdcf69
Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
6 years agoMerge "[centos jump] Add more missing required packages"
Alexandru Avadanii [Tue, 13 Feb 2018 15:32:36 +0000 (15:32 +0000)]
Merge "[centos jump] Add more missing required packages"

6 years ago[centos jump] Add more missing required packages 65/51965/1
Alexandru Avadanii [Thu, 8 Feb 2018 18:03:29 +0000 (19:03 +0100)]
[centos jump] Add more missing required packages

- gcc is required for NBD kernel module build;
- gdisk is required for resizing rootfs in nbd-mounted chroot;

Change-Id: I8863c126b75ce3a15998c03d2d159f53f5006f5d
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "[jump] Add simple check for required Linux bridges"
Alexandru Avadanii [Fri, 9 Feb 2018 15:40:52 +0000 (15:40 +0000)]
Merge "[jump] Add simple check for required Linux bridges"

6 years agosalt.sh: Fix one more broken online check 39/51939/1
Alexandru Avadanii [Fri, 9 Feb 2018 02:18:58 +0000 (03:18 +0100)]
salt.sh: Fix one more broken online check

Fixes: 5f95f52

Change-Id: I53e08a89b2e873829a78ff75c0f3532329ef87a3
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[virsh net] Fix virtual node check for public net 67/51867/1
Alexandru Avadanii [Wed, 7 Feb 2018 21:40:30 +0000 (22:40 +0100)]
[virsh net] Fix virtual node check for public net

Jinja variables set inside loops do not live outside current
iteration, so use a dictionary.update to work around that.

Fixes: 3d2c66e

Change-Id: I2bef64dddab080486a71952465edc4e528ae9e17
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[states] Fix broken online check for bm, vcp nodes 61/51861/3
Alexandru Avadanii [Wed, 7 Feb 2018 19:03:09 +0000 (20:03 +0100)]
[states] Fix broken online check for bm, vcp nodes

Previous commit replacing explicit loops with `wait_for` failed to
properly escape a nested variable, leading to deploy failure.

Also, the logic was flawed, not breaking for offline nodes, rendering
the whole barrier check useless.

Fixes: 1a0e8e7e

Change-Id: I038dbf90fb53c6b61da2e5c9b6867e31d78867af
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[jump] Add simple check for required Linux bridges 27/51827/2
Alexandru Avadanii [Wed, 7 Feb 2018 01:31:52 +0000 (02:31 +0100)]
[jump] Add simple check for required Linux bridges

- MaaS requires PXE/admin to be a Linux bridge;
- if virtual nodes are present, they should be hooked to a proper
  Linux bridge for the Public network, but only throw a warning if
  not (and create a mock public virsh network instead);
- if both virtual and baremetal nodes are present, Public bridge is
  indirectly mandatory (we can't mock it);

JIRA: FUEL-339

Change-Id: Idfe99d66c49eadc56cb3d94ca4db3467fb76d388
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years agoMerge "[virtual] Rename all to drop virtual prefix"
Alexandru Avadanii [Wed, 7 Feb 2018 14:53:12 +0000 (14:53 +0000)]
Merge "[virtual] Rename all to drop virtual prefix"

6 years agoMerge "[hybrid] Merge config/scenario/{baremetal,virtual}"
Alexandru Avadanii [Wed, 7 Feb 2018 14:53:05 +0000 (14:53 +0000)]
Merge "[hybrid] Merge config/scenario/{baremetal,virtual}"

6 years agoMerge "[states] maas, vcp: Use `wait_for` in online check"
Alexandru Avadanii [Wed, 7 Feb 2018 14:52:57 +0000 (14:52 +0000)]
Merge "[states] maas, vcp: Use `wait_for` in online check"

6 years agoMerge "Add NOVCP HA OVS scenario (baremetal, virtual)"
Alexandru Avadanii [Wed, 7 Feb 2018 14:52:46 +0000 (14:52 +0000)]
Merge "Add NOVCP HA OVS scenario (baremetal, virtual)"

6 years agoSwitch off broken sphinx state 43/51843/1
Michael Polenchuk [Wed, 7 Feb 2018 12:49:04 +0000 (16:49 +0400)]
Switch off broken sphinx state

Deactivate documentation related optional state
until it get fixed in upstream.

Change-Id: I5242ed307548c4f37f81d271a1f4f6bee9903f4e
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
6 years agoMerge changes from topic 'fix-public-bridge'
Michael Polenchuk [Wed, 7 Feb 2018 07:51:47 +0000 (07:51 +0000)]
Merge changes from topic 'fix-public-bridge'

* changes:
  [virtual] Add missing ODL dhcp_int
  [virsh net] public: rm addr cfg for pure baremetal

6 years agoMerge "[baremetal] cleanup: rm cloud-init iface config"
Michael Polenchuk [Wed, 7 Feb 2018 07:51:24 +0000 (07:51 +0000)]
Merge "[baremetal] cleanup: rm cloud-init iface config"

6 years ago[baremetal] cleanup: rm cloud-init iface config 59/49759/2
Alexandru Avadanii [Fri, 29 Dec 2017 13:22:00 +0000 (14:22 +0100)]
[baremetal] cleanup: rm cloud-init iface config

Upstream commit [1] now handles cloud init interface configuration
cleanup, so drop our explicit config from reclass.

[1] https://github.com/salt-formulas/salt-formula-linux/commit/b333d284

Change-Id: Ifd624f1e884d0d884513a3a7c3e18ffe3f00949a
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
6 years ago[virtual] Rename all to drop virtual prefix 85/51785/1
Alexandru Avadanii [Mon, 5 Feb 2018 17:31:18 +0000 (18:31 +0100)]
[virtual] Rename all to drop virtual prefix

JIRA: FUEL-322

Change-Id: I99c4b1774a7c3afbc834a6f6e8468f7baf1bc329
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>