MaaS: preseed Armband repo via curtin 35/50135/1
authorAlexandru Avadanii <Alexandru.Avadanii@enea.com>
Thu, 28 Dec 2017 17:41:09 +0000 (18:41 +0100)
committerAlexandru Avadanii <Alexandru.Avadanii@enea.com>
Fri, 5 Jan 2018 22:14:01 +0000 (23:14 +0100)
Instead of installing upstream vanilla kernel from Ubuntu just to
upgrade to the one from Armband repo later, preseed Armband repo
config via curtin.
This allows us to drop a kernel update cycle, thus saving a few
minutes at each deploy.

While at it, switch salt repo addition to use the maas region reclass
section instead of adding a dedicated mechanism for it in the curtin
template (only for arm64).

[ stable/ euphrates ]
Preseed all Armband repos, since Ocata does not have a "common" repo.

JIRA: ARMBAND-280

Change-Id: Iad5953a7de80d728f11a4ce3113bd7725100c31c
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
(cherry picked from commit 648ec98613d31e3d9701b9ac1d87aed640b245f3)

17 files changed:
patches/opnfv-fuel/0001-baremetal-Add-Armband-Openstack-repos-to-kvm-cmp.patch [deleted file]
patches/opnfv-fuel/0001-salt-formulas-Add-enable-armband-formula.patch [moved from patches/opnfv-fuel/0002-salt-formulas-Add-enable-armband-formula.patch with 100% similarity]
patches/opnfv-fuel/0002-network-public-Use-arm-virtual2-POD-config.patch [moved from patches/opnfv-fuel/0003-network-public-Use-arm-virtual2-POD-config.patch with 100% similarity]
patches/opnfv-fuel/0003-mcp-salt-formulas-opendaylight-AArch64-leveldb.patch [moved from patches/opnfv-fuel/0004-mcp-salt-formulas-opendaylight-AArch64-leveldb.patch with 100% similarity]
patches/opnfv-fuel/0004-maas-Add-curtin_userdata_arm64_generic_xenial.patch [moved from patches/opnfv-fuel/0005-maas-Add-curtin_userdata_arm64_generic_xenial.patch with 66% similarity]
patches/opnfv-fuel/0005-maas-boot-resources-Add-arm64-architecture.patch [moved from patches/opnfv-fuel/0006-maas-boot-resources-Add-arm64-architecture.patch with 100% similarity]
patches/opnfv-fuel/0006-libvirt-Use-libvirt-unix_sock_group.patch [moved from patches/opnfv-fuel/0007-libvirt-Use-libvirt-unix_sock_group.patch with 100% similarity]
patches/opnfv-fuel/0007-mcp-salt-formulas-armband-Extend-libvirt_domain.patch [moved from patches/opnfv-fuel/0008-mcp-salt-formulas-armband-Extend-libvirt_domain.patch with 100% similarity]
patches/opnfv-fuel/0008-virtng.py-virt.sls-Extend-libvirt_domain.patch [moved from patches/opnfv-fuel/0009-virtng.py-virt.sls-Extend-libvirt_domain.patch with 98% similarity]
patches/opnfv-fuel/0009-seedng-module-Add-AArch64-repo.patch [moved from patches/opnfv-fuel/0010-seedng-module-Add-AArch64-repo.patch with 100% similarity]
patches/opnfv-fuel/0010-aarch64-skip-configuration-of-hugepages.patch [moved from patches/opnfv-fuel/0011-aarch64-skip-configuration-of-hugepages.patch with 96% similarity]
patches/opnfv-fuel/0011-baremetal-virtual-Extend-arch-list-for-UCA-repo.patch [moved from patches/opnfv-fuel/0012-baremetal-virtual-Extend-arch-list-for-UCA-repo.patch with 100% similarity]
patches/opnfv-fuel/0012-Add-opnfv-user-to-the-deployment.patch [moved from patches/opnfv-fuel/0013-Add-opnfv-user-to-the-deployment.patch with 100% similarity]
patches/opnfv-fuel/0013-Add-pre-install-purge-support-for-base-image.patch [moved from patches/opnfv-fuel/0015-Add-pre-install-purge-support-for-base-image.patch with 100% similarity]
patches/opnfv-fuel/0014-AArch64-base-image-pre-install-salt-minion.patch [moved from patches/opnfv-fuel/0016-AArch64-base-image-pre-install-salt-minion.patch with 100% similarity]
patches/opnfv-fuel/0014-baremetal-linux-image-generic-hwe-16.04-edge.patch [deleted file]
patches/opnfv-fuel/0015-base-image-Explicitly-call-kpartx.patch [moved from patches/opnfv-fuel/0017-base-image-Explicitly-call-kpartx.patch with 100% similarity]

diff --git a/patches/opnfv-fuel/0001-baremetal-Add-Armband-Openstack-repos-to-kvm-cmp.patch b/patches/opnfv-fuel/0001-baremetal-Add-Armband-Openstack-repos-to-kvm-cmp.patch
deleted file mode 100644 (file)
index 85751ce..0000000
+++ /dev/null
@@ -1,49 +0,0 @@
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-: Copyright (c) 2017 Enea AB and others.
-:
-: All rights reserved. This program and the accompanying materials
-: are made available under the terms of the Apache License, Version 2.0
-: which accompanies this distribution, and is available at
-: http://www.apache.org/licenses/LICENSE-2.0
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
-Date: Sun, 26 Nov 2017 17:25:08 +0100
-Subject: [PATCH] baremetal: Add Armband Openstack repos to kvm, cmp
-
-Armband provides 2 sets of APT repositories:
-- linux.system.repo.mcp.armband.openstack;
-- linux.system.repo.mcp.armband.extra (currently empty);
-
-These repos are pre-installed for all infrastructure VMs, so we only
-handle them via Salt for baremetal kvm* and cmp* nodes.
-
-Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
----
- mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/kvm.yml     | 1 +
- .../classes/cluster/baremetal-mcp-ocata-common/openstack_compute.yml     | 1 +
- 2 files changed, 2 insertions(+)
-
-diff --git a/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/kvm.yml b/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/kvm.yml
-index ce7fc85..6491798 100644
---- a/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/kvm.yml
-+++ b/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/kvm.yml
-@@ -7,6 +7,7 @@
- ##############################################################################
- ---
- classes:
-+  - system.linux.system.repo.mcp.armband.openstack
-   - system.linux.system.repo.mcp.openstack
-   - system.linux.system.repo.mcp.extra
-   - system.linux.system.repo.glusterfs
-diff --git a/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/openstack_compute.yml b/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/openstack_compute.yml
-index e07e611..278e55b 100644
---- a/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/openstack_compute.yml
-+++ b/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/openstack_compute.yml
-@@ -7,6 +7,7 @@
- ##############################################################################
- ---
- classes:
-+  - system.linux.system.repo.mcp.armband.openstack
-   - system.linux.system.repo.mcp.extra
-   - system.linux.system.repo.glusterfs
-   - system.linux.storage.loopback
@@ -11,14 +11,16 @@ Date: Mon, 7 Aug 2017 19:45:01 +0200
 Subject: [PATCH] maas: Add curtin_userdata_arm64_generic_xenial
 
 Add AArch64 specific configuration.
+While at it, preseed Armband Openstack & APT-MK for Ocata,
+so we get the updated kernel & other packages from the start.
 
 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
 ---
- ...-Add-curtin_userdata_arm64_generic_xenial.patch | 35 ++++++++++++++++++
+ ...-Add-curtin_userdata_arm64_generic_xenial.patch | 35 +++++++++++++++++
  mcp/patches/patches.list                           |  1 +
- .../baremetal-mcp-ocata-common/infra/maas.yml      | 26 +++++++++++++
- .../files/curtin_userdata_arm64_generic_xenial     | 43 ++++++++++++++++++++++
- 4 files changed, 105 insertions(+)
+ .../baremetal-mcp-ocata-common/infra/maas.yml      | 44 ++++++++++++++++++++++
+ .../files/curtin_userdata_arm64_generic_xenial     | 35 +++++++++++++++++
+ 4 files changed, 115 insertions(+)
  create mode 100644 mcp/patches/0101-maas-Add-curtin_userdata_arm64_generic_xenial.patch
  create mode 100644 mcp/salt-formulas/maas/files/curtin_userdata_arm64_generic_xenial
 
@@ -73,48 +75,66 @@ index f2549d7..a2f6a19 100644
  /usr/share/salt-formulas/env: 0012-linux.storage.lvm-Disable-filter.patch
 +/usr/share/salt-formulas/env: 0101-maas-Add-curtin_userdata_arm64_generic_xenial.patch
 diff --git a/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/maas.yml b/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/maas.yml
-index e8c2965..b15f9cb 100644
+index e8c2965..e972df3 100644
 --- a/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/maas.yml
 +++ b/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/maas.yml
-@@ -110,6 +110,32 @@ parameters:
-           architecture: ${_param:opnfv_maas_node05_architecture}
-           distro_series: xenial
-           hwe_kernel: hwe-16.04
-+    cluster:
-+      enabled: true
-+      region:
-+        port: 80
-+        host: localhost
-+      saltstack_repo_key_arm64: |
-+        -----BEGIN PGP PUBLIC KEY BLOCK-----
-+        Version: GnuPG v1
+@@ -40,6 +40,50 @@ parameters:
+         enable_third_party_drivers: true
+         network_discovery: 'enabled'
+         default_min_hwe_kernel: 'hwe-16.04'
++      package_repositories:
++        armband_openstack:
++          name: armband_openstack
++          enabled: '1'
++          url: 'http://linux.enea.com/mcp-repos/${_param:openstack_version}/${_param:linux_system_codename}'
++          distributions: '${_param:openstack_version}'
++          components: 'main'
++          arches: 'arm64'
++          key: &armband_key |
++            -----BEGIN PGP PUBLIC KEY BLOCK-----
++            Version: GnuPG v1
 +
-+        mQENBFagAroBCADWboNIjuF6lB1mWv2+EbvqY3lKl5mLKhr2DnSUkKeHUPBv8gNM
-+        qK8Q00AMIyPiyEhgjA+dWizZ+5aBgxoiY7oMeLJ2Xym36U/8SYq2BWd3SGCbMNoz
-+        SJDxDUSM/HFVs6atF1M3DY9oN65hSVnu4uy5Tu6asf6k4rhAyk0z4+pRcPBCu2vq
-+        mnGi3COM/+9PShrEKeVOx5W2vRJywUFuq8EDvQnRoJ0GvM28JiJIanw17YwIPxhg
-+        BKZVpZjan5X+ihVMXwA2h/G/FS5Omhd50RqV6LWSYs94VJJgYqHx8UMm7izcxI+P
-+        ct3IcbD195bPbJ+SbuiFe45ZLsdY1MyGiU2BABEBAAG0K0VuZWEgQXJtYmFuZCBE
-+        ZXZvcHMgVGVhbSA8YXJtYmFuZEBlbmVhLmNvbT6JAT4EEwECACgFAlagAroCGwMF
-+        CQPCZwAGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEN6rkLp5irHRsG8H/2P2
-+        hO6jFHPRD1u8j9ufO9AxoInaEG9GeRjuvMc1tCUzovXvTs106TZ0as6MbAJ1S6u5
-+        E8UyQ1+VGepvdIJPXx3PCxLmUt6WIqmdOdqFrxIclohig/kardi3NfX1MBvlEV/c
-+        7Q51H43hrlMqMpqgY1Bm+53PLC4ujjlDJBtJOEU9dki319bUmng+gO9BAljDJFXv
-+        JqY6+P/er7IgCbRXUXYfyJzgjx9JwlKbmdQ8QnCrZjF/VHIW40/mu5IIJuFvuCti
-+        tcog+SNSlDmCOS+wE8CCojeuJqupIOcz9zypVGzeus/N5Q5EEFe7GYWYS/5NKUkE
-+        1TuuFeZKu5NJC5rkwIU=
-+        =7p60
-+        -----END PGP PUBLIC KEY BLOCK-----
-+      saltstack_repo_xenial_arm64: "http://linux.enea.com/saltstack/apt/ubuntu/16.04/arm64/2016.11 xenial main"
-   linux:
-     network:
-       interface:
++            mQENBFagAroBCADWboNIjuF6lB1mWv2+EbvqY3lKl5mLKhr2DnSUkKeHUPBv8gNM
++            qK8Q00AMIyPiyEhgjA+dWizZ+5aBgxoiY7oMeLJ2Xym36U/8SYq2BWd3SGCbMNoz
++            SJDxDUSM/HFVs6atF1M3DY9oN65hSVnu4uy5Tu6asf6k4rhAyk0z4+pRcPBCu2vq
++            mnGi3COM/+9PShrEKeVOx5W2vRJywUFuq8EDvQnRoJ0GvM28JiJIanw17YwIPxhg
++            BKZVpZjan5X+ihVMXwA2h/G/FS5Omhd50RqV6LWSYs94VJJgYqHx8UMm7izcxI+P
++            ct3IcbD195bPbJ+SbuiFe45ZLsdY1MyGiU2BABEBAAG0K0VuZWEgQXJtYmFuZCBE
++            ZXZvcHMgVGVhbSA8YXJtYmFuZEBlbmVhLmNvbT6JAT4EEwECACgFAlagAroCGwMF
++            CQPCZwAGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEN6rkLp5irHRsG8H/2P2
++            hO6jFHPRD1u8j9ufO9AxoInaEG9GeRjuvMc1tCUzovXvTs106TZ0as6MbAJ1S6u5
++            E8UyQ1+VGepvdIJPXx3PCxLmUt6WIqmdOdqFrxIclohig/kardi3NfX1MBvlEV/c
++            7Q51H43hrlMqMpqgY1Bm+53PLC4ujjlDJBtJOEU9dki319bUmng+gO9BAljDJFXv
++            JqY6+P/er7IgCbRXUXYfyJzgjx9JwlKbmdQ8QnCrZjF/VHIW40/mu5IIJuFvuCti
++            tcog+SNSlDmCOS+wE8CCojeuJqupIOcz9zypVGzeus/N5Q5EEFe7GYWYS/5NKUkE
++            1TuuFeZKu5NJC5rkwIU=
++            =7p60
++            -----END PGP PUBLIC KEY BLOCK-----
++        armband_mk_openstack:
++          name: armband_mk_openstack
++          enabled: '1'
++          url: 'http://linux.enea.com/apt-mk/${_param:linux_system_codename}/'
++          distributions: 'nightly'
++          components: '${_param:openstack_version}'
++          arches: 'arm64'
++          key: *armband_key
++        saltstack_armband:
++          name: saltstack_armband
++          enabled: '1'
++          url: 'http://linux.enea.com/saltstack/apt/ubuntu/16.04/arm64/2016.11'
++          distributions: '${_param:linux_system_codename}'
++          components: 'main'
++          arches: 'arm64'
++          key: *armband_key
+       subnets:
+         opnfv_maas_pxe:
+           name: ${_param:opnfv_maas_pxe_network_address}/24
 diff --git a/mcp/salt-formulas/maas/files/curtin_userdata_arm64_generic_xenial b/mcp/salt-formulas/maas/files/curtin_userdata_arm64_generic_xenial
 new file mode 100644
-index 0000000..96fd76f
+index 0000000..15d10a4
 --- /dev/null
 +++ b/mcp/salt-formulas/maas/files/curtin_userdata_arm64_generic_xenial
-@@ -0,0 +1,43 @@
+@@ -0,0 +1,35 @@
 +{%- from "maas/map.jinja" import cluster with context %}
 +{% raw %}
 +#cloud-config
@@ -134,14 +154,6 @@ index 0000000..96fd76f
 +{{endif}}
 +late_commands:
 +  maas: [wget, '--no-proxy', {{node_disable_pxe_url|escape.json}}, '--post-data', {{node_disable_pxe_data|escape.json}}, '-O', '/dev/null']
-+{% endraw %}
-+{%- if not cluster.saltstack_repo_key_arm64 == 'none' %}
-+{% set salt_repo_key = salt['hashutil.base64_b64encode'](cluster.saltstack_repo_key_arm64) %}
-+  apt_00_set_gpg: ["curtin", "in-target", "--", "sh", "-c", "echo '{{salt_repo_key}}' | base64 -d | apt-key add -"]
-+{%- endif %}
-+  apt_01_set_repo: ["curtin", "in-target", "--", "sh", "-c", "echo 'deb [arch=arm64] {{ cluster.saltstack_repo_xenial_arm64 }}' >> /etc/apt/sources.list"]
-+{% raw %}
-+  apt_03_update: ["curtin", "in-target", "--", "apt-get", "update"]
 +  salt_01_install: ["curtin", "in-target", "--", "apt-get", "-y", "install", "salt-minion"]
 +{% endraw %}
 +  salt_02_hostname_set: ["curtin", "in-target", "--", "echo", "{% raw %}{{node.hostname}}{% endraw %}.{{pillar.linux.system.domain}}"]
@@ -160,10 +160,10 @@ index 36e7f73..b7c1607 100644
 +/usr/share/salt-formulas/env: 0103-virtng-module-Extend-libvirt_domain.patch
 +/usr/share/salt-formulas/env: 0104-salt-control-virt-Extend-libvirt_domain.patch
 diff --git a/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/kvm.yml b/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/kvm.yml
-index 6491798..bf58f74 100644
+index ce7fc85..5e8d861 100644
 --- a/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/kvm.yml
 +++ b/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/infra/kvm.yml
-@@ -49,21 +49,33 @@ parameters:
+@@ -48,21 +48,33 @@ parameters:
          openstack.control:
            cpu: 4
            ram: 12288
@@ -197,7 +197,7 @@ index 6491798..bf58f74 100644
            disk_profile: xxlarge
            net_profile: default
          # stacklight.log:
-@@ -84,6 +96,9 @@ parameters:
+@@ -83,6 +95,9 @@ parameters:
          openstack.proxy:
            cpu: 2
            ram: 2048
@@ -20,10 +20,10 @@ Signed-off-by: Charalampos Kominos <charalampos.kominos@enea.com>
  1 file changed, 2 insertions(+), 2 deletions(-)
 
 diff --git a/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/openstack_compute.yml b/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/openstack_compute.yml
-index 278e55b..ff57ad3 100644
+index e07e611..f9d2896 100644
 --- a/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/openstack_compute.yml
 +++ b/mcp/reclass/classes/cluster/baremetal-mcp-ocata-common/openstack_compute.yml
-@@ -13,8 +13,8 @@ classes:
+@@ -12,8 +12,8 @@ classes:
    - system.linux.storage.loopback
    - system.glusterfs.client.cluster
    - system.nova.compute.cluster
diff --git a/patches/opnfv-fuel/0014-baremetal-linux-image-generic-hwe-16.04-edge.patch b/patches/opnfv-fuel/0014-baremetal-linux-image-generic-hwe-16.04-edge.patch
deleted file mode 100644 (file)
index 7f9ccc5..0000000
+++ /dev/null
@@ -1,50 +0,0 @@
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-: Copyright (c) 2017 Enea AB and others.
-:
-: All rights reserved. This program and the accompanying materials
-: are made available under the terms of the Apache License, Version 2.0
-: which accompanies this distribution, and is available at
-: http://www.apache.org/licenses/LICENSE-2.0
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
-Date: Sun, 5 Nov 2017 21:02:17 +0100
-Subject: [PATCH] baremetal: linux-image-generic-hwe-16.04-edge
-
-The Salt way to pin the kernel version is via its reclass model, by
-defining system.kernel.version. However, this only works with
-explicit kernel package names and not with meta package names
-(e.g. linux-image-4.13.0-9916-generic is expected instead of
-      linux-image-generic-hwe-16.04-edge, which would break the
-`linux_kernel_old_absent` state).
-
-For now, let's explicitly call `pkg.install` for the kernel and
-headers packages, using the meta package name to ensure automatic
-fallback to Ubuntu packages if Armband repos are missing,
-respectively to automatically pick up new kernel package names
-in case of meta + kernel packages bump later.
-
-JIRA: ARMBAND-280
-
-Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
----
- mcp/config/states/virtual_control_plane | 5 ++++-
- 1 file changed, 4 insertions(+), 1 deletion(-)
-
-diff --git a/mcp/config/states/virtual_control_plane b/mcp/config/states/virtual_control_plane
-index f326f4c..347b2b5 100755
---- a/mcp/config/states/virtual_control_plane
-+++ b/mcp/config/states/virtual_control_plane
-@@ -39,9 +39,12 @@ salt -C 'kvm* or cmp*' file.replace $debian_ip_source \
- salt -C 'kvm*' pkg.install bridge-utils
- salt -C '*' state.apply opnfv.adduser
- salt -C 'kvm*' state.apply linux.network
--salt -C 'cmp*' state.apply linux.system
-+salt -C 'kvm* or cmp*' state.apply linux.system
- salt -C 'cmp*' state.apply linux.network || true
- salt -C 'kvm*' state.apply armband.salt_minion || true
-+# NOTE(armband): Should be later moved to reclass as system.kernel.version
-+salt -C 'kvm* or cmp*' pkg.install \
-+  linux-image-generic-hwe-16.04-edge,linux-headers-generic-hwe-16.04-edge
- salt -C 'kvm* or cmp*' system.reboot
- wait_for 90 "! salt -C 'kvm* or cmp*' test.ping | " \
-   "tee /dev/stderr | grep -Fq 'Not connected'"