========
This document describes how to install the Brahmaputra 3.0 release of
-OPNFV when using Fuel as a deployment tool, with an AArch64 (only) target node pool.
+OPNFV when using Fuel as a deployment tool, with an AArch64 (only) target
+node pool.
Introduction
============
------------------------
First of all, the Fuel deployment ISO image needs to be retrieved, the
-Fuel .iso image of the AArch64 Brahmaputra release can be found at *Reference: 2*
+ArmbandFuel .iso image of the AArch64 Brahmaputra release can be found
+at *Reference: 2*
Building the ISO image
----------------------
-Alternatively, you may build the .iso from source by cloning the
-opnfv/armband git repository. To retrieve the repository for the AArch64 Brahmaputra 3.0 release use the following command:
+Alternatively, you may build the ArmbandFuel .iso from source by cloning the
+opnfv/armband git repository. To retrieve the repository for the AArch64
+Brahmaputra 3.0 release use the following command:
-$git clone https://<linux foundation uid>@gerrit.opnf.org/gerrit/armband
+.. code-block:: bash
-Check-out the Brahmaputra stable branch to set the branch to the
+ $ git clone https://gerrit.opnfv.org/gerrit/armband
+
+Check-out the Brahmaputra release tag to set the HEAD to the
baseline required to replicate the Brahmaputra release:
-$ git checkout stable/brahmaputra
+.. code-block:: bash
+
+ $ git checkout brahmaputra.3.0
Go to the armband directory and build the .iso:
-$ cd armband; make release
+.. code-block:: bash
+
+ $ cd armband
+ $ make release
For more information on how to build, please see *Reference: 14*
Other preparations
------------------
-
Next, familiarize yourself with Fuel by reading the following documents:
- Fuel planning guide, please see *Reference: 8*
=====================
The following minimum hardware requirements must be met for the
-installation of Brahmaputra using Fuel:
+installation of AArch64 Brahmaputra 3.0 using Fuel:
+--------------------+------------------------------------------------------+
| **HW Aspect** | **Requirement** |
| | |
+====================+======================================================+
-| **# of nodes** | Minimum 5 (3 for non redundant deployment): |
-| | |
-| | - 1 Fuel deployment master (may be virtualized) |
+| **AArch64 nodes** | Minimum 5 (3 for non redundant deployment): |
| | |
| | - 3(1) Controllers (1 colocated mongo/ceilometer |
| | role, 2 Ceph-OSD roles) |
| | - 1 Compute (1 co-located Ceph-OSD role) |
| | |
+--------------------+------------------------------------------------------+
-| **CPU** | Minimum 1 socket x86_AMD64 with Virtualization |
+| **CPU** | Minimum 1 socket AArch64 (ARMv8) with Virtualization |
| | support |
+--------------------+------------------------------------------------------+
| **RAM** | Minimum 16GB/server (Depending on VNF work load) |
| **Disk** | Minimum 256GB 10kRPM spinning disks |
| | |
+--------------------+------------------------------------------------------+
+| **Firmware** | UEFI compatible (e.g. EDK2) with PXE support |
++--------------------+------------------------------------------------------+
| **Networks** | 4 Tagged VLANs (PUBLIC, MGMT, STORAGE, PRIVATE) |
| | |
| | 1 Un-Tagged VLAN for PXE Boot - ADMIN Network |
| | or spread out over multiple NICs as your hardware |
| | supports. |
+--------------------+------------------------------------------------------+
+| **1 x86_64 node** | - 1 Fuel deployment master, x86 (may be virtualized) |
++--------------------+------------------------------------------------------+
Help with Hardware Requirements
===============================
#. In the "Network Setup" section - Configure DHCP/Static IP information for your FUEL node - For example, ETH0 is 10.20.0.2/24 for FUEL booting and ETH1 is DHCP/Static in your corporate/lab network (see figure below).
- - Configuration of ETH1 interface for connectivity into your corporate/lab network is mandatory. You need to have internet access over this interface.
- Due to the architecture of ports.ubuntu.com mirror, currently Fuel cannot create a full local mirror of all required AArch64 packages. Internet access is needed to deploy with Fuel on AArch64 nodes.
+ - **NOTE**: ArmbandFuel@OPNFV requires internet connectivity during bootstrap
+ image building, due to missing arm64 (AArch64) packages in the partial
+ local Ubuntu mirror (consequence of ports.ubuntu.com mirror architecture).
+
+ - Configuration of ETH1 interface for connectivity into your corporate/lab
+ network is mandatory, as internet connection is required during deployment.
.. figure:: img/fuelmenu2.png
.. figure:: img/fuelmenu4.png
+#. **DO NOT CHANGE** anything in "Bootstrap Image" section (see figure below).
-#. OPTION TO ENABLE PROXY SUPPORT - In the "Bootstrap Image" section (see figure below), edit the following fields to define a proxy. (**NOTE:** cannot be used in tandem with local repository support)
+ In ArmbandFuel@OPNFV, this data is **NOT** actually used for bootstrap
+ image building. Any change here will replace the configuration from
+ the OPNFV bootstrap build scripts and will lead to a failed bootstrap
+ image build.
- - Navigate to "HTTP proxy" and enter your http proxy address
-
- - Select <Check> and press [Enter]
+ **NOTE:** Cannot be used in tandem with local repository support.
.. figure:: img/fuelmenu5.png
#. Start the installation.
- - Select Quit Setup and press Save and Quit.
+ - Press <F8> or select Quit Setup and press Save and Quit.
- Installation starts, wait until the login screen is shown.
Compute/Control/Storage blades (nested or real) with a PXE booting
scheme so that the FUEL Master can pick them up for control.
+**NOTE**: AArch64 target nodes are expected to support PXE booting an
+EFI binary, i.e. an EFI-stubbed GRUB2 bootloader.
+
+**NOTE**: UEFI (EDK2) firmware is **highly** recommended, becoming
+the **de facto** standard for ARMv8 nodes.
+
#. Enable PXE booting
- - For every controller and compute server: enable PXE Booting as the first boot device in the BIOS boot order menu and hard disk as the second boot device in the same menu.
+ - For every controller and compute server: enable PXE Booting as the first boot device in the UEFI (EDK2) boot order menu and hard disk as the second boot device in the same menu.
#. Reboot all the control and compute blades.
.. figure:: img/nodes.png
-Target specific configuration
------------------------------
-
-#. AMD Softiron
-
- For these targets, "rx-vlan-filter" offloading has to be turned off on the interface destined for OpenStack traffic (not the interface used for PXE boot).
- For now this setting cannot be toggled from Fuel GUI, so it has to be done form the console.
-
- - From Fuel master console identify target nodes admin IPs (see figure below).
+Install additional Plugins/Features on the FUEL node
+----------------------------------------------------
- .. figure:: img/fuelconsole1.png
+#. SSH to your FUEL node (e.g. root@10.20.0.2 pwd: r00tme)
- - SSH into each of the target nodes and disable rx-vlan-filter on the physical interface allocated for OpenStack traffic (see figure below).
+#. Select wanted plugins/features from the /opt/opnfv/ directory.
- .. figure:: img/softiron1.png
+#. Install the wanted plugin with the command:
- - Repeat the step above for all AMD Softiron nodes in the POD.
+ .. code-block:: bash
+ $ fuel plugins --install /opt/opnfv/<plugin-name>-<version>.<arch>.rpm
-Install additional Plugins/Features on the FUEL node
-----------------------------------------------------
+ Expected output (see figure below):
-#. SSH to your FUEL node (e.g. root@10.20.0.2 pwd: r00tme)
+ .. code-block:: bash
-#. Select wanted plugins/features from the /opt/opnfv/ directory.
+ Plugin ....... was successfully installed.
-#. Install the wanted plugin with the command "fuel plugins --install /opt/opnfv/<plugin-name>-<version>.<arch>.rpm"
- Expected output: "Plugin ....... was successfully installed." (see figure below)
+ **NOTE**: AArch64 Brahmaputra 3.0 ships only with Opendaylight plugin,
+ see *Reference 15*.
.. figure:: img/plugin_install.png
- Select "Neutron with ML2 plugin"
- - Select "Neutron with tunneling segmentation" (Required when using the ODL or ONOS plugins)
+ - Select "Neutron with tunneling segmentation" (Required when using the ODL plugin)
- Press <Next>
#. Select "additional services" you wish to install.
- - Check option "Install Celiometer (OpenStack Telemetry)" and press <Next>
+ - Check option "Install Ceilometer (OpenStack Telemetry)" and press <Next>
#. Create the new environment.
- Enable and configure the plugins of your choice
- .. figure:: img/plugins.png
+ .. figure:: img/plugins_aarch64.png
Allocate nodes to environment and assign functional roles
---------------------------------------------------------
- Click on the <+Add Nodes> button
- - Check <Controller>, <Telemetry - MongoDB> and optionally an SDN Controller role (OpenDaylight controller/ONOS) in the Assign Roles Section.
+ - Check <Controller>, <Telemetry - MongoDB> and optionally an SDN Controller role (OpenDaylight controller) in the Assign Roles Section.
- Check one node which you want to act as a Controller from the bottom half of the screen
.. figure:: img/interfaceconf.png
-
-OPTIONAL - Set Local Mirror Repos
+OPTIONAL - UNTESTED - Set Local Mirror Repos
---------------------------------
+**NOTE**: AArch64 Brahmaputra 3.0 does not fully support local Ubuntu mirrors,
+or at least does not ship with arm64 packages in local repos by default.
+In order to use local (partial) Ubuntu mirrors, one should add arm64 packages
+by hand to the existing amd64 mirrors and re-generate repo metadata.
+Local MOS/Auxiliary repos contain packages for both amd64 and arm64.
+
+**NOTE**: Below instruction assume you already added (by hand) arm64
+Ubuntu necessary packages to the local repository!
+
The following steps can be executed if you are in an environment with
no connection to the Internet. The Fuel server delivers a local repo
that can be used for installation / deployment of openstack.
- Click <Save Settings> at the bottom to Save your changes
+Target specific configuration
+-----------------------------
+
+#. [AArch64 specific] Configure MySQL WSREP SST provider
+
+ **NOTE**: This option is only available for ArmbandFuel@OPNFV, since it
+ currently only affects AArch64 targets (see *Reference 15*).
+
+ When using some AArch64 platforms as controller nodes, WSREP SST
+ synchronisation using default backend provider (xtrabackup-v2) might fail,
+ so a mechanism that allows selecting a different WSREP SST provider
+ has been introduced.
+
+ In the FUEL UI of your Environment, click the <Settings> tab, click
+ <OpenStack Services> on the left side pane (see figure below), then
+ select one of the following options:
+
+ - xtrabackup-v2 (default provider, AArch64 stability issues);
+
+ - rsync (AArch64 validated, better or comparable speed to xtrabackup,
+ takes the donor node offline during state transfer);
+
+ - mysqldump (untested);
+
+ .. figure:: img/fuelwsrepsst.png
+
+#. Set up targets for provisioning with non-default "Offloading Modes"
+
+ Some target nodes may require additional configuration after they are
+ PXE booted (bootstrapped); the most frequent changes are in defaults
+ for ethernet devices' "Offloading Modes" settings (e.g. some targets'
+ ethernet drivers may strip VLAN traffic by default).
+
+ If your target ethernet drivers have wrong "Offloading Modes" defaults,
+ in "Configure interfaces" page (described above), expand affected
+ interface's "Offloading Modes" and [un]check the relevant settings
+ (see figure below):
+
+ .. figure:: img/offloadingmodes.png
+
+#. Set up targets for "Verify Networks" with non-default "Offloading Modes"
+
+ **NOTE**: Check *Reference 15* for an updated and comprehensive list of
+ known issues and/or limitations, including "Offloading Modes" not being
+ applied during "Verify Networks" step.
+
+ Setting custom "Offloading Modes" in Fuel GUI will only apply those settings
+ during provisiong and **not** during "Verify Networks", so if your targets
+ need this change, you have to apply "Offloading Modes" settings by hand
+ to bootstrapped nodes.
+
+ **E.g.**: Our driver has "rx-vlan-filter" default "on" (expected "off") on
+ the Openstack interface(s) "eth1", preventing VLAN traffic from passing
+ during "Verify Networks".
+
+ - From Fuel master console identify target nodes admin IPs (see figure below):
+
+ .. code-block:: bash
+
+ $ fuel nodes
+
+ .. figure:: img/fuelconsole1.png
+
+ - SSH into each of the target nodes and disable "rx-vlan-filter" on the
+ affected physical interface(s) allocated for OpenStack traffic (eth1):
+
+ .. code-block:: bash
+
+ $ ssh root@10.20.0.6 ethtool -K eth1 rx-vlan-filter off
+
+ - Repeat the step above for all affected nodes/interfaces in the POD.
+
Verify Networks
---------------
- Allow tests to run and investigate results where appropriate
+ - Check *Reference 15* for known issues / limitations on AArch64, like
+ unsupported migration tests when using a GICv3 interrupt controller
+
.. figure:: img/health.png
References
OPNFV
-----
-1) `OPNFV Home Page <www.opnfv.org>`_
+1) `OPNFV Home Page <http://www.opnfv.org>`_
2) `OPNFV documentation- and software downloads <https://www.opnfv.org/software/download>`_
Code Review / armband.git / blobdiff