releng-xci.git
7 years agoRemove opnfv vm from known_hosts 83/53883/1
Fatih Degirmenci [Thu, 15 Mar 2018 14:32:40 +0000 (15:32 +0100)]
Remove opnfv vm from known_hosts

Change-Id: If53b51fc26d642afa39c5e618afcd23fab395acc
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoMerge "Do not grep log text to determine failure"
Fatih Degirmenci [Wed, 14 Mar 2018 18:58:39 +0000 (18:58 +0000)]
Merge "Do not grep log text to determine failure"

7 years agoMerge "xci: Merge configure-network and syncronize-time roles"
Markos Chandras [Wed, 14 Mar 2018 08:01:27 +0000 (08:01 +0000)]
Merge "xci: Merge configure-network and syncronize-time roles"

7 years agoMerge "xci: config: evn-vars: Dynamically determine the Ansible version"
Markos Chandras [Wed, 14 Mar 2018 07:57:05 +0000 (07:57 +0000)]
Merge "xci: config: evn-vars: Dynamically determine the Ansible version"

7 years agoMerge "xci: Fix group for the OPFNV host"
Markos Chandras [Wed, 14 Mar 2018 07:56:52 +0000 (07:56 +0000)]
Merge "xci: Fix group for the OPFNV host"

7 years agoMerge "Set VM node names and log the installer version based on installer"
Fatih Degirmenci [Tue, 13 Mar 2018 23:30:47 +0000 (23:30 +0000)]
Merge "Set VM node names and log the installer version based on installer"

7 years agoMerge "xci: osa: Switch git repositories to github.com"
Markos Chandras [Tue, 13 Mar 2018 16:39:44 +0000 (16:39 +0000)]
Merge "xci: osa: Switch git repositories to github.com"

7 years agoxci: Merge configure-network and syncronize-time roles 65/53665/3
Markos Chandras [Tue, 13 Mar 2018 10:30:16 +0000 (10:30 +0000)]
xci: Merge configure-network and syncronize-time roles

Those roles were too small and only used as part of the host
bootstrapping process. As such, we merge them into a common
'bootstrap-host' role which can be used to prepare the hosts
after the initial deployment

Change-Id: Ifc84cf40b98ced91b31aae699dc28e9642380550
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: Fix group for the OPFNV host 73/53573/4
Markos Chandras [Tue, 13 Mar 2018 16:28:46 +0000 (16:28 +0000)]
xci: Fix group for the OPFNV host

Ansible gets upset if we use the same name for a host and a group so
add the OPNFV host to the deployment group. This fixes the following
warning:

[WARNING]: Found both group and host with same name: opnfv

Change-Id: Idbaff712d40e8234721332aee6d10d5ba44c62ce
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoMerge changes from topics 'allow-pass-ansible-arguments', 'allow-pass-ansible-argumen...
Markos Chandras [Tue, 13 Mar 2018 16:20:20 +0000 (16:20 +0000)]
Merge changes from topics 'allow-pass-ansible-arguments', 'allow-pass-ansible-arguments-force-verify'

* changes:
  xci: OSA: Fix warning about missing inventory file
  xci: Pass the XCI_PATH variable to all Ansible calls
  xci: Rename XCI_ANSIBLE_VERBOSITY to XCI_ANSIBLE_PARAMS

7 years agoxci: osa: Switch git repositories to github.com 67/53667/1
Markos Chandras [Tue, 13 Mar 2018 12:03:55 +0000 (12:03 +0000)]
xci: osa: Switch git repositories to github.com

CI sometimes fails with the following error

"msg": "fatal: unable to access 'https://git.openstack.org/openstack/dragonflow/':
Failed to connect to git.openstack.org port 443: Connection timed out"

We assume that the upstream OpenStack mirrors may get very busy at times
or the connectivity between the CI and the OpenStack repos is not very
stable. As such, lets switch to github.com for the majority of OSA
repositories such as the roles and the services which may improve the
situation.

Change-Id: Ia4668d692077a785c76adeda128eb4cf9f4516a7
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoDo not grep log text to determine failure 01/53601/2
Fatih Degirmenci [Mon, 12 Mar 2018 20:10:48 +0000 (21:10 +0100)]
Do not grep log text to determine failure

Pipefail should be sufficient to determine the failure so this
change gets rid of grepping the logs for failure and unreachable
to leave the evaluation to bash itself.

Change-Id: Ie928438c1503b086159276af4308f5549b58cb71
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoSet VM node names and log the installer version based on installer 05/53605/1
Fatih Degirmenci [Mon, 12 Mar 2018 22:22:55 +0000 (23:22 +0100)]
Set VM node names and log the installer version based on installer

The VM nodes are named using OpenStack terminology but since we
now have Kubernetes scenario, it is appropriate to name the nodes
accordingly.

Apart from that, we normally log the version of OSA which is not
correct for kubernetes deployments. Logging Kubespray version would
be correct for this scenario.

Change-Id: I3964574c8592450f9fa3126f19e27bb78fd9d174
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoMerge "Combine vars setting installer type in INSTALLER_TYPE"
Fatih Degirmenci [Mon, 12 Mar 2018 18:02:33 +0000 (18:02 +0000)]
Merge "Combine vars setting installer type in INSTALLER_TYPE"

7 years agoMerge "Combine vars setting installer type in INSTALLER_TYPE"
Fatih Degirmenci [Mon, 12 Mar 2018 18:02:27 +0000 (18:02 +0000)]
Merge "Combine vars setting installer type in INSTALLER_TYPE"

7 years agoremove .ansible directory 77/53577/3
Periyasamy Palanisamy [Mon, 12 Mar 2018 15:16:13 +0000 (16:16 +0100)]
remove .ansible directory

It's needed to remove .ansible directory which could be a leftovers by
some other ansible execution enviroment which would affect XCI
deployment.

Change-Id: Ic334ef1ca53878721762b23b238fe9a8aeefa0e0
Signed-off-by: Periyasamy Palanisamy <periyasamy.palanisamy@ericsson.com>
7 years agoxci: config: evn-vars: Dynamically determine the Ansible version 67/53567/1
Markos Chandras [Thu, 8 Mar 2018 16:55:52 +0000 (16:55 +0000)]
xci: config: evn-vars: Dynamically determine the Ansible version

There is no good reason why we should pin the Ansible version without
the ability to override it. OpenStack Ansible is doing a great work
testing Ansible releases as early as possible so we could benefit from
that work and use whatever Ansible our pinned OSA uses.

Change-Id: I4677dbefc68f7140417ece34d013ca9674baab43
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoCombine vars setting installer type in INSTALLER_TYPE 61/53561/2
Fatih Degirmenci [Mon, 12 Mar 2018 13:35:22 +0000 (14:35 +0100)]
Combine vars setting installer type in INSTALLER_TYPE

Change-Id: Ie196d1df537d09f0f91e43ab5e0305a45d543815
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoCombine vars setting installer type in INSTALLER_TYPE 59/53559/2
Fatih Degirmenci [Mon, 12 Mar 2018 13:34:18 +0000 (14:34 +0100)]
Combine vars setting installer type in INSTALLER_TYPE

Change-Id: I330bc036f901d4ba61bc94ee6e085cadf54b4d8b
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoxci: OSA: Fix warning about missing inventory file 19/53519/1
Markos Chandras [Mon, 12 Mar 2018 10:40:39 +0000 (10:40 +0000)]
xci: OSA: Fix warning about missing inventory file

This is similar to 346079ea1b8dbda0c5e282c18f30cbac7e907d1e
("xci: Fix warning about missing inventory file")

This playbook only affects localhost so make it explicit. Fixes:

[WARNING]: Host file not found: inventory
[WARNING]: provided hosts list is empty, only localhost is available

Change-Id: If8548a36498ea3191de33cff94acbaee6ed04ce3
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: Pass the XCI_PATH variable to all Ansible calls 09/53509/2
Markos Chandras [Mon, 12 Mar 2018 09:50:06 +0000 (09:50 +0000)]
xci: Pass the XCI_PATH variable to all Ansible calls

The XCI_PATH variable is used by various playbooks to find the root
directory of the XCI repository so it's much cleaner to pass it on every
Ansible call.

Change-Id: Ifc0b77fa767e48bd1dc6daa44c6251c02983fd4d
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: Rename XCI_ANSIBLE_VERBOSITY to XCI_ANSIBLE_PARAMS 07/53507/1
Markos Chandras [Mon, 12 Mar 2018 09:47:55 +0000 (09:47 +0000)]
xci: Rename XCI_ANSIBLE_VERBOSITY to XCI_ANSIBLE_PARAMS

We don't quite need a specific env variable just for Ansible
verbosity so we can rename this variable to make it clear that
it can be used to pass any Ansible option to XCI.

Change-Id: Ie20517d4b563bfc6daeb27848168d36da7014cee
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: Drop the python-crypto dependency 05/53505/1
Markos Chandras [Mon, 12 Mar 2018 09:40:10 +0000 (09:40 +0000)]
xci: Drop the python-crypto dependency

The upstream pw-token-gen tool doesn't need python-crypto anymore since
e9f957861b4160640f6debb2b939084ec43b43b2 ("Make pw-token-gen.py more
random") so we no longer need to install that package.

Change-Id: Ib53f246db999ff8ecfed2e3f62143c780c483fbd
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoMerge "xci: Fix warning about missing inventory file"
Markos Chandras [Mon, 12 Mar 2018 08:44:12 +0000 (08:44 +0000)]
Merge "xci: Fix warning about missing inventory file"

7 years agoMerge "xci: roles: configure-network: Avoid passing empty string to module"
Markos Chandras [Mon, 12 Mar 2018 08:44:06 +0000 (08:44 +0000)]
Merge "xci: roles: configure-network: Avoid passing empty string to module"

7 years agoMerge changes from topic 'set-proper-constrains'
Markos Chandras [Mon, 12 Mar 2018 08:44:00 +0000 (08:44 +0000)]
Merge changes from topic 'set-proper-constrains'

* changes:
  xci: OSA: Ensure proper constrains are applied to pip installations
  xci: Move functest required packages to the functest role

7 years agoMerge "xci: scripts: vm: Update package manager cache and ignore errors"
Markos Chandras [Mon, 12 Mar 2018 08:43:53 +0000 (08:43 +0000)]
Merge "xci: scripts: vm: Update package manager cache and ignore errors"

7 years agoxci: scripts: vm: Update package manager cache and ignore errors 49/53449/1
Markos Chandras [Sat, 10 Mar 2018 09:59:00 +0000 (09:59 +0000)]
xci: scripts: vm: Update package manager cache and ignore errors

We need to make sure that the local package cache is updated before we
try to install packages. Moreover, from time to time, something in the
repos may be broken so we don't want to break the CI job. As such, make
that step non-fatal as simply send the output to stdout so we can see
what's wrong.

Change-Id: I148e3b5435c604bb0d54339935255740c8ce07c0
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: roles: configure-network: Avoid passing empty string to module 25/53425/3
Markos Chandras [Fri, 9 Mar 2018 16:31:13 +0000 (16:31 +0000)]
xci: roles: configure-network: Avoid passing empty string to module

The package module gets upset if we pass it an empty string as a package
name

Mar 09 15:15:33 TASK [configure-network : Ensure networking packages are present] **************
Mar 09 15:15:49 fatal: [opnfv]: FAILED! => {"changed": false, "failed":
true, "msg": "No package matching '' found available, installed or
updated", "rc": 126, "results": ["iproute-3.10.0-87.el7.x86_64 providing
iproute is already installed", "No package matching '' found available,
installed or updated"]}

As such, lets pass an existing package for non-Debian distros to make
the module happy.

Change-Id: Ib1128057097193278cfe50f79daf35332b54e781
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: OSA: Ensure proper constrains are applied to pip installations 05/53305/15
Markos Chandras [Thu, 8 Mar 2018 12:37:07 +0000 (12:37 +0000)]
xci: OSA: Ensure proper constrains are applied to pip installations

We need to make sure that the pip packages that we install are
compatible with the OSA components.

Change-Id: I87e80dc9b8fb862d9f9039d835b0908f752de3ca
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: Move functest required packages to the functest role 89/53389/8
Markos Chandras [Fri, 9 Mar 2018 09:17:16 +0000 (09:17 +0000)]
xci: Move functest required packages to the functest role

The docker packages that we install in the OPNFV VM are needed by
functest so add them to the related role.

Change-Id: I6ebe76fd030859f757d41ecf20c30ab76888ee9c
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoMerge "bifrost: Do not run virsh commands if it's not installed"
Markos Chandras [Sat, 10 Mar 2018 09:48:28 +0000 (09:48 +0000)]
Merge "bifrost: Do not run virsh commands if it's not installed"

7 years agoMerge "bifrost: Move DIB configuration variables to bifrost"
Markos Chandras [Sat, 10 Mar 2018 09:48:17 +0000 (09:48 +0000)]
Merge "bifrost: Move DIB configuration variables to bifrost"

7 years agoMerge "bifrost: Optimize the OPNFV bifrost playbook"
Markos Chandras [Sat, 10 Mar 2018 09:48:08 +0000 (09:48 +0000)]
Merge "bifrost: Optimize the OPNFV bifrost playbook"

7 years agoxci: Fix warning about missing inventory file 15/53415/2
Markos Chandras [Fri, 9 Mar 2018 14:24:00 +0000 (14:24 +0000)]
xci: Fix warning about missing inventory file

We no longer have a global inventory file and we don't need one right
now since the bootstrapping opearations happen on the localhost anyway
so we could be explicit and get rid of the following warning:

[WARNING]: Host file not found: inventory
[WARNING]: provided hosts list is empty, only localhost is available

Change-Id: I5cfe7b0971397665cf9eae9c54985c44c4506449
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoMerge "xci: roles: configure-network: Wait for connection when configuring the network"
Fatih Degirmenci [Fri, 9 Mar 2018 17:20:46 +0000 (17:20 +0000)]
Merge "xci: roles: configure-network: Wait for connection when configuring the network"

7 years agobifrost: Optimize the OPNFV bifrost playbook 91/53391/3
Markos Chandras [Fri, 9 Mar 2018 10:09:42 +0000 (10:09 +0000)]
bifrost: Optimize the OPNFV bifrost playbook

We could combine several plays into a single one when they are executed
to the same set of nodes. This has the added benefit that we avoid all
the Ansible warm-up operations when a new play is started.

Change-Id: Ibcad07d1b74f1d7cc5a9c60e4b2a6130fb6178fd
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agobifrost: Do not run virsh commands if it's not installed 19/53419/1
Markos Chandras [Fri, 9 Mar 2018 14:37:16 +0000 (14:37 +0000)]
bifrost: Do not run virsh commands if it's not installed

Check if 'virsh' is available before trying to remove the VMs:

Fixes the following warning

./scripts/destroy-env.sh: line 36: virsh: command not found
./scripts/destroy-env.sh: line 37: virsh: command not found
./scripts/destroy-env.sh: line 36: virsh: command not found
./scripts/destroy-env.sh: line 37: virsh: command not found
./scripts/destroy-env.sh: line 36: virsh: command not found
./scripts/destroy-env.sh: line 37: virsh: command not found

Change-Id: I9c82c8b8b40462584c6324871284ba12d660ae9d
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agobifrost: Move DIB configuration variables to bifrost 13/53413/1
Markos Chandras [Fri, 9 Mar 2018 14:15:16 +0000 (14:15 +0000)]
bifrost: Move DIB configuration variables to bifrost

The DIB variables only make sense on bifrost so they should live in the
bifrost provision script.

Change-Id: I05baca7b9c6889a0d401a9288998991324d270af
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: roles: configure-network: Wait for connection when configuring the network 35/53335/7
Markos Chandras [Thu, 8 Mar 2018 19:51:51 +0000 (19:51 +0000)]
xci: roles: configure-network: Wait for connection when configuring the network

Ansible some times loses connection with the host after we configure its
network interface. This shouldn't happen since the host is actually up
and running. As a result of which, we need to fire-and-forget the
restart of the network service and then try to reach the SSH port. We
also need to make sure that all required packages are installed.
Finally, we drop the 'ignore_errors' parameter since we really need to
know when the networking configuration has failed.

Change-Id: I366192737b5c4e01964eaf187396eababbdc808e
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoMerge "xci: Rename provision-vm-nodes playbook"
Markos Chandras [Fri, 9 Mar 2018 10:47:50 +0000 (10:47 +0000)]
Merge "xci: Rename provision-vm-nodes playbook"

7 years agoMerge "Add subproject INFO.yaml"
Fatih Degirmenci [Fri, 9 Mar 2018 10:24:27 +0000 (10:24 +0000)]
Merge "Add subproject INFO.yaml"

7 years agoxci: Rename provision-vm-nodes playbook 85/53385/1
Markos Chandras [Fri, 9 Mar 2018 09:51:52 +0000 (09:51 +0000)]
xci: Rename provision-vm-nodes playbook

The provision-vm-nodes playbook doesn't do any VM provisioning. It
basically just bootstraps the bifrost repository so rename it to
make it clear what this is about.

Change-Id: I1c10bc6135686c5197d553a885412a94312a41bc
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agobifrost: Ensure OS images are present in the PXE directory 01/53301/1
Markos Chandras [Thu, 8 Mar 2018 11:39:42 +0000 (11:39 +0000)]
bifrost: Ensure OS images are present in the PXE directory

The XCI deployment script may wipe the PXE directories so we need
to place the OS images after this has happened so we don't download them
every time.

Change-Id: Iccf943bc75592e7f5dddea59dfee14b2b080e7ef
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoAdd subproject INFO.yaml 29/52729/4
Fatih Degirmenci [Tue, 27 Feb 2018 11:39:56 +0000 (11:39 +0000)]
Add subproject INFO.yaml

This change adds INFO.yaml for releng-xci Gerrit Project which is
constructed by subset of committers of Releng project who are active in
xci. New committers to releng-xci Gerrit Project will be nominated
on Gerrit via separate changes and voted by releng-xci committers that
are listed within releng-xci/INFO.yaml file as of today.

A single mail with all the nominations will be sent to opnfv-tsc and
opnfv-tech-discuss mailing lists announcing nominations. A followup
mail will also be sent to announce the results.

Change-Id: I1d49585276304b29ac86b03aff5fd45974a02ccc
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agobifrost: Use the pre-built XCI OS images instead of dib 53/53253/1
Markos Chandras [Wed, 7 Mar 2018 16:36:03 +0000 (16:36 +0000)]
bifrost: Use the pre-built XCI OS images instead of dib

We already have images for all 3 distros which we could simply re-use
for all the XCI VM deployments instead of building new ones everytime
with DIB. The images will be copied to the new VM from the cache
directory if they are available otherwise we will simply download them
during the XCI execution phase.

Change-Id: I2a8391650558511668654c6b54a10db316f867a2
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoMerge "Bump bifrost sha and set UPPER_CONSTRAINTS_FILE"
Markos Chandras [Wed, 7 Mar 2018 17:40:09 +0000 (17:40 +0000)]
Merge "Bump bifrost sha and set UPPER_CONSTRAINTS_FILE"

7 years agoMerge "Remove opnfv vm from known_hosts"
Fatih Degirmenci [Wed, 7 Mar 2018 16:02:20 +0000 (16:02 +0000)]
Merge "Remove opnfv vm from known_hosts"

7 years agoBump bifrost sha and set UPPER_CONSTRAINTS_FILE 39/53239/2
Fatih Degirmenci [Wed, 7 Mar 2018 13:11:17 +0000 (14:11 +0100)]
Bump bifrost sha and set UPPER_CONSTRAINTS_FILE

The lack of UC means that pip is attempting to install libvirt-python 4.1.0,
which is failing on Ubuntu due to
    https://bugs.launchpad.net/openstack-requirements/+bug/1753539.
Currently upper-constraints limits libvirt-python to 4.0.0, which does not
have this issue.

Change-Id: Idc97a0d9928cebafc9167614723236d2584cc93f
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoxci: Make few of OpenStack-Ansible components optional 93/52993/4
Markos Chandras [Fri, 2 Mar 2018 14:06:53 +0000 (14:06 +0000)]
xci: Make few of OpenStack-Ansible components optional

Ironic and Horizon are not quite needed for a functional deployment
and they are not currently required by functest so we can remove them
from the default deployment.

Change-Id: I171483f7b774951f84687529e98cb519afa48043
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: Bump SHAs to the latest stable/queens 91/52991/1
Fatih Degirmenci [Fri, 2 Mar 2018 13:49:13 +0000 (13:49 +0000)]
xci: Bump SHAs to the latest stable/queens

Rocky is still in early stages of development so bump SHAs to the
latest version for Queens.

Change-Id: I0721e6188889c3548c266c042414397385b85ca2
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoMerge "Pin kubespray to working sha"
Fatih Degirmenci [Fri, 2 Mar 2018 10:37:13 +0000 (10:37 +0000)]
Merge "Pin kubespray to working sha"

7 years agoPin kubespray to working sha 27/52927/2
Fatih Degirmenci [Thu, 1 Mar 2018 10:02:22 +0000 (10:02 +0000)]
Pin kubespray to working sha

Kubespray works fine on supported distros so it is important we pin
the sha and bump it in a controlled manner since people might be trying
xci/k8 scenario and we want them to have working versions.

Apart from that, scenario specific verify/merge jobs are being created
at the moment and we need a working version to test the CI/testing side
of things.

Change-Id: I8702ed697057386449dc99d8cd14391de18f345a
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoRemove opnfv vm from known_hosts 49/52949/1
Fatih Degirmenci [Thu, 1 Mar 2018 18:05:17 +0000 (18:05 +0000)]
Remove opnfv vm from known_hosts

Change-Id: I21998a31a26a201d767734d574ade8db7fd50c92
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoDo not fail the if virsh command fails 43/52943/1
Fatih Degirmenci [Thu, 1 Mar 2018 14:31:03 +0000 (14:31 +0000)]
Do not fail the if virsh command fails

Change-Id: Id9c55b45719399c6c78c0d3d8b97926a2907b7f8
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoos-odl-bgpvpn scenario support 05/48705/14
Periyasamy Palanisamy [Tue, 12 Dec 2017 09:22:43 +0000 (10:22 +0100)]
os-odl-bgpvpn scenario support

- Integrate XCI with outband od-odl-bgpvpn role
- Install python-neutronclient on opnfv vm for the
  openstack bgpvpn specific cli commands

Change-Id: Ib737349e2b2429bd366881f1e3657daf8c5c30ac
Signed-off-by: Periyasamy Palanisamy <periyasamy.palanisamy@ericsson.com>
7 years agoBe quiet while downloading images 57/52857/3
Fatih Degirmenci [Wed, 28 Feb 2018 18:48:25 +0000 (18:48 +0000)]
Be quiet while downloading images

Change-Id: I8848d5bae22a408482d51a84636d230d25ca22be
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agofunctest: Move run-functest.sh execution out of ansible 17/52817/4
Fatih Degirmenci [Wed, 28 Feb 2018 11:37:52 +0000 (11:37 +0000)]
functest: Move run-functest.sh execution out of ansible

Ansible throws out the log at the end of task execution which
makes it hard to read. This change renames the role to prepare-functest
and then takes the script execution out which will be executed by
releng/jjb/xci/xci-run-functest.sh.

Change-Id: Icf399ce4f04357814ed7109cd11113a9decddc50
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoEnsure the ssh config files are created per distro 61/52861/2
Fatih Degirmenci [Wed, 28 Feb 2018 18:25:14 +0000 (18:25 +0000)]
Ensure the ssh config files are created per distro

If 2 jobs for different distros start on the same host, the
generated xci-vm-config files might be rewritten by both jobs
causing trouble for the other. This change creates ssh config
per distro to prevent potential issues.

Change-Id: If6c0a86fd51bf3ba920e9206849ed1537894c7e1
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoFix the proxy command and hostname for opnfv vm in ssh config 37/52837/1
Fatih Degirmenci [Wed, 28 Feb 2018 15:11:38 +0000 (15:11 +0000)]
Fix the proxy command and hostname for opnfv vm in ssh config

Change-Id: Ic036a4f5ef8b24b88154f9314957339d3954839a
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoxci: script: start-new-vm: Switch to using 'curl' instead of 'wget' 71/52771/2
Markos Chandras [Tue, 27 Feb 2018 16:13:04 +0000 (16:13 +0000)]
xci: script: start-new-vm: Switch to using 'curl' instead of 'wget'

curl supports the 'retry' argument to handle all sorts of connection
problems so we can use it to retry if we encounter a broken connection.

Change-Id: I3c8afd2be36ee36c5477f05495704bbbee5b9757
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: scripts: start-new-vm: Mask the pgrep exit code 35/52735/1
Markos Chandras [Tue, 27 Feb 2018 12:10:17 +0000 (12:10 +0000)]
xci: scripts: start-new-vm: Mask the pgrep exit code

pgrep will exit with non-zero exit code if it didn't find a matching
process. This breaks the script since we set 'errexit' so we need to
mask it by simply piping it to a 'cat' command.

Change-Id: I80dcf06b597a769242bd59d628bbd8e09f0f199e
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoMerge "xci: scripts: vm: start-new-vm.sh: Re-arrange debug messages"
Markos Chandras [Tue, 27 Feb 2018 10:34:17 +0000 (10:34 +0000)]
Merge "xci: scripts: vm: start-new-vm.sh: Re-arrange debug messages"

7 years agoMerge "xci: config: env-vars: Allow git urls overrides"
Markos Chandras [Mon, 26 Feb 2018 21:59:27 +0000 (21:59 +0000)]
Merge "xci: config: env-vars: Allow git urls overrides"

7 years agoRename var XCI_LOOP to CI_LOOP to align with Functest 67/52167/2
Fatih Degirmenci [Wed, 14 Feb 2018 20:25:31 +0000 (21:25 +0100)]
Rename var XCI_LOOP to CI_LOOP to align with Functest

Change-Id: I1d58f55a1bda258cc3afbfb81e2dd5a1c8e792a1
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoMerge changes I15f5d41f,I7c7125ac
Fatih Degirmenci [Mon, 26 Feb 2018 10:34:39 +0000 (10:34 +0000)]
Merge changes I15f5d41f,I7c7125ac

* changes:
  Export CI_LOOP as expected by Functest
  Export an new var related to Energy

7 years agoMerge "XCI: k8s: Add support for CentOS distro"
Markos Chandras [Mon, 26 Feb 2018 10:28:22 +0000 (10:28 +0000)]
Merge "XCI: k8s: Add support for CentOS distro"

7 years agoMerge "xci: osa: configure-opnvhost: Set time in the opnfv node"
Markos Chandras [Mon, 26 Feb 2018 09:41:48 +0000 (09:41 +0000)]
Merge "xci: osa: configure-opnvhost: Set time in the opnfv node"

7 years agoMerge "xci: Switch default CPU model to 'host-model'"
Markos Chandras [Mon, 26 Feb 2018 09:41:40 +0000 (09:41 +0000)]
Merge "xci: Switch default CPU model to 'host-model'"

7 years agoExport CI_LOOP as expected by Functest 85/51985/4
Cédric Ollivier [Sat, 10 Feb 2018 12:04:00 +0000 (13:04 +0100)]
Export CI_LOOP as expected by Functest

CI_LOOP is hardcoded to daily (the common default value) [1].

[1] https://gerrit.opnfv.org/gerrit/#/c/51981/

Change-Id: I15f5d41f7f10a5c8fddcc4a4d303552ef45c5fb2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoExport an new var related to Energy 47/51947/4
Cédric Ollivier [Fri, 9 Feb 2018 06:34:31 +0000 (07:34 +0100)]
Export an new var related to Energy

It conforms with "Unlink Energy from functest utils and constants":
https://gerrit.opnfv.org/gerrit/#/c/51943/

Depends-On: Iecd5d49b4d4c961c9dc51e43e54df1fcd4027266
Change-Id: I7c7125ac2fee9a2e8f24dcf060bdaab1b39ae552
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoxci: osa: configure-opnvhost: Set time in the opnfv node 39/52539/1
Markos Chandras [Fri, 23 Feb 2018 16:41:39 +0000 (16:41 +0000)]
xci: osa: configure-opnvhost: Set time in the opnfv node

Set the time properly in the opnfv node like we do for the target hosts

Change-Id: I97d9f875dbb37db3cedd87e5915bc5b5f9a7a0ad
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: scripts: start-new-vm.sh: Disable nested virtualization in clean VM 27/52527/1
Markos Chandras [Fri, 23 Feb 2018 14:00:41 +0000 (14:00 +0000)]
xci: scripts: start-new-vm.sh: Disable nested virtualization in clean VM

When using the clean VM for the tests, we will end up with 3 levels of
virtualization when functest is executed (1st level = clean vm, 2nd
level = compute node, 3rd level = functest VMs). This makes the functest
VM terribly slow leading to all sorts of random failures. It appears
that KVM can't handle this case, so we need to switch to the QEMU
interpreter instead which is slower but functional. This only affects
the Jenkins jobs, so deployments on baremetal will still use 2 levels of
nested KVM virtualization which should work fine.

Change-Id: If274129fbf347526982ac4cf577d216173eb4d1b
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: scripts: start-new-vm: Set pipefail option 25/52525/1
Markos Chandras [Fri, 23 Feb 2018 13:48:33 +0000 (13:48 +0000)]
xci: scripts: start-new-vm: Set pipefail option

In commit I75a8cb8c05957bb205e63210d6cafaf54d989f32 the 'ts' utility
was added in order to print timestamps for deployment. However, this
breaks the exit code since 'pipefail' was not set so every job was
returning 'success'.

Change-Id: Ia5f47ea96a4a5be9ed0664f5c7100accc30cde51
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: Switch default CPU model to 'host-model' 07/52507/1
Markos Chandras [Fri, 23 Feb 2018 08:50:03 +0000 (08:50 +0000)]
xci: Switch default CPU model to 'host-model'

From upstream docs:

'The host-model mode is essentially a shortcut to copying host CPU
definition from capabilities XML into domain XML. Since the CPU
definition is copied just before starting a domain, exactly the
same XML can be used on different hosts while still providing
the best guest CPU each host supports'

It's probably safer to use that instead of 'host-passthrough' so
we can get a CPU (both for main VM and the nested ones) that libvirt
understands. Moreover, it's important to present a CPU that the
guest OS understands as well.

Change-Id: I25a8ff0e8635df9804c793d184f048cc86059ce0
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoXCI: k8s: Add support for CentOS distro 07/51607/10
wutianwei [Sat, 3 Feb 2018 08:53:28 +0000 (16:53 +0800)]
XCI: k8s: Add support for CentOS distro

Kubespray already supports the CentOS distribution so make the
necessary changes to allow it to work in XCI.

Change-Id: I3cf1db055a5fd563b107b46456bc3e18eeafb3ab
Co-authored-by: Markos Chandras <mchandras@suse.de>
Signed-off-by: wutianwei <wutianwei1@huawei.com>
7 years agoMerge "xci: osa: Do not wipe cinder volumes"
Fatih Degirmenci [Thu, 22 Feb 2018 21:38:04 +0000 (21:38 +0000)]
Merge "xci: osa: Do not wipe cinder volumes"

7 years agoUpdate the .gitignore in roles 63/52463/1
Manuel Buil [Thu, 22 Feb 2018 17:38:18 +0000 (18:38 +0100)]
Update the .gitignore in roles

THe prepare-functest role changed the name but .gitignore is still
using the old name

Change-Id: I67e962704a62756663abdc721fb8b9a9ac8648a2
Signed-off-by: Manuel Buil <mbuil@suse.com>
7 years agoxci: config: env-vars: Allow git urls overrides 47/52447/1
Markos Chandras [Thu, 22 Feb 2018 11:02:29 +0000 (11:02 +0000)]
xci: config: env-vars: Allow git urls overrides

It's useful to be able to pick alternative git repositories
for the deployment. This facilitates testing in-flight features
or simply save some bandwidth by keeping internal mirrors.

Change-Id: I3eb2e48da1e91f6a52bb3ba14c22f0902c6cd777
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: osa: Do not wipe cinder volumes 19/52419/1
Markos Chandras [Wed, 21 Feb 2018 18:18:50 +0000 (18:18 +0000)]
xci: osa: Do not wipe cinder volumes

When a volume is removed, cinder will fill the entire disk with 0s. This
can slow down the volume removal process. Since we do not care what
happens with the volume after it's being removed, we don't need to
wipe it at all.

Change-Id: I5ef1b01a18cdf2694252b450ca4f1c6a37090f71
Link: https://docs.openstack.org/cinder/latest/sample_config.html
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: roles: configure-network: Determine host NIC from Ansible facts 81/52181/4
Markos Chandras [Thu, 15 Feb 2018 08:57:47 +0000 (08:57 +0000)]
xci: roles: configure-network: Determine host NIC from Ansible facts

Hardcoding the interface as a variable is very fragile since it varies
from host to host. We could use the Ansible facts to find out the
interface name and then use that to configure all the VLANs and
networking.

Change-Id: Ie7e2409d638625b9bede23b6c1fe33dc36f81840
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoCorrect the BIFROST_IRONIC_VERSION variable 97/52197/2
Dimitrios Markou [Thu, 15 Feb 2018 16:15:41 +0000 (18:15 +0200)]
Correct the BIFROST_IRONIC_VERSION variable

By mistake the BIFROST_IRONIC_VERSION variable was
written BIFROST_IRONIC_CLIENT_VERSION

Change-Id: Ie35a4938a2fb38c633dd149c1d18f1da20a4e82a
Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
7 years agoMerge "xci: xci-deploy.sh: Calculate and print the total deployment time"
Markos Chandras [Wed, 21 Feb 2018 08:51:50 +0000 (08:51 +0000)]
Merge "xci: xci-deploy.sh: Calculate and print the total deployment time"

7 years agoMerge "xci: playbooks: run-functest: Collect functest results"
Markos Chandras [Wed, 21 Feb 2018 08:38:41 +0000 (08:38 +0000)]
Merge "xci: playbooks: run-functest: Collect functest results"

7 years agoxci: playbooks: run-functest: Collect functest results 65/52365/1
Markos Chandras [Tue, 20 Feb 2018 15:55:24 +0000 (15:55 +0000)]
xci: playbooks: run-functest: Collect functest results

Add a new volume to the docker container which will hold the results
from the functest run.

Change-Id: I70541a8337c9198a97304312adfab74d4dd06b70
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: Switch VM disk cache to 'unsafe' and use 'iothreads' for I/O 17/52317/3
Markos Chandras [Mon, 19 Feb 2018 17:51:42 +0000 (17:51 +0000)]
xci: Switch VM disk cache to 'unsafe' and use 'iothreads' for I/O

According to the docs[1]

"writeback: This mode causes the hypervisor to interact with the disk
image file or block device with neither O_DSYNC nor O_DIRECT semantics.
The host page cache is used and writes are reported to the guest as
completed when they are placed in the host page cache. The normal page
cache management will handle commitment to the storage device.
Additionally, the guest's virtual storage adapter is informed of the
writeback cache, so the guest would be expected to send down flush
commands as needed to manage data integrity. Analogous to a raid
controller with RAM cache."

and

"writeback: This mode informs the guest of the presence of a write
cache, and relies on the guest to send flush commands as needed to
maintain data integrity within its disk image. This is a common
storage design which is completely accounted for within modern file
systems. This mode exposes the guest to data loss in the unlikely case
of a host failure, because there is a window of time between the time
a write is reported as completed, and that write being committed to the
storage device."

"unsafe: This mode is similar to writeback caching except for the
following: the guest flush commands are ignored, nullifying the data
integrity control of these flush commands, and resulting in a higher
risk of data loss because of host failure. The name “unsafe” should
serve as a warning that there is a much higher potential for data
loss because of a host failure than with the other modes. As the
guest terminates, the cached data is flushed at that time."

It's beneficial to use the host page cache to cache I/O from the guest
instead of waiting for data to reach the actual disk device. We do not
normally care about data integrity so data loss is not a problem.

Moreover, we drop the cache configuration from the flavor files since
it's independent of the flavor that's being deployed.

[1] https://www.suse.com/documentation/sles-12/singlehtml/book_virt/book_virt.html#cha.cachemodes

Change-Id: I118ffdf84b1be672185b3eff60fe5d0b5f1a590d
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: scripts: vm: start-new-vm.sh: Re-arrange debug messages 31/52331/2
Markos Chandras [Mon, 19 Feb 2018 20:28:31 +0000 (20:28 +0000)]
xci: scripts: vm: start-new-vm.sh: Re-arrange debug messages

This also improves (again!) the check for parallel packages managers
to print some useful messages for debugging.

Change-Id: I7756db80aa3ed42cd3ef37293f7116b5a9c150c1
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: xci-deploy.sh: Calculate and print the total deployment time 19/52319/3
Markos Chandras [Mon, 19 Feb 2018 18:24:58 +0000 (18:24 +0000)]
xci: xci-deploy.sh: Calculate and print the total deployment time

It's useful to know how XCI performs in terms of duration time so
use the bash $SECONDS variable to calculate the total amount of time
it takes for a full XCI deployment. Moreover, use the 'ts' command to
print a timestamp for every command executed. This will help developers
identify tasks which take a lot of time to complete and possibly
optimize them.

Change-Id: I75a8cb8c05957bb205e63210d6cafaf54d989f32
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agobifrost: scripts: bifrost-provision.sh: Set defaults for Ironic variables 91/52291/1
Markos Chandras [Mon, 19 Feb 2018 10:29:48 +0000 (10:29 +0000)]
bifrost: scripts: bifrost-provision.sh: Set defaults for Ironic variables

When running the bifrost jobs for the OPNFV 3rd Party CI, the Ironic
variables are not defined and we get the following error:

./scripts/bifrost-provision.sh: line 112: BIFROST_IRONIC_INSPECTOR_VERSION: unbound variable

As such, we need to provide reasonable defaults when we are running this
script outside of XCI.

Change-Id: I9cc3ae1b70494637a85558b9cae99fc1eeaebc74
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoxci: Bump OpenStack Ansbile SHAs 67/50367/8
Markos Chandras [Tue, 13 Feb 2018 21:42:05 +0000 (21:42 +0000)]
xci: Bump OpenStack Ansbile SHAs

* Bump up upstream SHAs
* With these upstream SHAs, OSA installs neutron-agent container on
controller node which would require eth12 interface on controller for
establishing vlan provider networks. so adding eth12 interface and
linking it with br-vlan over br-vlan-veth link.

Change-Id: Iaf7cbe6e41dcfd15ec6527c5a50701f2d05eaad8
Signed-off-by: Markos Chandras <mchandras@suse.de>
Signed-off-by: Periyasamy Palanisamy <periyasamy.palanisamy@ericsson.com>
7 years agobifrost: Add ability to pin Ironic-{,client,inspector,inspector-client} SHAs 91/52091/5
Markos Chandras [Tue, 13 Feb 2018 10:11:16 +0000 (10:11 +0000)]
bifrost: Add ability to pin Ironic-{,client,inspector,inspector-client} SHAs

We should pin the SHAs for Ironic components as part of the regular
bifrost bump to ensure maximum long term stability. This also bumps
the bifrost SHA to the latest available one.

Change-Id: Ia66d8ab6566db58e2ddab5134eb8445bd84d2a8f
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agobifrost: Always deploy Ironic from master 87/52087/3
Markos Chandras [Tue, 13 Feb 2018 09:14:41 +0000 (09:14 +0000)]
bifrost: Always deploy Ironic from master

The Ironic components are currently being installed from PIP but we
should install them from the git repo instead so we can always try
the latest code from master or the pinned version when such
functionality is implemented.

Change-Id: I0138c54befe52ed9df8bf617b78e063df3cf8750
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agoMerge "xci: scripts: vm: Use LVM backend if available"
Markos Chandras [Wed, 7 Feb 2018 07:41:25 +0000 (07:41 +0000)]
Merge "xci: scripts: vm: Use LVM backend if available"

7 years agoxci: scripts: vm: Use LVM backend if available 49/51449/2
Markos Chandras [Thu, 1 Feb 2018 09:49:42 +0000 (09:49 +0000)]
xci: scripts: vm: Use LVM backend if available

The CI nodes come with a big secondary mechanical disk which can be
attached directly to the clean VM. As such, we split the disk into one
logical volume per distribution and flush the qcow2 image to it before
attaching it to the VM. If the XCI volume group is not available, then
we revert back to using a file-based storage backend. Finally, we hide
stdout/stderr from the package installation process to minimize the
script output.

Change-Id: I11ea82899c411f8a4168e5f23cbbd8187cc891df
Signed-off-by: Markos Chandras <mchandras@suse.de>
7 years agobifrost: Bump bifrost sha1s to latest on master 67/51667/2
Fatih Degirmenci [Mon, 5 Feb 2018 12:18:22 +0000 (13:18 +0100)]
bifrost: Bump bifrost sha1s to latest on master

This change brings in the fix for policy.json removal.

https://review.openstack.org/#/c/536349/

Change-Id: Ib18a99502c0ce61278e13f25a01622e073749d14
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
7 years agoRename Functest creds 01/51501/2
Cédric Ollivier [Fri, 2 Feb 2018 17:03:57 +0000 (18:03 +0100)]
Rename Functest creds

It follows the change "Rename a common credential file for OS and
K8S" which stops forcing OpenStack creds.

[1] https://gerrit.opnfv.org/gerrit/#/c/51283/

Change-Id: Ic83a2e696c9ea8821e29e6b3f4e69db25c528a0d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMerge "Adapt to change in functest"
Fatih Degirmenci [Thu, 1 Feb 2018 16:34:20 +0000 (16:34 +0000)]
Merge "Adapt to change in functest"

7 years agoMerge "Enable deltarpm"
Markos Chandras [Thu, 1 Feb 2018 15:07:21 +0000 (15:07 +0000)]
Merge "Enable deltarpm"

7 years agoMerge "xci: scripts: vm: Install additional packages to clean VM"
Markos Chandras [Thu, 1 Feb 2018 09:53:51 +0000 (09:53 +0000)]
Merge "xci: scripts: vm: Install additional packages to clean VM"

7 years agoxci: scripts: vm: Install additional packages to clean VM 61/51361/2
Markos Chandras [Tue, 30 Jan 2018 16:59:16 +0000 (16:59 +0000)]
xci: scripts: vm: Install additional packages to clean VM

Install some additional packages to the clean VM which can be useful
during debugging. Moreover, we move the common distribution packages
into a variable in order to simplify the code a little bit.

Change-Id: I0405820900a15fec730a2bbfc95a1d1f3485bf0f
Signed-off-by: Markos Chandras <mchandras@suse.de>