Merge "Add vm live migration support"

Add vm live migration support

JIRA: -

Add libvirtd configure to support vm live migration.

Test cmd:

    nova live-migration <vm-id> <dest-host-name>

Change-Id: Ic7f47f1e13a7cd84a2d1c67089fedf76662bf412
Signed-off-by: QiLiang <liangqi1@huawei.com>

Merge "Add deploy log to file"

Add deploy log to file

JIRA: -

The default log file path is: work/deploy/log/compass-deploy.log

Change-Id: I6f8ebb028d169a0da88f2a588e4563db4f2f1f76
Signed-off-by: QiLiang <liangqi1@huawei.com>

Add network_openo.yml in all virtual pods

add network_openo.yml to adapt with open-o scenario in:
1. huawei-virtual1
2. huawei-virtual2
3. huawei-virtual3
4. huawei-virtual4
5. huawei-virtual7

Change-Id: I96046470ddb05f8f613489db35f4a6b623a5cf4c
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

Upgrade tacker from mitaka to newton

JIRA: COMPASS-528

Add tacker package making scripts into repo/pip/ directory.
Compass will install tacker using pip in this release.
Upgrade tacker role including tasks, templates and vars.
Modify the installation method from using source code to using whl package.

Change-Id: Iead323c7b46d8d1df084ed868d0e01bc1b0f7e55
Signed-off-by: Yifei Xue <xueyifei@huawei.com>

Merge "add open-o scenario"

add open-o scenario

add os-nosdn-openo-ha.yml and network_openo.yml in:
1. huawei-pod1
2. huawei-pod2
3. intel-pod8

Change-Id: I89ad426c1e3006bf2ef218cd9b55d97e53aa8721
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

Merge "Development documentations update"

Merge "Add License header for files without it"

Merge "add ntp for compass-core"

Development documentations update

JIRA: DOCS-165

Add the document for feature integration which introduces how
to integrate a feature into compass4nfv including creating a role,
making repo and building a feature included ISO.

Change-Id: I8ea70a298cbf079ed30e318ce2cfc4b803218434
Signed-off-by: Yifei Xue <xueyifei@huawei.com>

Add License header for files without it

JIRA: -

use the first author's company for license holder
use the latest year for license year
add license for yml,c,cpp,shell,python,java.
don't add license for template files such as config and j2
patch for auto add license:
https://github.com/Justin-chi/Lab/blob/master/add_license.sh
delete the files with the Eclipse License

Change-Id: I4f17da4ebb36a074c0b6722a2a4154e6c235d4bb
Signed-off-by: chigang <chigang@huawei.com>

Merge "Bug fix for odl l2"

Merge "Update deploy.sh"

Update deploy.sh

JIRA: -

Remove some environment variables in deploy.sh, such as EXPANSION
DEPLOY_COMPASS DEPLOY_HOST DEPLOY_RECOVERY. Those environment
variables's instructions can be found in installation guide document.

Change-Id: I1a7a2e1ad48d7488793ef80742bad1dab47b4270
Signed-off-by: liyuenan <liyuenan@huawei.com>

Bug fix for odl l2

JIRA: COMPASS-512

1. Correct the script of unarchiving odl package for rerunnable
2. Delete the useless code in the script of installing networking-odl
3. Correct the variable "odl_pip" in vars/main.yml

Change-Id: I5488dd4caedcbda735de24fd594207d66310e828
Signed-off-by: Yifei Xue <xueyifei@huawei.com>

Installation and release documentations update

JIRA: DOCS-165

update document for Danube release according to new doc guide;
https://wiki.opnfv.org/display/DOC/Documentation+Guide
add network topology for virtual and bm deployment;
add a section for network_cfg.yaml configuration;
add feature section, listing supported openstack version and features
update installation for deployment and expansion.

Change-Id: If43d94554bebc78bbd56cdf889c8c34dda926218
Signed-off-by: Qingyu <qingyuwang33@gmail.com>
Signed-off-by: Justin <chigang@huawei.com>

add ntp for compass-core

JIRA: COMPASS-514

add ntp for compass-core and set up timezone, make timezone
in compass-core same as where hosts' are
To set timezone of both compass-core and hosts, edit
    /deploy/conf/compass.conf
    TIMEZONE: Asia/Shanghai

Change-Id: If60623dbff793e56e65c32f80addc5e1d956f3e1
Signed-off-by: qiwei-li-huawei <qiwei.li@huawei.com>

Upgrade OpenDaylight to Boron ( centos )

JIRA: COMPASS-512

1. Install requirements of networking-odl
2. Adjust order of removing neutron l3 agent
3. Modify file head

Change-Id: I27869821dc2f534500ec96b48df7a52ce27b58d8
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

Merge "FIX access database failed"

FIX access database failed

JIRA: COMPASS-521

Use a dynamic value to set number of workers for Openstack API
service when the number of cpu more than 4.

Change-Id: Ic6482cfad9b5eba8fad9e56ea1a33f65e8c71001
Signed-off-by: liyuenan <liyuenan@huawei.com>

update info for PTL election

JIRA: -
update info for new PTL election.
add irc channel information.

Change-Id: I7da3e8b40fa90b0f70fc477be9577d6da29a22f6
Signed-off-by: Justin chi <chigang@huawei.com>

Merge "FIX access database failed"

FIX access database failed

JIRA: COMPASS-521

Nova compute use osapi_compute and metadata api. Both of their worker
processes should be limited, otherwise mysql would not deal with those
processes.

Change-Id: Ib00b4731369c1c677639c9437ab83887550db572
Signed-off-by: liyuenan <liyuenan@huawei.com>

Fix create stack failed

JIRA: COMPASS-523

Heat needs grant heat admin role and create a menber role, otherwise
functest smoke test will failed when create stack by heat.

Change-Id: I925b473861bf82becf119bfd490c38732814c336
Signed-off-by: liyuenan <liyuenan@huawei.com>

Merge "Ease to switch themes of Horizon"

Update shade version to avoid the conflict

JIRA: COMPASS-520

Shade and python-openstackclient both depends on python client libraries
for OpenStack.If install one from pip and one from yum or apt, then there
can be weird conflicts between the two sets of libraries. Update shade
packages to resolve this problem.

Change-Id: Icfd82d2818abfcf5061133fa6670b23daac9fb5b
Signed-off-by: liyuenan <liyuenan@huawei.com>

Upgrade OpenDaylight to Boron ( l3 scenario )

JIRA: COMPASS-512

1. Use package distribution-karaf-0.5.2-Boron.tar.gz
2. Add configuring ACL
3. Remove neutron-l3 agent in odl_cluster role
4. Fix ODL l2 metadata problem

Change-Id: Ib14cdf3539deeeb91ebfec0450a9e7f0228f7741
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

Ease to switch themes of Horizon

JIRA: -

Default Horizon theme is ubuntu. If you want to use openstack theme,
you can run `export ENABLE_UBUNTU_THEME=False` before deploy.

And if you want to change theme after you installed openstack, you need
to run this commands:
  1. access compass VM, password is root
      `ssh root@192.168.200.2`
  2. change path:
      `cd /var/ansible/run/openstack_newton-opnfv2`
  3. Edit the `group_vars/all` file, change `enable_ubuntu_theme` var:
      `enable_ubuntu_theme: False`
     if `enable_ubuntu_theme: True`, the Horizon theme is ubuntu, otherwise
     the Horizon theme is openstack.
  4. run ansible playbook:
       `ansible-playbook -i inventories/inventory.yml HA-ansible-multinodes.yml
        -t horizon_theme`

Change-Id: I5465107228af32a482879c8a058899494f33e2d0
Signed-off-by: liyuenan <liyuenan@huawei.com>

Fix Aodh Bug

JIRA: COMPASS-522

Bug:
Connection error happens when using aodh command.
Curl aodh-api also returns empty reply.

Solution:
Use apache to be aodh api server. Disable aodh-api
service. Delete aodh-api restart in ext-network role
to avoid aodh-api start again.

Change-Id: I04a4af38325bcbe4a0b86ec3d3beb7c9f3abd73b
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

Congress support for Newton

Modification:
  1.Add congress support in deploy/compass_conf
  2.Optimize congress role to be more independant.
    Separate congress role into install, keystone,
    database, start.
  3.Support both ubuntu and centos

Change-Id: I1c9a458addd84133652fb7c0563bc287d161d129
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

Merge "Add opera support"

Add opera support

JIRA: COMPASS-504
JIRA: OPERA-1

Function:
  support open-o deployment by calling opera project.
  open-o deployment will start at the end of normal
  compass4nfv deployment.

Changes:
  1. add scenario and network yaml for opera
  2. add opera_adapter.py for calling opera

Attention:
  The git URL for opera will change after opera patch
  OPERA-2 is meraged. This patch should be modifed
  before meraged.

Change-Id: Id0afcece920c4107bb23f42b460c46eec3ca97e6
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

New patch for orange-pod3 POD with indentation correction for network*.yml files

Change-Id: Ifa100d6e444c4190d8f15d980c4fd72f8b953a1d
Signed-off-by: Francois DESPRES <francois.despres@orange.com>

Ansible Module substitute for Shell Commands

JIRA: COMPASS-520

After update ansible version to v3, keystone_user module only support v2 API.
So we use Shell Commands now, but it will failed with high probability.
Those Shell Commands should be instead by ansible modules to manage Identity users,
projects or some other work like crate networks.

Change-Id: I63d38b4a811a9c063ac4404da72787f594411b53
Signed-off-by: liyuenan <liyuenan@huawei.com>

Merge "Adjust make repo"

Adjust make repo

Demand:
  1.Compass4nfv master code only support one OPNFV version
    now, no need to keep previous version's repo scripts
  2.Currently java packages are in /var/www/package/onos
    directory which is unreasonable
  3.repo.conf has a lot of download URL which can turn into
    pip requirements

Modification:
  1.Remove previous version's directory in compass4nfv/repo
  2.Only leave java and odl in compass4nfv/repo/features.
    Other features can be added when they are ready
  3.Seperate URLs in repo.conf into compass-core and openstack
    pip requirements. Therefore pip.tar.gz only contains pip
    packages for compass-core and pip-openstack.tar.gz only
    contains pip packages for openstack
  3.Move java packages into /var/www/package/java directory
  4.Add make pip wheel package function to support congress
  5.Use ODL package distribution-karaf-0.5.0-Boron.tar.gz
  6.Use python package netwoking-odl-3.2.0

Change-Id: Ifa818e7630ac4435e3bdb7756b83f80f50c24a76
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

FIX access database failed

JIRA: COMPASS-521

Number of separate API worker processes for service. The default is calculated by
the number of CPUs available for best performance. But some pods have many CPUs,
so that some service API worker processes is too many to access database.
For example, huawei-pod2 has 47 CPUs, nova-api has 98 processes, cinder-api has
50 processes and heat-api has 33 processes, when aodh want to initialize the database,
mysql can not deal with this processes.

Change-Id: I00513883ff55cc28cd676c3085615aaac36eb744
Signed-off-by: liyuenan <liyuenan@huawei.com>

[compass] bugfix: it is noneffective to retry the network checking

Change-Id: I97413575e1cac87da4ac99bc45e1c55777f38243
Signed-off-by: meimei <meimei@huawei.com>

Support ceilometer for Newton CentOS

JIRA: COMPASS-518

The WSGIScriptAlias file should be "/var/www/cgi-bin/ceilometer/app".
This file is copied from
"/usr/lib/python2.7/site-packages/ceilometer/api/app.wsgi" in CentOS
or
"/usr/lib/python2.7/dist-packages/ceilometer/api/app.wsgi" in Ubuntu

Change-Id: Idcc37b0ed6c1d4c6e39d821a02a6579c57b6f035
Signed-off-by: liyuenan <liyuenan@huawei.com>

Merge "Create openstack flavors"

update intel pod8 configuration

the IPMI IP assignment and vlan of intel-pod8 have been changed

Change-Id: Ia6f2880ab2388b031b79046c00e03bc918b957e1
Signed-off-by: meimei <meimei@huawei.com>

Deploy OpenStack Newton on CentOS hosts

JIRA: COMPASS-518

Make Centos Newton ppa.
Update host Centos version to 7.3.
Support Centos Newton Deploy.
Support Ceph Deploy.

Change-Id: I6dd110cc10d425a9d590e1dcb5777ade5cce96aa
Signed-off-by: liyuenan <liyuenan@huawei.com>

Remove some duplicate or useless files

JIRA: -

Remove some templates because it is duplicated.
Remove some repo files because master support newton only.

Change-Id: I6651191c6edb2ef6ab5da109922a889a8cd58a78
Signed-off-by: liyuenan <liyuenan@huawei.com>

Create openstack flavors

JIRA: COMPASS-519

Demand:
  Openstack Newton doesn't create flavors defaultly. But
functest needs to run with flavors exist

Solution:
  Add an ansible role to create flavors after openstack
deployed

Change-Id: Ib7ac56b3d42d9ddfcbefffc44ac92fa81342368a
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

FIX kvm_cpu_num fetch bug

If the compute node not support kvm virtualization, will get error:

TASK: [nova-compute | get number of cpu support virtualization] *********
failed: [host-1] => {"changed": true, "cmd": "egrep -c '(vmx|svm)'
 /proc/cpuinfo", "delta": "0:00:00.054521", "end":
"2017-01-11 10:46:12.093745", "rc": 1, "start":
"2017-01-11 10:46:12.039224", "warnings": []}

JIRA: -

Change-Id: I773d6c703c23c8c0eb477e1d61eb0ca5366e6722
Signed-off-by: QiLiang <liangqi1@huawei.com>

Upgrade OpenDaylight to Boron ( l2 scenario )

JIRA: COMPASS-512

1. Use package distribution-karaf-0.5.0-Boron.tar.gz
2. Use package networking-odl-3.2.0.tar.gz
3. Set both neutron and odl as ovs manager

Change-Id: I126ba21346afaaccf1f71a17a500146968bdb4eb
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

FIX external net create

JIRA: -

The dictionary public_net_info in network.yml had changed to pass the
yamllint:
  public_net_info:
    enable: True
change to:
  public_net_info:
    enable: "True"
Because of that, the public_net_info.enable in group_var/all had changed.
  'public_net_info': {..., 'enable': True, ...}
change to:
  'public_net_info': {..., 'enable': 'True', ...}
So the conditionals in roles/ext-network/tasks/main.yml should be changed,
otherwise the ext-net can not be created.

Change-Id: I7b4aeb1d0edec0ad77252849704bdf3055adfa5a
Signed-off-by: liyuenan <liyuenan@huawei.com>

Merge "Fix multi-nics BM deploy network unreachable issue"

FIX Ceph user error

The command of create ceph user for openstack is destroied in Yamllint patch.
There is a error in the roles/ceph-openstack/tasks/ceph-openstack-pre.yml.
It should not change a line between the quotations.

JIRA: -

Change-Id: I5347e8bfde129855c28cb5313b8367f4edb065ca
Signed-off-by: liyuenan <liyuenan@huawei.com>

Fix multi-nics BM deploy network unreachable issue

Configure OS_MGMT_NIC when openstack external network and mgmt
network use different nics. Otherwise openstack mgmt is unreachabe
from jumpserver, then compass cann't pass functest healthcheck.

JIRA: -

Change-Id: If156806d3b92933ac316f178d3eb2d0aaa9e1b4a
Signed-off-by: QiLiang <liangqi1@huawei.com>

Fix instance can't get key bug

JIRA:COMPASS-517

modify metadata_agent.ini to support keystone api V3

Change-Id: If0a3a9f687cdb48af740ed6231e96e3e712755f4
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

Yamllint test

JIRA: COMPASS-516

Change-Id: I482ce9bc86f4f963258c5b8823e0b00e83556eef
Signed-off-by: liyuenan <liyuenan@huawei.com>

Update ansible version

JIRA: COMPASS-511

Change-Id: Id0e83c65a08c9f9036b5d3a597b905e8a7bde6e8
Signed-off-by: liyuenan <liyuenan@huawei.com>

Change newton_xenial to newton

JIRA: COMPASS-513

Change-Id: I937a02bcd69f0df1f5d951dd6630ef2e76e8d386
Signed-off-by: liyuenan <liyuenan@huawei.com>

master only support newton

JIRA: COMPASS-513

Remove other roles and ppa, master only support newton.

Change-Id: I47ddb16baa25902c3e05cc7f9d0d6430f5dc7e00
Signed-off-by: liyuenan <liyuenan@huawei.com>

Merge "add congress support"

Merge "Add VIM registration for Tacker Mitaka"

Merge "Support expansion for newton"

Merge "Print information about openstack deployed."

Add VIM registration for Tacker Mitaka

JIRA: COMPASS-510

Change-Id: I05c2a0670d4c75f516f53f9849793e02e92cb559
Signed-off-by: Yifei Xue <xueyifei@huawei.com>

Support expansion for newton

JIRA: COMPASS-481

Change-Id: I2dc6230f808a62f6b852f0668771b7302c7b540c
Signed-off-by: liyuenan <liyuenan@huawei.com>

Print information about openstack deployed.

Change-Id: I80a379bcb5851ef2b29a52c8d45450dd1d36ade0
Signed-off-by: liyuenan <liyuenan@huawei.com>

Merge "Support compute node expansion with ceph-osd"

Merge "Offline deployment fails on jumphost with docker installed"

because of replacing the huawei-pod5 with a new pod,
we need to change the configuratuion according the new pod.

Change-Id: Ib2bfa54b5fd300fbed5fae77a7f1e6c36fc63b04
Signed-off-by: wutianwei <wutianwei1@huawei.com>

Support compute node expansion with ceph-osd

JIRA: COMPASS-481

- Use public_vip to fetch ceph related conf from controller nodes.
- Add ceph-uuid for libvirt store and fetch for expansion.

Change-Id: I9e93183ac108bd959a1226546fbb89706e3004c9
Signed-off-by: QiLiang <liangqi1@huawei.com>

Fix manual ci trigger build

JIRA: COMPASS-509

Change-Id: I17bc50b855203af5c1fa9227dbb02e7ccbb87965
Signed-off-by: liyuenan <liyuenan@huawei.com>

Merge "Fix keystone token flush"

Merge "Deploy some special scenarios automatically in CI"

Deploy some special scenarios automatically in CI

For daily build or verify build, adjust OS_VERSION and OPENSTACK_VERSION
value according to OS_VERSION and DEPLOY_SCENARIO pass from CI.

JIRA: COMPASS-509

Change-Id: I4db4d759e62a3b574ac8decde11ee8cba80069e2
Signed-off-by: liyuenan <liyuenan@huawei.com>

Merge "add compass-core conf/ into compass4nfv"

Fix keystone token flush

JIRA: COMPASS-491

Change-Id: I5986f8152f88ac7a77dd8bf2bbbcb92faf208aa2
Signed-off-by: liyuenan <liyuenan@huawei.com>

Support aodh for Newton

JIRA: COMPASS-491

Change-Id: I209aec69eec1376432807b44c0a0e890261f3dff
Signed-off-by: liyuenan <liyuenan@huawei.com>

add congress support

JIRA: COMPASS-367

add congress support in openstack_mitaka and openstack_newton_xenial

Change-Id: I3aa19cb54081ad2300964955b899c1b56a875b25
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

Offline deployment fails on jumphost with docker installed

JIRA: COMPASS-508

Change-Id: I92fa3a40d010c74ca7d7d4f638413e7c9e571e36
Signed-off-by: Yifei Xue <xueyifei@huawei.com>
(cherry picked from commit ff2621f0dbe1e49e4d7fad752ec6d9a2e9f1b444)

Fix some xenial baremetal deploy bugs

1. Fix pip upgrade

2. Fix apache conf file

JIRA: COMPASS-489

Change-Id: I2d45114b559298898ab354f884ced74fc9fc7abf
Signed-off-by: liyuenan <liyuenan@huawei.com>

Merge "fix demo user password bug"

Merge "update pip and split pip source"

Merge "Support ceilometer for Newton"

Merge "Fix the wait_for modules"

Merge "Fix the apache service restart failed in xenial"

Fix the apache service restart failed in xenial

JIRA: COMPASS-491

Change-Id: I8e12a308e6ba48117b906a0e101cddaeb9ee8a2e
Signed-off-by: liyuenan <liyuenan@huawei.com>

Fix the wait_for modules

After restart the service, service's port should be polled rather than haproxy.

JIRA: COMPASS-491

Change-Id: I52165e9673b997883109fbf93a93b360a25dabba
Signed-off-by: liyuenan <liyuenan@huawei.com>

add compass-core conf/ into compass4nfv

JIRA: COMPASS-507

1.create deploy/compass_conf/ directory to keep local
compass-core conf/ files. build.sh will add it into
compass.iso

2.adjust conf files to support congress

Change-Id: I91dde6f62f2385d1289a5cf9a39eafce945a1bc7
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

fix demo user password bug

JIRA: COMPASS-505

add demo user password in keystone ansible

Change-Id: I369934b415c91af0a25616cefcea70041a43a63d
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

update pip and split pip source

JIRA: COMPASS-506

update pip to 9.0.1 in ansible common
adjust pip download source to http://10.1.0.12/pip-openstack

Change-Id: I043d199faf57342f89e46bc9d90229ed40c510f0
Signed-off-by: Harry Huang <huangxiangyu5@huawei.com>

Merge "Improve compass ci verification--clear all flake8 warnings & errors of compass4nfv JIRA: COMPASS-494"

Merge "Create make_repo directory and optimize the current repo making related script"

Support ceilometer for Newton

JIRA: COMPASS-491

Change-Id: Iffa7e37e931af4c212fe523e529ee34dd2c14c24
Signed-off-by: liyuenan <liyuenan@huawei.com>

Merge "Fix libvirt error on jumphost"

Merge "Add refresh.sh exec on compass-core"

Merge "up INSTALL_NIC during installation"

Fix libvirt error on jumphost

Fix permission denied (Libvirt::Error) on the disk image, when
boot compass-core vm on some jumphost env.

Ref:
* https://github.com/adrahon/vagrant-kvm/issues/163
* https://ubuntuforums.org/showthread.php?t=1985773

JIRA: -

Change-Id: I17adad6ff6984beac4a4c65d8953a36d3a39f7ce
Signed-off-by: QiLiang <liangqi1@huawei.com>

Add refresh.sh exec on compass-core

Add refresh.sh exec on compass-core, so user don't need to manually execute
the refresh.sh script when they what redeploy the openstack without redeploy
the compass-core vm.

Change-Id: I1a6b5ca84d4dd00a3f326b30b68e0ecac022d5ef
Signed-off-by: QiLiang <liangqi1@huawei.com>

FIX cobbler system edit cmd args error

use "--static=1" to set server nic to static mode

JIRA: COMPASS-489

Change-Id: I4d8d90f861b6b264575fc0781f585a6ef6bd1c70
Signed-off-by: QiLiang <liangqi1@huawei.com>

Improve compass ci verification--clear all flake8 warnings & errors of compass4nfv
JIRA: COMPASS-494

Change-Id: Iec4664a376100f2a0f5fe6c46c1a45360f9c2324
Signed-off-by: KingPoo <haojingbo@huawei.com>

up INSTALL_NIC during installation

Some jumpserver's INSTALL_NIC is down during compass4nfv installation,
this will cause pxe boot failure. So add auto up the INSTALL_NIC during
installation.

JIRA: -

Change-Id: I109b45360edf03e575f00e88d4b2ce12c01b2869
Signed-off-by: QiLiang <liangqi1@huawei.com>

Create make_repo directory and optimize the current repo making related script

JIRA: COMPASS-496

Change-Id: I662d996382d38a3e9d0de6598dabe403b9d51665
Signed-off-by: Yifei Xue <xueyifei@huawei.com>