Make the DB URIs host-independent for all services
authorMichele Baldessari <michele@acksyn.org>
Thu, 9 Feb 2017 10:14:03 +0000 (11:14 +0100)
committerMichele Baldessari <michele@acksyn.org>
Fri, 17 Feb 2017 16:22:42 +0000 (17:22 +0100)
When fixing LP#1643487 we added ?bind_address to all DB URIs.
Since this clashes with Cellsv2 due to the URIs becoming host
dependent, we need a new approach to pass bind_address to pymysql
that leaves the DB URIs host-independent.

In change Iff8bd2d9ee85f7bb1445aa2e1b3cfbff1f397b18 we first create a
/etc/my.cnf.d/tripleo.cnf file with a [tripleo] section with the correct
bind-address option.

In this change we make sure that the DB URIs will point to the added
file and to the specific section containing the necessary bind-address
option. We do introduce a new MySQLClient profile which will hold all
this more client-specific configuration so that this change can fit
better in the composable roles work. Also, in the future it might
contain the necessary configuration for SSL for example.

Note that in case the /etc/my.cnf.d/tripleo.cnf file does not exist
(because it is created via the mysqlclient profile), things keep on
working as usual and the bind-address option simply won't be set, which
has no impact on hosts where there are no VIPs.

Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>
Change-Id: Ieac33efe38f32e949fd89545eb1cd8e0fe114a12
Related-Bug: #1643487
Closes-Bug: #1663181
Closes-Bug: #1664524
Depends-On: Iff8bd2d9ee85f7bb1445aa2e1b3cfbff1f397b18

32 files changed:
ci/environments/multinode-3nodes.yaml
ci/environments/multinode.yaml
ci/environments/multinode_major_upgrade.yaml
ci/environments/scenario001-multinode.yaml
ci/environments/scenario002-multinode.yaml
ci/environments/scenario003-multinode.yaml
ci/environments/scenario004-multinode.yaml
deployed-server/deployed-server-roles-data.yaml
overcloud-resource-registry-puppet.j2.yaml
puppet/services/aodh-base.yaml
puppet/services/barbican-api.yaml
puppet/services/ceilometer-base.yaml
puppet/services/cinder-base.yaml
puppet/services/congress.yaml
puppet/services/database/mysql-client.yaml [new file with mode: 0644]
puppet/services/ec2-api.yaml
puppet/services/glance-api.yaml
puppet/services/gnocchi-base.yaml
puppet/services/heat-engine.yaml
puppet/services/ironic-base.yaml
puppet/services/keystone.yaml
puppet/services/manila-base.yaml
puppet/services/mistral-base.yaml
puppet/services/neutron-api.yaml
puppet/services/neutron-plugin-plumgrid.yaml
puppet/services/nova-base.yaml
puppet/services/octavia-api.yaml
puppet/services/panko-base.yaml
puppet/services/sahara-base.yaml
puppet/services/tacker.yaml
roles_data.yaml
tools/yaml-validate.py

index f35a080..03065c6 100644 (file)
@@ -65,6 +65,7 @@
     - OS::TripleO::Services::Core
     - OS::TripleO::Services::Kernel
     - OS::TripleO::Services::MySQL
+    - OS::TripleO::Services::MySQLClient
     - OS::TripleO::Services::RabbitMQ
     - OS::TripleO::Services::HAproxy
     - OS::TripleO::Services::Keepalived
index 212f6a2..0609dd5 100644 (file)
@@ -15,6 +15,7 @@ parameter_defaults:
     - OS::TripleO::Services::HeatApiCloudwatch
     - OS::TripleO::Services::HeatEngine
     - OS::TripleO::Services::MySQL
+    - OS::TripleO::Services::MySQLClient
     - OS::TripleO::Services::NeutronDhcpAgent
     - OS::TripleO::Services::NeutronL3Agent
     - OS::TripleO::Services::NeutronMetadataAgent
index 4859c23..6710fef 100644 (file)
@@ -28,6 +28,7 @@ parameter_defaults:
     - OS::TripleO::Services::SaharaApi
     - OS::TripleO::Services::SaharaEngine
     - OS::TripleO::Services::MySQL
+    - OS::TripleO::Services::MySQLClient
     - OS::TripleO::Services::RabbitMQ
     - OS::TripleO::Services::HAproxy
     - OS::TripleO::Services::Keepalived
index 72e2570..88a9dfc 100644 (file)
@@ -19,6 +19,7 @@ parameter_defaults:
     - OS::TripleO::Services::HeatApiCloudwatch
     - OS::TripleO::Services::HeatEngine
     - OS::TripleO::Services::MySQL
+    - OS::TripleO::Services::MySQLClient
     - OS::TripleO::Services::NeutronDhcpAgent
     - OS::TripleO::Services::NeutronL3Agent
     - OS::TripleO::Services::NeutronMetadataAgent
index bf4721e..dea5104 100644 (file)
@@ -15,6 +15,7 @@ parameter_defaults:
     - OS::TripleO::Services::HeatApiCloudwatch
     - OS::TripleO::Services::HeatEngine
     - OS::TripleO::Services::MySQL
+    - OS::TripleO::Services::MySQLClient
     - OS::TripleO::Services::NeutronDhcpAgent
     - OS::TripleO::Services::NeutronL3Agent
     - OS::TripleO::Services::NeutronMetadataAgent
index 9167010..4ed19f8 100644 (file)
@@ -17,6 +17,7 @@ parameter_defaults:
     - OS::TripleO::Services::HeatApiCloudwatch
     - OS::TripleO::Services::HeatEngine
     - OS::TripleO::Services::MySQL
+    - OS::TripleO::Services::MySQLClient
     - OS::TripleO::Services::NeutronDhcpAgent
     - OS::TripleO::Services::NeutronL3Agent
     - OS::TripleO::Services::NeutronMetadataAgent
index 87b10ca..3694e5a 100644 (file)
@@ -19,6 +19,7 @@ parameter_defaults:
     - OS::TripleO::Services::HeatApiCloudwatch
     - OS::TripleO::Services::HeatEngine
     - OS::TripleO::Services::MySQL
+    - OS::TripleO::Services::MySQLClient
     - OS::TripleO::Services::NeutronDhcpAgent
     - OS::TripleO::Services::NeutronL3Agent
     - OS::TripleO::Services::NeutronMetadataAgent
index 9795a00..04da556 100644 (file)
@@ -41,6 +41,7 @@
     - OS::TripleO::Services::HeatApiCloudwatch
     - OS::TripleO::Services::HeatEngine
     - OS::TripleO::Services::MySQL
+    - OS::TripleO::Services::MySQLClient
     - OS::TripleO::Services::NeutronDhcpAgent
     - OS::TripleO::Services::NeutronL3Agent
     - OS::TripleO::Services::NeutronMetadataAgent
index 9a8de34..df94a3f 100644 (file)
@@ -245,6 +245,7 @@ resource_registry:
   OS::TripleO::Services::OctaviaHealthManager: OS::Heat::None
   OS::TripleO::Services::OctaviaHousekeeping: OS::Heat::None
   OS::TripleO::Services::OctaviaWorker: OS::Heat::None
+  OS::TripleO::Services::MySQLClient: puppet/services/database/mysql-client.yaml
 
 parameter_defaults:
   EnablePackageInstall: false
index f5ca329..c2c2d02 100644 (file)
@@ -69,8 +69,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/aodh'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         aodh::debug: {get_param: Debug}
         aodh::auth::auth_url: {get_param: [EndpointMap, KeystoneInternal, uri_no_suffix] }
         aodh::rabbit_userid: {get_param: RabbitUserName}
index 239b6ca..ffc4c83 100644 (file)
@@ -105,8 +105,7 @@ outputs:
                   - '@'
                   - {get_param: [EndpointMap, MysqlInternal, host]}
                   - '/barbican'
-                  - '?bind_address='
-                  - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+                  - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
             tripleo.barbican_api.firewall_rules:
               '117 barbican':
                 dport:
index 5658e41..874c689 100644 (file)
@@ -93,8 +93,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/ceilometer'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         ceilometer_backend: {get_param: CeilometerBackend}
         # we include db_sync class in puppet-tripleo
         ceilometer::db::sync_db: false
index a5d7fcf..88e7edb 100644 (file)
@@ -100,8 +100,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/cinder'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         cinder::debug: {get_param: Debug}
         cinder::rabbit_use_ssl: {get_param: RabbitClientUseSSL}
         cinder::rabbit_userid: {get_param: RabbitUserName}
index 1b82f55..aa8d9a9 100644 (file)
@@ -64,8 +64,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/congress'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         congress::keystone::auth::tenant: 'service'
         congress::keystone::auth::password: {get_param: CongressPassword}
         congress::keystone::authtoken::auth_uri: {get_param: [EndpointMap, KeystoneInternal, uri]}
diff --git a/puppet/services/database/mysql-client.yaml b/puppet/services/database/mysql-client.yaml
new file mode 100644 (file)
index 0000000..1415391
--- /dev/null
@@ -0,0 +1,30 @@
+heat_template_version: ocata
+
+description: >
+  Mysql client settings
+
+parameters:
+  ServiceNetMap:
+    default: {}
+    description: Mapping of service_name -> network name. Typically set
+                 via parameter_defaults in the resource registry.  This
+                 mapping overrides those in ServiceNetMapDefaults.
+    type: json
+  DefaultPasswords:
+    default: {}
+    type: json
+  EndpointMap:
+    default: {}
+    description: Mapping of service endpoint -> protocol. Typically set
+                 via parameter_defaults in the resource registry.
+    type: json
+
+outputs:
+  role_data:
+    description: Role for setting mysql client parameters
+    value:
+      service_name: mysql_client
+      config_settings:
+        tripleo::profile::base:database::mysql::client_bind_address: {get_param: [ServiceNetMap, MysqlNetwork]}
+      step_config: |
+        include ::tripleo::profile::base::database::mysql::client
index 002342b..bb10140 100644 (file)
@@ -90,8 +90,7 @@ outputs:
                 - '@'
                 - {get_param: [EndpointMap, MysqlInternal, host]}
                 - '/ec2_api'
-                - '?bind_address='
-                - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+                - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         -
           if:
           - nova_workers_zero
index c4f97d5..d26d96a 100644 (file)
@@ -91,8 +91,7 @@ outputs:
                   - '@'
                   - {get_param: [EndpointMap, MysqlInternal, host]}
                   - '/glance'
-                  - '?bind_address='
-                  - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+                  - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
             glance::api::bind_port: {get_param: [EndpointMap, GlanceInternal, port]}
             glance::api::authtoken::auth_uri: {get_param: [EndpointMap, KeystoneInternal, uri] }
             glance::api::authtoken::auth_url: { get_param: [EndpointMap, KeystoneInternal, uri_no_suffix] }
index 8fddae4..c631005 100644 (file)
@@ -67,8 +67,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/gnocchi'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         gnocchi::db::sync::extra_opts: '--skip-storage'
         gnocchi::storage::swift::swift_user: 'service:gnocchi'
         gnocchi::storage::swift::swift_auth_version: 2
index e85b753..f7ec9a4 100644 (file)
@@ -111,8 +111,7 @@ outputs:
                   - '@'
                   - {get_param: [EndpointMap, MysqlInternal, host]}
                   - '/heat'
-                  - '?bind_address='
-                  - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+                  - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
             heat::keystone_ec2_uri: {get_param: [EndpointMap, KeystoneEC2, uri]}
             heat::keystone::domain::domain_password: {get_param: HeatStackDomainAdminPassword}
             heat::engine::auth_encryption_key:
index ad7ef6e..d186b04 100644 (file)
@@ -60,8 +60,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/ironic'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         ironic::debug: {get_param: Debug}
         ironic::rabbit_userid: {get_param: RabbitUserName}
         ironic::rabbit_password: {get_param: RabbitPassword}
index b2374ec..9c4cc60 100644 (file)
@@ -193,8 +193,7 @@ outputs:
                   - '@'
                   - {get_param: [EndpointMap, MysqlInternal, host]}
                   - '/keystone'
-                  - '?bind_address='
-                  - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+                  - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
             keystone::admin_token: {get_param: AdminToken}
             keystone::admin_password: {get_param: AdminPassword}
             keystone::roles::admin::password: {get_param: AdminPassword}
index 2a9745a..c183bc0 100644 (file)
@@ -67,8 +67,7 @@ outputs:
             - '@'
             - {get_param: [EndpointMap, MysqlInternal, host]}
             - '/manila'
-            - '?bind_address='
-            - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+            - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
       service_config_settings:
         mysql:
           manila::db::mysql::password: {get_param: ManilaPassword}
index 4d02049..e103034 100644 (file)
@@ -65,8 +65,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/mistral'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         mistral::rabbit_userid: {get_param: RabbitUserName}
         mistral::rabbit_password: {get_param: RabbitPassword}
         mistral::rabbit_use_ssl: {get_param: RabbitClientUseSSL}
index b3a07fb..4d671e1 100644 (file)
@@ -127,8 +127,7 @@ outputs:
                   - '@'
                   - {get_param: [EndpointMap, MysqlInternal, host]}
                   - '/ovs_neutron'
-                  - '?bind_address='
-                  - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+                  - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
             neutron::keystone::authtoken::auth_uri: {get_param: [EndpointMap, KeystoneInternal, uri] }
             neutron::keystone::authtoken::auth_url: {get_param: [EndpointMap, KeystoneInternal, uri_no_suffix]}
             neutron::server::api_workers: {get_param: NeutronWorkers}
index ad1dcfb..f948dd0 100644 (file)
@@ -100,8 +100,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/ovs_neutron'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         neutron::plugins::plumgrid::controller_priv_host: {get_param: [EndpointMap, KeystoneInternal, host]}
         neutron::plugins::plumgrid::admin_password: {get_param: AdminPassword}
         neutron::plugins::plumgrid::metadata_proxy_shared_secret: {get_param: NeutronMetadataProxySharedSecret}
index d892c36..ceacb0b 100644 (file)
@@ -150,6 +150,7 @@ outputs:
                 - '@'
                 - {get_param: [EndpointMap, MysqlInternal, host]}
                 - '/nova'
+                - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
           nova::api_database_connection:
             list_join:
               - ''
@@ -159,6 +160,7 @@ outputs:
                 - '@'
                 - {get_param: [EndpointMap, MysqlInternal, host]}
                 - '/nova_api'
+                - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
           nova::placement_database_connection:
             list_join:
               - ''
@@ -168,6 +170,7 @@ outputs:
                 - '@'
                 - {get_param: [EndpointMap, MysqlInternal, host]}
                 - '/nova_placement'
+                - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
           nova::debug: {get_param: Debug}
           nova::purge_config: {get_param: EnableConfigPurge}
           nova::network::neutron::neutron_project_name: 'service'
index 37ba1f7..909a303 100644 (file)
@@ -66,8 +66,7 @@ outputs:
                   - '@'
                   - {get_param: [EndpointMap, MysqlInternal, host]}
                   - '/octavia'
-                  - '?bind_address='
-                  - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+                  - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
             octavia::keystone::authtoken::auth_url: {get_param: [EndpointMap, KeystoneInternal, uri_no_suffix]}
             octavia::keystone::authtoken::project_name: 'service'
             octavia::keystone::authtoken::password: {get_param: OctaviaPassword}
index 2c2586a..998e64e 100644 (file)
@@ -46,8 +46,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/panko'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         panko::debug: {get_param: Debug}
         panko::auth::auth_url: {get_param: [EndpointMap, KeystoneInternal, uri_no_suffix] }
         panko::keystone::authtoken::project_name: 'service'
index e208418..224989b 100644 (file)
@@ -64,8 +64,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/sahara'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
         sahara::rabbit_password: {get_param: RabbitPassword}
         sahara::rabbit_user: {get_param: RabbitUserName}
         sahara::rabbit_use_ssl: {get_param: RabbitClientUseSSL}
index 5cf09a6..1ac165d 100644 (file)
@@ -64,8 +64,7 @@ outputs:
               - '@'
               - {get_param: [EndpointMap, MysqlInternal, host]}
               - '/tacker'
-              - '?bind_address='
-              - "%{hiera('tripleo::profile::base::database::mysql::client_bind_address')}"
+              - '?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo'
 
         tacker::keystone::auth::tenant: 'service'
         tacker::keystone::auth::password: {get_param: TackerPassword}
index 31b1298..70193cf 100644 (file)
@@ -47,6 +47,7 @@
     - OS::TripleO::Services::HeatApiCloudwatch
     - OS::TripleO::Services::HeatEngine
     - OS::TripleO::Services::MySQL
+    - OS::TripleO::Services::MySQLClient
     - OS::TripleO::Services::NeutronDhcpAgent
     - OS::TripleO::Services::NeutronL3Agent
     - OS::TripleO::Services::NeutronMetadataAgent
index 0eacbc6..1d0dba0 100755 (executable)
@@ -62,14 +62,12 @@ def validate_mysql_connection(settings):
         return items == ['EndpointMap', 'MysqlInternal', 'protocol']
 
     def client_bind_address(item):
-        return 'bind_address' in item
+        return 'read_default_file' in item and \
+               'read_default_group' in item
 
     def validate_mysql_uri(key, items):
         # Only consider a connection if it targets mysql
-        # TODO(owalsh): skip nova mysql uris,temporary workaround for
-        # tripleo/+bug/1662344
-        if not key.startswith('nova') and \
-           key.endswith('connection') and \
+        if key.endswith('connection') and \
            search(items, mysql_protocol, no_op):
             # Assume the "bind_address" option is one of
             # the token that made up the uri