Description of problem: keystone_public_api_node_ips and keystone_admin_api_node_ips are locked on controller nodes which leads to a broken overcloud deployment when keystone service is deployed on any other role than the controller. Deployment with the following services allocation: - name: Controller CountDefault: 1 ServicesDefault: - OS::TripleO::Services::CACerts - OS::TripleO::Services::CinderBackup - OS::TripleO::Services::CinderVolume - OS::TripleO::Services::Core - OS::TripleO::Services::Kernel - OS::TripleO::Services::MySQL - OS::TripleO::Services::RabbitMQ - OS::TripleO::Services::HAproxy - OS::TripleO::Services::Keepalived - OS::TripleO::Services::Memcached - OS::TripleO::Services::Pacemaker - OS::TripleO::Services::Redis - OS::TripleO::Services::Ntp - OS::TripleO::Services::SwiftProxy - OS::TripleO::Services::Snmp - OS::TripleO::Services::Timezone - OS::Tripleo::Services::ManilaShare - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::SensuClient - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::VipHosts - name: ServiceApi CountDefault: 1 ServicesDefault: - OS::TripleO::Services::CACerts - OS::TripleO::Services::CephMon - OS::TripleO::Services::CephExternal - OS::TripleO::Services::CephRgw - OS::TripleO::Services::CinderApi - OS::TripleO::Services::CinderScheduler - OS::TripleO::Services::Core - OS::TripleO::Services::Kernel - OS::TripleO::Services::Keystone - OS::TripleO::Services::GlanceApi - OS::TripleO::Services::GlanceRegistry - OS::TripleO::Services::HeatApi - OS::TripleO::Services::HeatApiCfn - OS::TripleO::Services::HeatApiCloudwatch - OS::TripleO::Services::HeatEngine - OS::TripleO::Services::NeutronDhcpAgent - OS::TripleO::Services::NeutronL3Agent - OS::TripleO::Services::NeutronMetadataAgent - OS::TripleO::Services::NeutronApi - OS::TripleO::Services::NeutronCorePlugin - OS::TripleO::Services::NeutronOvsAgent - OS::TripleO::Services::NovaConductor - OS::TripleO::Services::MongoDb - OS::TripleO::Services::NovaApi - OS::TripleO::Services::NovaMetadata - OS::TripleO::Services::NovaScheduler - OS::TripleO::Services::NovaConsoleauth - OS::TripleO::Services::NovaVncProxy - OS::TripleO::Services::Ntp - OS::TripleO::Services::SwiftStorage - OS::TripleO::Services::SwiftRingBuilder - OS::TripleO::Services::Snmp - OS::TripleO::Services::Timezone - OS::TripleO::Services::CeilometerApi - OS::TripleO::Services::CeilometerCollector - OS::TripleO::Services::CeilometerExpirer - OS::TripleO::Services::CeilometerAgentCentral - OS::TripleO::Services::CeilometerAgentNotification - OS::TripleO::Services::Horizon - OS::TripleO::Services::GnocchiApi - OS::TripleO::Services::GnocchiMetricd - OS::TripleO::Services::GnocchiStatsd - OS::Tripleo::Services::ManilaApi - OS::Tripleo::Services::ManilaScheduler - OS::Tripleo::Services::ManilaBackendGeneric - OS::Tripleo::Services::ManilaBackendNetapp - OS::Tripleo::Services::ManilaBackendCephFs - OS::TripleO::Services::AodhApi - OS::TripleO::Services::AodhEvaluator - OS::TripleO::Services::AodhNotifier - OS::TripleO::Services::AodhListener - OS::TripleO::Services::SaharaApi - OS::TripleO::Services::SaharaEngine - OS::TripleO::Services::IronicApi - OS::TripleO::Services::IronicConductor - OS::TripleO::Services::NovaIronic - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::OpenDaylight - OS::TripleO::Services::SensuClient - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::VipHosts results in the following haproxy.cfg on the controller nodes: listen keystone_admin bind 172.16.18.30:35357 transparent bind 192.168.0.20:35357 transparent mode http http-request set-header X-Forwarded-Proto https if { ssl_fc } http-request set-header X-Forwarded-Proto http if !{ ssl_fc } server cloudy-controller-0 192.168.0.22:35357 check fall 5 inter 2000 rise 2 server cloudy-controller-1 192.168.0.12:35357 check fall 5 inter 2000 rise 2 server cloudy-controller-2 192.168.0.21:35357 check fall 5 inter 2000 rise 2 listen keystone_public bind 10.0.0.13:5000 transparent bind 172.16.18.30:5000 transparent mode http http-request set-header X-Forwarded-Proto https if { ssl_fc } http-request set-header X-Forwarded-Proto http if !{ ssl_fc } server cloudy-controller-0 10.0.0.11:5000 check fall 5 inter 2000 rise 2 server cloudy-controller-1 10.0.0.19:5000 check fall 5 inter 2000 rise 2 server cloudy-controller-2 10.0.0.24:5000 check fall 5 inter 2000 rise 2 hiera keystone_admin_api_node_ips ["192.168.0.22", "192.168.0.12", "192.168.0.21"] hiera keystone_public_api_node_ips ["10.0.0.11", "10.0.0.19", "10.0.0.24"] These are ip addresses set on the controller nodes where the Keystone service is not listening. [root@cloudy-controller-0 heat-admin]# ip a | grep 10.0.0.11 inet 10.0.0.11/25 brd 10.0.0.127 scope global vlan200 [root@cloudy-controller-0 heat-admin]# lsof -i :5000 -n -P COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME haproxy 953 haproxy 27u IPv4 107227 0t0 TCP 10.0.0.13:5000 (LISTEN) haproxy 953 haproxy 28u IPv4 107228 0t0 TCP 172.16.18.30:5000 (LISTEN) [root@cloudy-controller-0 heat-admin]# ip a | grep 192.168.0.22 inet 192.168.0.22/25 brd 192.168.0.127 scope global eth0 [root@cloudy-controller-0 heat-admin]# lsof -i :35357 -n -P COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME haproxy 953 haproxy 25u IPv4 107225 0t0 TCP 172.16.18.30:35357 (LISTEN) haproxy 953 haproxy 26u IPv4 107226 0t0 TCP 192.168.0.20:35357 (LISTEN) Version-Release number of selected component (if applicable): openstack-tripleo-heat-templates-5.0.0-0.20160929150845.4cdc4fc.el7ost.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-2948.html