Bug 1557327

Just ran into this with a fresh OSP 13 deploy.  I enabled tls for public-IP endpoints and deployed to my existing OSP13 overcloud (which was deployed fresh a few hours prior).  The deploy completed and I was able to still browse to the non-https overcloud as well as make non-SSL api calls.  When I tried to restart haproxy in PCS, it wouldn't start as and the haprox-bundle-docker-0,1 & 2 containers were not running on the respective controllers.  I did the following workaround which fixed it:

1.  mkdir -p  /var/lib/config-data/puppet-generated/haproxy/etc/pki/tls/
2.  cp /etc/pki/tls/private/overcloud_endpoint.pem /var/lib/config-data/puppet-generated/haproxy/etc/pki/tls/private
3.  pcs resource cleanup

We just hit this issue (or something very much like it) during an initial OSP 13 deployment (i.e. *not* while updating an existing stack).

The docker-puppet-haproxy container failed, with these errors:

[ALERT] 240/112930 (364) : Proxy 'aodh': no SSL certificate specified for bind '192.168.122.150:13042' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:28] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'cinder': no SSL certificate specified for bind '192.168.122.150:13776' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:42] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'glance_api': no SSL certificate specified for bind '192.168.122.150:13292' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:56] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'gnocchi': no SSL certificate specified for bind '192.168.122.150:13041' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:69] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'heat_api': no SSL certificate specified for bind '192.168.122.150:13004' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:90] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'heat_cfn': no SSL certificate specified for bind '192.168.122.150:13005' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:106] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'horizon': no SSL certificate specified for bind '172.17.1.150:443' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:121] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'horizon': no SSL certificate specified for bind '192.168.122.150:443' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:123] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'keystone_public': no SSL certificate specified for bind '192.168.122.150:13000' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:149] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'neutron': no SSL certificate specified for bind '192.168.122.150:13696' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:175] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'nova_novncproxy': no SSL certificate specified for bind '192.168.122.150:13080' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:199] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'nova_osapi': no SSL certificate specified for bind '192.168.122.150:13774' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:212] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'nova_placement': no SSL certificate specified for bind '192.168.122.150:13778' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:226] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'panko': no SSL certificate specified for bind '192.168.122.150:13977' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:240] (use 'crt').
[ALERT] 240/112930 (364) : Proxy 'swift_proxy_server': no SSL certificate specified for bind '192.168.122.150:13808' at [/etc/haproxy/haproxy.cfg20180829-12-4v19wf:267] (use 'crt').
[ALERT] 240/112930 (364) : Fatal errors found in configuration.

/etc/pki/tls/private/overcloud_endpoint.pem is a directory on the controllers.

What were the contents of enable-tls.yaml ?

Gonna try to reproduce it.

I just deployed a Queens environment and wasn't able to reproduce this on a new deployment. How did you try to deploy and what are the contents of enable-tls.yaml?

Ken, currently it's only possible to make such an update from non-TLS to TLS in containerized environments as part of a minor update or an upgrade. Did you use that workflow? This is because puppet-pacemaker in OSP13 doesn't have the ability to recreate the container if there is a change in the definition. This functionality was introduced on OSP14, so if you want this to work for stack updates, you would need to request a backport of that feature.

(In reply to Juan Antonio Osorio from comment #19)
> I just deployed a Queens environment and wasn't able to reproduce this on a
> new deployment. How did you try to deploy and what are the contents of
> enable-tls.yaml?

Unfortunately, this happened during a class, so we had to blow away the environment to move on.

I will try to reproduce next week once the course is over.

Not very useful I know, but I just wanted to get a note in this bug before it totally slipped my mind.  ;-)

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3789