Bug 1441982

Summary:	[UPDATES] Update of mod_ssl package prevents haproxy from starting
Product:	Red Hat OpenStack	Reporter:	Marius Cornea <mcornea>
Component:	openstack-tripleo-heat-templates	Assignee:	Lukas Bezdicka <lbezdick>
Status:	CLOSED ERRATA	QA Contact:	Yurii Prokulevych <yprokule>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	10.0 (Newton)	CC:	achernet, bperkins, emacchi, jcoufal, lbezdick, mandreou, mburns, mcornea, rhel-osp-director-maint, sasha, sathlang, yprokule
Target Milestone:	async	Keywords:	Triaged, ZStream
Target Release:	10.0 (Newton)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	puppet-tripleo-5.5.0-12.el7ost openstack-tripleo-heat-templates-5.2.0-15.el7ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1441977
Clones:	1446289 1446292 1446293 1450825 (view as bug list)		Environment:
Last Closed:	2017-05-17 12:25:20 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1441977
Bug Blocks:	1446289, 1446292, 1446293, 1450825

Description Marius Cornea 2017-04-13 08:49:36 UTC

+++ This bug was initially created as a clone of Bug #1441977 +++

Description of problem:
-----------------------
Minor update of RHOS-11 fails cause haproxy is not running.
Looks like mod_ssl package is updated and pulls in  /etc/httpd/conf.d/ssl.conf, which has 'Listen 443' directive uncommented.
Apache gets restarted, binds to port and causes haproxy to fail:

Apr 13 08:27:26 controller-0.localdomain systemd[1]: Started Cluster Controlled haproxy.
Apr 13 08:27:26 controller-0.localdomain systemd[1]: Starting Cluster Controlled haproxy...
Apr 13 08:27:26 controller-0.localdomain haproxy-systemd-wrapper[168082]: haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
Apr 13 08:27:26 controller-0.localdomain haproxy-systemd-wrapper[168082]: [WARNING] 102/082726 (168083) : Setting tune.ssl.default-dh-param to 1024 by default, if your workload permits it you should set it to at
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy aodh started.
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy ceilometer started.
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy cinder started.
Apr 13 08:27:26 controller-0.localdomain haproxy-systemd-wrapper[168082]: [ALERT] 102/082726 (168083) : Starting proxy horizon: cannot bind socket [2620:52:0:13b8:5054:ff:fe3e:1:443]
Apr 13 08:27:26 controller-0.localdomain haproxy-systemd-wrapper[168082]: [ALERT] 102/082726 (168083) : Starting proxy horizon: cannot bind socket [fd00:fd00:fd00:2000::16:443]
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy glance_api started.
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy gnocchi started.
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy haproxy.stats started.
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy heat_api started.
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy heat_cfn started.
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy heat_cloudwatch started.
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy keystone_admin started.
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy keystone_public started.
Apr 13 08:27:26 controller-0.localdomain haproxy[168083]: Proxy mysql started.
Apr 13 08:27:26 controller-0.localdomain haproxy-systemd-wrapper[168082]: haproxy-systemd-wrapper: exit, haproxy RC=1
Apr 13 08:27:26 controller-0.localdomain systemd[1]: haproxy.service: main process exited, code=exited, status=1/FAILURE
Apr 13 08:27:26 controller-0.localdomain systemd[1]: Unit haproxy.service entered failed state.
Apr 13 08:27:26 controller-0.localdomain systemd[1]: haproxy.service failed.

ss -anp | grep 443
u_dgr  UNCONN     0      0      /run/systemd/cgroups-agent 1443                  * 0                   users:(("systemd",pid=1,fd=23))
tcp    LISTEN     0      128      :::443                  :::*                   users:(("httpd",pid=172103,fd=13),("httpd",pid=172102,fd=13),("httpd",pid=172101,fd=13),("httpd",pid=172100,fd=13),("httpd",pid=172099,fd=13),("httpd",pid=172098,fd=13),("httpd",pid=172097,fd=13),("httpd",pid=172096,fd=13),("httpd",pid=172050,fd=13))
tcp    SYN-SENT   0      1       fd00:fd00:fd00:2000::21:44350               fd00:fd00:fd00:2000::16:3306                users:(("neutron-server",pid=140764,fd=41))


Version-Release number of selected component (if applicable):
-------------------------------------------------------------
mod_ssl-2.4.6-45.el7_3.4.x86_64
openstack-tripleo-heat-templates-6.0.0-4.el7ost.noarch

Steps to Reproduce:
-------------------
1. Deploy RHOS-11 (2017-03-30.4)
2. Setup repos(2017-04-12.4)
3. Update UC
4. Try update OC

Comment 1 Red Hat Bugzilla Rules Engine 2017-04-13 08:49:45 UTC

This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 2 Marius Cornea 2017-04-13 08:50:42 UTC

The same issue happens with OSP10 minor update.

Comment 3 Sofer Athlan-Guyot 2017-04-28 12:13:21 UTC

This need more work if we don't want to have problem with osp9->osp10 upgrade.

As mod_ssl package is not in the osp9 image, we're going to have a similar problem than in https://bugzilla.redhat.com/show_bug.cgi?id=1445886.

We need to add the mod_ssl package during the upgrade of osp9->osp10.

Comment 4 Sofer Athlan-Guyot 2017-04-28 14:48:12 UTC

Added the review that add mod_ssl during the osp9->osp10 upgrade.

Comment 10 Yurii Prokulevych 2017-05-15 14:19:56 UTC

Updated to 2017-05-04.4 with 
- openstack-tripleo-heat-templates-5.2.0-15.el7ost.noarch 
- puppet-tripleo-5.5.0-12.el7ost.noarch

openstack stack list
+--------------------------------------+------------+-----------------+----------------------+----------------------+
| ID                                   | Stack Name | Stack Status    | Creation Time        | Updated Time         |
+--------------------------------------+------------+-----------------+----------------------+----------------------+
| b1f2adf4-b5f6-45c3-bda0-c1346174903f | overcloud  | UPDATE_COMPLETE | 2017-05-11T08:58:18Z | 2017-05-11T11:10:03Z |
+--------------------------------------+------------+-----------------+----------------------+----------------------+

Comment 12 errata-xmlrpc 2017-05-17 12:25:20 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:1242