Bug 1336365 - Satellite 6.2 - httpd fails to start after being updated
Summary: Satellite 6.2 - httpd fails to start after being updated
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Upgrades
Version: 6.2.0
Hardware: All
OS: Linux
high
high
Target Milestone: Unspecified
Assignee: Chris Roberts
QA Contact: Lukas Pramuk
URL: http://projects.theforeman.org/issues...
Whiteboard:
: 1339298 1340310 1367188 (view as bug list)
Depends On:
Blocks: GSS_Sat6Beta_Tracker, GSS_Sat6_Tracker
TreeView+ depends on / blocked
 
Reported: 2016-05-16 09:37 UTC by Julio Entrena Perez
Modified: 2020-02-14 17:46 UTC (History)
30 users (show)

Fixed In Version: katello-installer-base-3.0.0.61-1
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-10 16:00:52 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 16972 0 Normal Closed httpd fails to start after mod_ssl updated 2021-02-10 19:00:24 UTC
Red Hat Knowledge Base (Solution) 2362111 0 None None None 2016-06-13 09:16:27 UTC
Red Hat Knowledge Base (Solution) 2751421 0 None None None 2016-11-09 21:09:48 UTC
Red Hat Product Errata RHBA-2016:1885 0 normal SHIPPED_LIVE Satellite 6.2.2 bug fix update 2016-09-15 00:57:56 UTC

Description Julio Entrena Perez 2016-05-16 09:37:47 UTC
Description of problem:
Running "yum update" on a Satellite 6.2 beta installed on RHEL 7 results in package httpd being updated from 2.4.6-40.el7 to 2.4.6-40.el7_2.1.
This causes previously non existent file /etc/httpd/conf.d/ssl.conf to be created.
Running "foreman-installer --scenario katello --upgrade" does not remove /etc/httpd/conf.d/ssl.conf and httpd fails to start.
"Listen 443" in /etc/httpd/conf.d/ssl.conf conflicts with "Listen 443" on /etc/httpd/conf/ports.conf .

Version-Release number of selected component (if applicable):
foreman-installer-1.11.0.3-1.el7sat

How reproducible:
Always

Steps to Reproduce:
1. Run "yum update" on the Satellite and have httpd package updated.
2. Running "foreman-installer --scenario katello --upgrade"
3.

Actual results:
httpd fails to start:

# foreman-installer --scenario katello --upgrade
Upgrading...
Upgrade Step: stop_services...
Upgrade Step: start_databases...
Upgrade Step: migrate_pulp...
Upgrade Step: start_httpd...
Upgrade step start_httpd failed. Check logs for more information.

# tail /var/log/httpd/error_log
[Mon May 16 09:26:03.951254 2016] [core:crit] [pid 46664] (22)Invalid argument: AH00069: make_sock: for address [::]:443, apr_socket_opt_set: (IPV6_V6ONLY)
(98)Address already in use: AH00072: make_sock: could not bind to address [::]:443
(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:443
[Mon May 16 09:26:03.951314 2016] [mpm_prefork:alert] [pid 46664] no listening sockets available, shutting down
[Mon May 16 09:26:03.951317 2016] [:emerg] [pid 46664] AH00019: Unable to open logs, exiting
/usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:105:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED)
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:105:in `new'
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:105:in `connect'
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:86:in `socket'
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:90:in `head_request'
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:145:in `<main>'
/usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:105:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED)
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:105:in `new'
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:105:in `connect'
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:112:in `connect'
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:86:in `socket'
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:90:in `head_request'
        from /usr/share/gems/gems/passenger-4.0.18/helper-scripts/prespawn:145:in `<main>'

Duplicated "Listen 443" directives as a result of httpd package provided file /etc/httpd/conf.d/ssl.conf not being cleared by "foreman-installer --scenario katello --upgrade":

# grep -r "Listen 443" /etc/httpd
/etc/httpd/conf/ports.conf:Listen 443
/etc/httpd/conf.d/ssl.conf:Listen 443 https

Expected results:
"foreman-installer --scenario katello --upgrade" resolves the conflict and httpd starts successfully.

Additional info:

Comment 1 Tomer Brisker 2016-05-25 08:58:17 UTC
*** Bug 1339298 has been marked as a duplicate of this bug. ***

Comment 2 Tomer Brisker 2016-05-25 09:05:10 UTC
To reproduce if you installed satellite after mod_ssl was updated, run:
mv /etc/httpd/conf.d/ssl.conf /etc/httpd/conf.d/ssl.conf.katello && yum reinstall mod_ssl
This is also present in Satellite 6.1

Comment 4 Tomer Brisker 2016-05-30 09:13:20 UTC
*** Bug 1340310 has been marked as a duplicate of this bug. ***

Comment 10 Alexander Love 2016-06-21 18:41:18 UTC
For me I was getting "pgrade step start_httpd failed. Check logs for more information" and "httpd[8068]: (98)Address already in use: AH00072: make_sock: could not bind to address [::]:443"

I edited the /etc/httpd/conf/ports.conf commited out "Listen 443"

Ran a systemctl restart httpd and checked the status with systemctl status httpd to ensure it was running.

after this I reran "foreman-installer --scenario katello --upgrade"

The installed then ran correctly and I checked /etc/httpd/conf/ports.conf file and the "Listen 443" was no longer commited out.

Hope this works for you.

Comment 12 Chris Roberts 2016-07-25 20:34:56 UTC
Hi Alexander,

We use the ports.conf for the 443 directive and we filebucket the ssl.conf. I am working so instead of removing the ssl.conf we make a blank file so when mod_ssl gets updated it will not replace the empty file.

Since we use ports.conf the file got corrected with a puppet run when you ran the installer.

Comment 13 Chris Roberts 2016-07-25 20:36:25 UTC
non existent*

Comment 14 jcallaha 2016-09-12 18:05:45 UTC
Verified in Satellite 6.2.2 Snap 1.1

Followed instructions in Comment #2. Install completed correctly and httpd had no issues post-install.

[root@ibm-x3550m3-07 yum.repos.d]# grep -r "Listen 443" /etc/httpd
/etc/httpd/conf/ports.conf:Listen 443

Comment 16 Chris Roberts 2016-09-14 17:44:22 UTC
*** Bug 1367188 has been marked as a duplicate of this bug. ***

Comment 17 errata-xmlrpc 2016-09-14 21:00:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1885

Comment 18 Lukas Pramuk 2016-10-11 10:34:06 UTC
By a fleeting glance at fix I'm pretty sure that bug is not fixed and persists ... thus reopening.

Comment 19 Lukas Pramuk 2016-10-11 10:46:18 UTC
1. To check whether is your Satellite vulnerable against any upcoming mod_ssl package update run following command,i.e to check that it's not fixed: 

# ll /etc/httpd/conf.d/ssl.conf
ls: cannot access /etc/httpd/conf.d/ssl.conf: No such file or directory

No such file or dir? => Is vulnerable!


2. To manually prevent this to happen run following command:

# cat <<< '' > /etc/httpd/conf.d/ssl.conf
(as shown in my duplicate https://bugzilla.redhat.com/show_bug.cgi?id=1367188#c3)

That's it. Now you're safe.


3. We still need to fix it that way in Satellite's installation.
Post-deleting or pre-deleting the file doesn't help at all.
The file has to be empty. That's it.

Comment 23 Bryan Kearney 2016-10-11 12:15:07 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/15380 has been resolved.

Comment 25 Bryan Kearney 2016-10-11 14:15:26 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/15380 has been resolved.

Comment 33 Lukas Pramuk 2016-11-01 15:46:43 UTC
VERIFIED.

@satellite-6.2.4-1.0.el7sat.noarch
katello-installer-base-3.0.0.63-2.el7sat.noarch

with this manual reproducer:

1. Install (or upgrade) Satellite/Capsule
# satellite-installer (--upgrade)

2. Verify that mod_ssl configuration file ssl.conf is present
$ cat /etc/httpd/conf.d/ssl.conf
# This file is managed by the foreman-installer, do not alter.

>>> ssl.conf is present, so it wont get re-created by mod_ssl package updates with undesirable content ofc

Comment 35 gabicr 2016-11-07 06:28:20 UTC
Hello!

Just upgraded my 6.2.3 Sat OS from 7.2 to 7.3....of course httpd included and the bug hit me! So i just mv ssl.conf and cat <<< '' > /etc/httpd/conf.d/ssl.conf.

Comment 36 Fred van Zwieten 2016-11-07 09:25:53 UTC
Same happened to me as #35. I don't know what the content of the file sss.conf was before doing the upgrade, but I was on 6.2.3 doing a normal sat upgrade procedure.

Comment 42 Peter Vreman 2016-11-08 15:42:38 UTC
Same for me. Doing only an OS upgrade of 7.2 to 7.3 with the standard 'yum update' and reboot

Comment 44 Jason Powell 2016-11-09 22:10:55 UTC
Confirming the bug exists in our environment as well. Just updated satellite server from 7.2 -> 7.3, mod_ssl was updated, and httpd would not start because of a conflicting Listen directive on port 443 (Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:443) and removing /etc/httpd/conf.d/ssl.conf allowed httpd to start normally.

Comment 45 Bryan Kearney 2016-11-10 16:00:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:2699

Comment 46 David Rock 2016-11-10 16:56:38 UTC
I don't think you have the correct errata referenced.  This BZ is not listed in RHBA-2016:2699.  Either it wasn't fixed, or it's in a different errata.

Comment 48 Ryan Sawhill 2016-11-10 17:04:59 UTC
(In reply to David Rock from comment #46)
> I don't think you have the correct errata referenced.  This BZ is not listed
> in RHBA-2016:2699.  Either it wasn't fixed, or it's in a different errata.

... Or they mistakenly omitted it. I'm seeking confirmation about the oddity, David.

Comment 49 David Rock 2016-11-10 17:08:47 UTC
(In reply to Ryan Sawhill from comment #48)
> ... Or they mistakenly omitted it. I'm seeking confirmation about the
> oddity, David.

true.. :-)

FWIW, I'm also curious how it was fixed.  Does satellite not care about the mod_ssl ssl.conf file anymore?

Comment 50 Bryan Kearney 2016-11-10 17:30:13 UTC
Per comment 33, we are adding an empty ssl.conf file at install so that it will not be overwritten by the http package. See http://projects.theforeman.org/issues/16972.

Comment 51 Ryan Sawhill 2016-11-10 17:40:12 UTC
So (In reply to David Rock from comment #46)
> I don't think you have the correct errata referenced.  This BZ is not listed
> in RHBA-2016:2699.  Either it wasn't fixed, or it's in a different errata.

So to recap David: according to Bryan, they mistakenly omitted mention of this BZ in the RHBA announced today.

Comment 52 David Rock 2016-11-10 18:17:23 UTC
Sounds good on both counts.  I've verified the ssl.conf looks as advertised at this end. 

Thanks! :-)

Comment 53 Peter Vreman 2016-11-11 10:33:22 UTC
As a user i was also suprised when my case got closed. I checked also after upgrading to 6.2.4 and confirm that ssl.conf is now managed by the satellite installer

$ cat /etc/httpd/conf.d/ssl.conf
# This file is managed by the foreman-installer, do not alter.


Note You need to log in before you can comment on or make changes to this bug.