Bug 976666 - httpd does not start at boottime
httpd does not start at boottime
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: httpd (Show other bugs)
19
x86_64 Linux
unspecified Severity unspecified
: ---
: ---
Assigned To: Joe Orton
Fedora Extras Quality Assurance
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-21 02:59 EDT by A.J. Werkman
Modified: 2013-08-09 13:10 EDT (History)
12 users (show)

See Also:
Fixed In Version: httpd-2.4.6-2.fc19
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-08-09 13:10:27 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Httpd error_log (451 bytes, text/plain)
2013-06-21 02:59 EDT, A.J. Werkman
no flags Details
proposed patch (8.97 KB, patch)
2013-06-26 08:26 EDT, Jan Kaluža
no flags Details | Diff

  None (edit)
Description A.J. Werkman 2013-06-21 02:59:23 EDT
Created attachment 763676 [details]
Httpd error_log

Description of problem:
httpd does not start at boottime, because at httpd start network is not yet up and running

Version-Release number of selected component (if applicable):
19-TC6

How reproducible:
Everytime

Steps to Reproduce:
1. Install Fedora using the Web Server group
2. Reboot into the installed system.
3. See that the httpd service is not enabled (systemctl status httpd)
4. Enable the httpd service and reboot
5. See that the httpd service is not running (systemctl status httpd)
6. Inspect the http log directory (/var/log/httpd)
7. Only error_log has content

Actual results:
Httpd is not enabled and started, altough user asks for webserver installation.
Even when enabled httpd service does not start at boottime

Expected results:
Httpd being enabled and started at boottime without explicit user intervention

Additional info:
This does not seem to be httpd specific. I also noticed ntpd being effected. So probably all network related services are effected.

Looks like services are started before network is succsfully configured.

Because we are not able to start the services at boottime without user interventions I would think this is a candidate for blocker or freeze exeption.

Is this also related to bug #976034?
Comment 1 Jóhann B. Guðmundsson 2013-06-21 03:31:56 EDT
Moving against correct component and please try to refrain yourself from fiddling with the priority and severity status on the reports your file. 

These fields are used internally by developers themselves to scheduler their work load. 

Please attach the output from  journalctl -ba 

Thanks
Comment 2 Jan Kaluža 2013-06-21 04:34:26 EDT
I think that's the same problem as already described in Bug 916143.

I'm quoting my comment from that bug:
> You should be able to fix it by enabling this service:

> systemctl enable NetworkManager-wait-online.service

> Systemd presumes that applications can react on networking changes, but httpd
> does not support that. There is no way to configure httpd to wait until the IP
> address you want to use is bound. So if you have configured httpd to listen on
> specific IP instead of 0.0.0.0, you should ensure it's available when httpd is
> started by enabling NetworkManager-wait-online.service.

The proper fix for that is probably to make httpd aware of network configuration changes, but that needs big changes to happen in httpd upstream and current stable releases (or even httpd's upstream repository) don't contain that.

Please try to verify it's the same problem as in Bug 916143.

More documentation on that topic can be found here: http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/
Comment 3 A.J. Werkman 2013-06-22 07:39:58 EDT
(In reply to Jan Kaluža from comment #2)
> 
> The proper fix for that is probably to make httpd aware of network
> configuration changes, but that needs big changes to happen in httpd
> upstream and current stable releases (or even httpd's upstream repository)
> don't contain that.
> 
> Please try to verify it's the same problem as in Bug 916143.
> 
> More documentation on that topic can be found here:
> http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

This solution makes http boot at system startup.

Leaves two points:
- This solution is not generic. It does not work for ntpd.service. Should I file this as a separate bug?
- Why is not NetwerkManager-wait-online enabled if httpd.service is enabled? I would think, the end-user expects httpd to start at boottime if he requests a 'Web-server' install of fedora. The avarage user might not be able to find this work-around.
Comment 4 Joe Orton 2013-06-24 04:57:40 EDT
Enabling the "NetworkManager-wait-online" service if httpd is enabled would be surprising behaviour.

I'm afraid there is no "generally correct" solution here.  In most httpd configurations (and the default) it should not matter if httpd is started early, because httpd will bind to 0.0.0.0/:: and not care about specific interfaces, etc.

Whether or not you want httpd to start earlier or later than completion of network configuration is a per-system administrator choice; so our default is always going to be "wrong" for some users.  That's just the nature of defaults.
Comment 5 Michal Schmidt 2013-06-24 06:56:39 EDT
Jan, Joe,
does httpd use the IP_FREEBIND socket option (man 7 ip)? That would allow it to bind() to IP addresses that are not yet configured on any interface. And it's easier to implement than adding the awareness of network configuration changes.
Comment 6 Jan Kaluža 2013-06-24 08:16:00 EDT
That option would have to be added to APR package, since httpd does not use raw sockets (if I'm right), but uses APR for networking. I have suggested using that some months ago to fix this problem, but according to Joe there were some problems with using this option in APR.

I can still try to patch APR to include this option, but Joe has the final word as APR maintainer.
Comment 7 Joe Orton 2013-06-25 04:20:59 EDT
No, we can't enable FREEBIND for all listening sockets, it would be a change of semantics for the Listen directive and undesirable.  In any case, Listen is only one way in which httpd can be configured to fail if started w/o a network at boot time.
Comment 8 A.J. Werkman 2013-06-26 01:58:34 EDT
(In reply to Joe Orton from comment #4)
> Enabling the "NetworkManager-wait-online" service if httpd is enabled would
> be surprising behaviour.
> 
> I'm afraid there is no "generally correct" solution here.  In most httpd
> configurations (and the default) it should not matter if httpd is started
> early, because httpd will bind to 0.0.0.0/:: and not care about specific
> interfaces, etc.


I would like to point out, that in this bug I am using default settings. So it is not me configuring httpd in a way that it won't start at boot time.

IMHO this violates against the final release criterion no 13. "All services in a default install must start properly"
Comment 9 Jan Kaluža 2013-06-26 02:58:39 EDT
Hm, you are right. I have checked the error_log and it looks the root of problem is mod_uniqueid, which needs to know IP address of your hostname to compute unique IDs later. So it's not caused by not assigned IP addresses or change in default configuration, but by the fact that during boot your hostname does not have IP address assigned.

I will think what to do with that...
Comment 10 Jan Kaluža 2013-06-26 03:06:06 EDT
Just for a test, could you please try to disable mod_unique_id in /etc/httpd/conf.modules.d/00-base.conf and verify it's the only problematic module causing the httpd start problems during the boot on your network configuration?
Comment 11 Jan Kaluža 2013-06-26 08:26:36 EDT
Created attachment 765538 [details]
proposed patch
Comment 12 A.J. Werkman 2013-06-26 08:36:39 EDT
(In reply to Jan Kaluža from comment #10)
> Just for a test, could you please try to disable mod_unique_id in
> /etc/httpd/conf.modules.d/00-base.conf and verify it's the only problematic
> module causing the httpd start problems during the boot on your network
> configuration?

Disabling this module lets httpd start at bootup.
Comment 13 Jan Kaluža 2013-06-26 08:54:36 EDT
can you please try it with following scratch-build? it contains the patch I've attached in Commnent 11.

http://koji.fedoraproject.org/koji/taskinfo?taskID=5545990
Comment 14 Jan Kaluža 2013-06-26 08:55:15 EDT
(... with the module enabled of course)
Comment 15 A.J. Werkman 2013-06-27 15:45:00 EDT
Did a fresh Web-server install.
Enabled httpd.service.
Rebooted and verified that httpd did not start

Updated the httpd as asked in comment #13
Rebooted and now httpd does start at bootup
Comment 16 Fedora Update System 2013-07-31 10:21:38 EDT
httpd-2.4.6-2.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/httpd-2.4.6-2.fc19
Comment 17 Fedora Update System 2013-08-01 23:39:17 EDT
Package httpd-2.4.6-2.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing httpd-2.4.6-2.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-13994/httpd-2.4.6-2.fc19
then log in and leave karma (feedback).
Comment 18 Fedora Update System 2013-08-09 13:10:27 EDT
httpd-2.4.6-2.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.