Bug 1294506 - named.service does not bind to ethernet interface(s) on startup
named.service does not bind to ethernet interface(s) on startup
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: bind (Show other bugs)
7.2
All Linux
medium Severity medium
: rc
: ---
Assigned To: Tomáš Hozza
Petr Sklenar
Mirek Jahoda
: Patch
: 1278015 1301912 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-28 09:59 EST by John L Magee
Modified: 2016-11-03 21:26 EDT (History)
8 users (show)

See Also:
Fixed In Version: bind-9.9.4-36.el7
Doc Type: Release Note
Doc Text:
The *named* service now binds to all interfaces With this update, *BIND* is able to react to situations when a new IP address is added to an interface. If the new address is allowed by the configuration, *BIND* will automatically start to listen on that interface.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-11-03 21:26:00 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description John L Magee 2015-12-28 09:59:04 EST
Description of problem: the named.service does not bind to ethernet interface(s) on startup when the NetworkManager service is not installed


Version-Release number of selected component (if applicable):
CentOS 7 at 1511 level (equivalent to RHEL 7.2)


How reproducible: always

Steps to Reproduce:
1. create operational bind service on minimal VM
2. yum remove NetworkManager
3. reboot VM

Actual results:  named.service only bound to lo interface


Expected results:  named.service bound to all interfaces


Additional info:  changed dependencies in /etc/systemd/system/multi-user.target.wants/named.service  and  the named.service starts as expected

#After=network.target
After=network.service
Comment 2 Lukáš Nykrýn 2016-01-05 06:32:55 EST
This is weird, network.target is After network.service.

Can you please attach output of systemctl show network.target in the case without your workaround?

Can you also add "debug" to kernel cmdline and send me output of journactl -b?
Comment 3 Onmeac 2016-01-05 07:04:57 EST
similar https://bugzilla.redhat.com/show_bug.cgi?id=1278015?

I had run in this issue as well.
https://lists.centos.org/pipermail/centos-devel/2016-January/014194.html


One machine, multiple interfaces.

After a reboot named only listens to 127.0.0.1, not any of the IPv4 addresses.


sudo systemctl show network.target --no-pager

Id=network.target
Names=network.target
Conflicts=shutdown.target
Before=salt-minion.service named.service rc-local.service sshd.service network-online.target
After=network-pre.target network.service
Documentation=man:systemd.special(7) http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget
Description=Network
LoadState=loaded
ActiveState=inactive
SubState=dead
FragmentPath=/usr/lib/systemd/system/network.target
UnitFileState=static
UnitFilePreset=disabled
InactiveExitTimestampMonotonic=0
ActiveEnterTimestampMonotonic=0
ActiveExitTimestampMonotonic=0
InactiveEnterTimestampMonotonic=0
CanStart=no
CanStop=yes
CanReload=no
CanIsolate=no
StopWhenUnneeded=no
RefuseManualStart=yes
RefuseManualStop=no
AllowIsolate=no
DefaultDependencies=yes
OnFailureJobMode=replace
IgnoreOnIsolate=no
IgnoreOnSnapshot=no
NeedDaemonReload=no
JobTimeoutUSec=0
JobTimeoutAction=none
ConditionResult=no
AssertResult=no
ConditionTimestampMonotonic=0
AssertTimestampMonotonic=0
Transient=no
Comment 4 Carl Henrik Lunde 2016-02-23 04:47:42 EST
We've run into the same issue on RHEL 7.2 without NetworkManager with named-chroot-9.9.4-29.el7_2.2.x86_64.

It looks like bind-9.9.4 iterates over available interfaces on startup.  Listening to 0.0.0.0 is not an option[0].  This changes in bind-9.10 [1].

Waiting for network-online.target[2] seems to work:

$ diff -u /usr/lib/systemd/system/named-chroot.service /etc/systemd/system/named-chroot.service
--- /usr/lib/systemd/system/named-chroot.service        2016-01-18 13:57:35.000000000 +0100
+++ /etc/systemd/system/named-chroot.service    2016-02-23 10:12:25.276687016 +0100
@@ -7,7 +7,7 @@
 Wants=nss-lookup.target
 Requires=named-chroot-setup.service
 Before=nss-lookup.target
-After=network.target
+After=network-online.target
 After=named-chroot-setup.service

 [Service]

[0] http://permalink.gmane.org/gmane.network.dns.bind.user/31854

[1] https://kb.isc.org/article/AA-01153/0/BIND-9.10.0-Release-Notes.html

 * On operating systems that support routing sockets, including Mac OSX, *BSD and Linux, network interfaces are re-scanned automatically whenever they change.  Use "automatic-interface-scan no;" to disable this feature. [RT #23027]
 * Added "rndc scan" to trigger an interface scan manually. [RT #23027]

  https://source.isc.org/cgi-bin/gitweb.cgi?p=bind9.git;a=blob;f=bin/named/interfacemgr.c;h=ad8046421250663e332daf7f11c003300cbaa3c8;hb=HEAD#l929

[2] https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

"network.target has very little meaning during start-up. It only indicates that the network management stack is up after it has been reached."

"network-online.target is a target that actively waits until the nework is "up", where the definition of "up" is defined by the network management software. Usually it indicates a configured, routable IP address of some kind."
Comment 5 Tomáš Hozza 2016-05-10 04:16:15 EDT
*** Bug 1301912 has been marked as a duplicate of this bug. ***
Comment 6 Tomáš Hozza 2016-05-10 04:16:43 EDT
*** Bug 1278015 has been marked as a duplicate of this bug. ***
Comment 8 Tomáš Hozza 2016-05-11 13:58:14 EDT
Hello.

BIND 9.9.4 binds to any addresses, which are available during the server startup and are allowed by the configuration. Versions previous to 9.10.x are not able to automatically detect, that there is a new address on some interface or that some address disappeared (as noted in comment #4).

This is technically solved by the dispatcher script when using NetworkManger. It reloads the server on any network configuration change and thus makes it to bind to new addresses or stop listening on addresses which disappeared. Note that the dispatcher script causes BIND to do complete reload, which is unnecessary and may take long time in case your zone file is really big.

This naturally does not work when you uninstall NetworkManger, as the dispatcher script is not run.

However BIND 9.9.4 is scanning interfaces regularly, but the interval is by default set to 60 minutes. This can be changed using "interface-interval" option in the general "options" section. I don't think that changing the default is really a solution.

Note, that if you wait 60 minutes, BIND will eventually bind to all addresses ;)

I don't think that changing the "network.target" to "network-online.target" is either a good solution. The reason is that for some system services which depend on DNS, it may be too late if BIND starts only after the network is fully configured.

I propose to:
- keep the "After=network.target" in all named service files
- add the automatic scan functionality from BIND 9.10.x to BIND in RHEL-7
- remove the NetworkManager dispatcher script from the package

I backported the needed changes and everything seems to work fine. IOW BIND binds to all addresses even if NetworkManager is not installed.
Comment 15 Blaz Podrzaj 2016-09-02 06:34:50 EDT
(In reply to Carl Henrik Lunde from comment #4)

> "network.target has very little meaning during start-up. It only indicates
> that the network management stack is up after it has been reached."

I wouldn't agree with the first part of this statement, given the fact that there is quite a number of servers still (maybe on purpose) running without NetworkManager.

The problem is in fact with systemd which changed its way of activating network.target and network-online.target somewhere between version 208 and 219 and I've found that totally by accident... I restarted the server after 6 months and Postfix didn't come up. I checked the journal and found out that it was started but died afterwards because the system still haven't got an IP on which Postfix was configured. And I knew that this was not happening on RHEL 7.1.

With version 208 both targets were still working with network.service as well as with NM but with 219 network.target is NM-dependent only whereas network-online.target is still network.service dependent! But nearly all network services still have "After=network.target" in their configuration. So if you run your server without NM (not even installed) then your system ends up without network.target being activated at all and result is network services being started as soon as multi-user.target kicks in and possibly before network interfaces are brought up.

Whether they did that on purpose or was there some greater good in such a decision, I have no idea, but surely such an update messed up a lot of servers. 

So there are two possibilities if you would still like to bring up your network via network.service only... one is reconfiguring all your network services' systemd configs to "After=network-online.target" or two, install NM but leave it disabled. In 2nd case, you would get an info during the boot process that says "systemd[1]: Dependency failed for Network Manager Wait Online." and that's it.

Well, maybe I'm missing something but as I see it right now, the current implementation of systemd in RHEL 7.2 is broken. If you would still like to use network.service only, you would end up without network.target, but if you would like to switch completely to NM (without network.service), you would end up without network-online.target. This was not the case in RHEL 7.1 where you could run either network.service or NM alone or both at the same time and both network.target and network-online.target were active in each and every situation.
Comment 18 errata-xmlrpc 2016-11-03 21:26:00 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2233.html

Note You need to log in before you can comment on or make changes to this bug.