Description of problem: Using Fedora 20 on both x86_64 and 686, all recent patches, but with NetworkManager disabled/removed (using network scripts). Hardware is physical - bare metal machine with single cpu AMD 140. On bootup, Named is starting before the configured NIC interfaces are available. It is only able to bind to localhost, later when the interfaces are available, it doesn't rebind and is therefore offline to external devices. By changing /usr/lib/systemd/system/named.service [Unit] Description=Berkeley Internet Name Domain (DNS) Wants=nss-lookup.target Before=nss-lookup.target After=network.target network-online.target (add network-online.target), it corrects the problem. I know the same problem is also happening to ipsec, and suspect bad things for DHCPD as well. See below for startup sequence. This problem became much more pronounced when I removed NetworkManager via yum, if it was happening I didn't recogonize the problem. Version-Release number of selected component (if applicable): rpm -qa|grep bind bind-license-9.9.4-12.P2.fc20.noarch bind-utils-9.9.4-12.P2.fc20.x86_64 system-config-bind-4.0.15-8.fc20.noarch rpcbind-0.2.1-0.2.fc20.x86_64 bind-libs-lite-9.9.4-12.P2.fc20.x86_64 bind-9.9.4-12.P2.fc20.x86_64 bind-libs-9.9.4-12.P2.fc20.x86_64 rpm -qa|grep network system-config-network-1.6.11-2.fc20.x86_64 dracut-network-037-11.git20140402.fc20.x86_64 glib-networking-2.38.2-1.fc20.x86_64 rpm -qa|grep systemd systemd-libs-208-19.fc20.x86_64 systemd-python-208-19.fc20.x86_64 systemd-python3-208-19.fc20.x86_64 systemd-208-19.fc20.x86_64 How reproducible: Near 100% Steps to Reproduce: 1. Enable named, bind to internal NIC only, not all. 2. systemctl disable NetworkManager, systemctl enable network 3. Reboot Actual results: Named is started before interfaces and unable to bind to them. Expected results: Able to serve named requests on internal interfaces Additional info: Portions of /var/log/messages showing relative startup. Jul 6 21:06:37 murdock named[841]: using default UDP/IPv4 port range: [1024, 65535] Jul 6 21:06:37 murdock named[841]: using default UDP/IPv6 port range: [1024, 65535] Jul 6 21:06:37 murdock named[841]: listening on IPv4 interface lo, 127.0.0.1#53 Jul 6 21:06:37 murdock named[841]: listening on IPv6 interface lo, ::1#53 Jul 6 21:06:37 murdock named[841]: generating session key for dynamic DNS Jul 6 21:06:37 murdock named[841]: sizing zone task pool based on 21 zones Jul 6 21:06:37 murdock named[841]: using built-in DLV key for view _default Jul 6 21:06:37 murdock named[841]: set up managed keys zone for view _default, file '/var/named/dynamic/managed-keys.bind' Jul 6 21:06:37 murdock named[841]: automatic empty zone: 10.IN-ADDR.ARPA ... Jul 6 21:06:37 murdock named[841]: managed-keys-zone: journal file is out of date: removing journal file Jul 6 21:06:37 murdock named[841]: managed-keys-zone: loaded serial 1891 Jul 6 21:06:37 murdock named[841]: zone 0.in-addr.arpa/IN: loaded serial 0 Jul 6 21:06:37 murdock named[841]: zone 0.20.10.IN-ADDR.ARPA/IN: loaded serial 581 Jul 6 21:06:37 murdock named[841]: zone 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa/IN: loaded serial 0 Jul 6 21:06:37 murdock named[841]: zone localhost.localdomain/IN: loaded serial 0 Jul 6 21:06:37 murdock named[841]: zone 1.0.0.127.in-addr.arpa/IN: loaded serial 0 Jul 6 21:06:37 murdock named[841]: zone localhost/IN: loaded serial 0 Jul 6 21:06:37 murdock named[841]: zone foddy.home/IN: ia.foddy.home/NS 'windward.ia.foddy.home' has no REQUIRED GLUE address records (A or AAAA) Jul 6 21:06:37 murdock named[841]: zone foddy.home/IN: loaded serial 987 Jul 6 21:06:37 murdock named[841]: all zones loaded Jul 6 21:06:37 murdock systemd: Started Berkeley Internet Name Domain (DNS). Jul 6 21:06:37 murdock systemd: Starting NFS file locking service.... Jul 6 21:06:37 murdock systemd: Starting Host and Network Name Lookups. Jul 6 21:06:37 murdock named[841]: running Jul 6 21:06:37 murdock systemd: Reached target Host and Network Name Lookups. Jul 6 21:06:38 murdock rpc.statd[921]: Version 1.3.0 starting Jul 6 21:06:38 murdock sm-notify[922]: Version 1.3.0 starting Jul 6 21:06:38 murdock kernel: [ 26.679136] ip_set: protocol 6 Jul 6 21:06:38 murdock kernel: ip_set: protocol 6 Jul 6 21:06:38 murdock kernel: [ 26.719113] ipt_CLUSTERIP: ClusterIP Version 0.8 loaded successfully Jul 6 21:06:38 murdock kernel: ipt_CLUSTERIP: ClusterIP Version 0.8 loaded successfully Jul 6 21:06:38 murdock systemd: Started NFS file locking service.. Jul 6 21:06:38 murdock systemd: Starting Network File System Server. Jul 6 21:06:38 murdock systemd: Reached target Network File System Server. Jul 6 21:06:38 murdock kernel: [ 27.050077] ipt_ULOG: ULOG is deprecated and it will be removed soon, use NFLOG instead Jul 6 21:06:38 murdock kernel: ipt_ULOG: ULOG is deprecated and it will be removed soon, use NFLOG instead Jul 6 21:06:38 murdock ModemManager[623]: <warn> Couldn't find support for device at '/sys/devices/pci0000:00/0000:00:05.0/0000:02:00.0': not supported by any plugin Jul 6 21:06:38 murdock ModemManager[623]: <warn> Couldn't find support for device at '/sys/devices/pci0000:00/0000:00:14.4/0000:03:05.0': not supported by any plugin Jul 6 21:06:38 murdock ModemManager[623]: <warn> Couldn't find support for device at '/sys/devices/pci0000:00/0000:00:14.4/0000:03:06.0': not supported by any plugin Jul 6 21:06:38 murdock shorewall: Compiling /etc/shorewall/zones... Jul 6 21:06:38 murdock shorewall: Running /etc/shorewall/initdone... Jul 6 21:06:38 murdock shorewall: WARNING: There are interfaces or zones with the 'blacklist' option, but the 'blacklist' file is either missing or has zero size Jul 6 21:06:38 murdock shorewall: Adding rules for DHCP Jul 6 21:06:38 murdock shorewall: Compiling TCP Flags filtering... Jul 6 21:06:38 murdock shorewall: Compiling Kernel Route Filtering... Jul 6 21:06:38 murdock shorewall: Compiling Martian Logging... Jul 6 21:06:38 murdock shorewall: Compiling /etc/shorewall/masq... Jul 6 21:06:38 murdock shorewall: Compiling MAC Filtration -- Phase 1... Jul 6 21:06:38 murdock shorewall: Compiling /etc/shorewall/rules... Jul 6 21:06:38 murdock shorewall: Compiling /etc/shorewall/conntrack... Jul 6 21:06:38 murdock shorewall: Compiling /etc/shorewall/tunnels... Jul 6 21:06:38 murdock shorewall: Compiling MAC Filtration -- Phase 2... Jul 6 21:06:38 murdock shorewall: Applying Policies... Jul 6 21:06:38 murdock shorewall: Compiling /usr/share/shorewall/action.Drop for chain Drop... Jul 6 21:06:38 murdock shorewall: Compiling /usr/share/shorewall/action.Broadcast for chain Broadcast... Jul 6 21:06:38 murdock shorewall: Compiling /usr/share/shorewall/action.Reject for chain Reject... Jul 6 21:06:38 murdock shorewall: Generating Rule Matrix... Jul 6 21:06:38 murdock shorewall: Creating iptables-restore input... Jul 6 21:06:38 murdock shorewall: Shorewall configuration compiled to /var/lib/shorewall/.start Jul 6 21:06:38 murdock shorewall: Starting Shorewall.... Jul 6 21:06:38 murdock shorewall: Initializing... Jul 6 21:06:39 murdock kernel: [ 27.757442] u32 classifier Jul 6 21:06:39 murdock kernel: [ 27.757445] Performance counters on Jul 6 21:06:39 murdock kernel: [ 27.757446] input device check on Jul 6 21:06:39 murdock kernel: [ 27.757447] Actions configured Jul 6 21:06:39 murdock kernel: u32 classifier Jul 6 21:06:39 murdock kernel: Performance counters on Jul 6 21:06:39 murdock kernel: input device check on Jul 6 21:06:39 murdock kernel: Actions configured Jul 6 21:06:39 murdock shorewall: Processing /etc/shorewall/init ... Jul 6 21:06:39 murdock shorewall: Processing /etc/shorewall/tcclear ... Jul 6 21:06:39 murdock shorewall: Setting up Route Filtering... Jul 6 21:06:39 murdock shorewall: WARNING: Cannot set route filtering on ppp0 Jul 6 21:06:39 murdock shorewall: Setting up Martian Logging... Jul 6 21:06:39 murdock shorewall: WARNING: Cannot set Martian logging on ppp0 Jul 6 21:06:39 murdock shorewall: Setting up Proxy ARP... Jul 6 21:06:39 murdock shorewall: Preparing iptables-restore input... Jul 6 21:06:39 murdock shorewall: Running /sbin/iptables-restore... Jul 6 21:06:39 murdock shorewall: IPv4 Forwarding Enabled Jul 6 21:06:39 murdock shorewall: Processing /etc/shorewall/start ... Jul 6 21:06:39 murdock shorewall: Processing /etc/shorewall/started ... Jul 6 21:06:39 murdock dbus-daemon: dbus[631]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (using servicehelper) Jul 6 21:06:39 murdock dbus[631]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (using servicehelper) Jul 6 21:06:39 murdock shorewall: cat: /var/tmp/ssh-blacklist-pending: Permission denied Jul 6 21:06:39 murdock logger: Shorewall started Jul 6 21:06:39 murdock shorewall: done. Jul 6 21:06:39 murdock systemd: Started Shorewall IPv4 firewall. Jul 6 21:06:39 murdock systemd: Starting Internet Key Exchange (IKE) Protocol Daemon for IPsec... Jul 6 21:06:39 murdock kernel: [ 28.198599] sha512_ssse3: Neither AVX nor SSSE3 is available/usable. Jul 6 21:06:39 murdock kernel: sha512_ssse3: Neither AVX nor SSSE3 is available/usable. Jul 6 21:06:39 murdock kernel: [ 28.209180] sha256_ssse3: Neither AVX nor SSSE3 is available/usable. Jul 6 21:06:39 murdock kernel: sha256_ssse3: Neither AVX nor SSSE3 is available/usable. Jul 6 21:06:39 murdock kernel: [ 28.254676] AVX instructions are not detected. Jul 6 21:06:39 murdock kernel: AVX instructions are not detected. Jul 6 21:06:39 murdock kernel: [ 28.349870] AVX instructions are not detected. Jul 6 21:06:39 murdock kernel: AVX instructions are not detected. Jul 6 21:06:39 murdock kernel: [ 28.352711] AVX instructions are not detected. Jul 6 21:06:39 murdock kernel: AVX instructions are not detected. Jul 6 21:06:39 murdock kernel: [ 28.404945] AVX instructions are not detected. Jul 6 21:06:39 murdock kernel: AVX instructions are not detected. Jul 6 21:06:39 murdock kernel: [ 28.419371] AVX instructions are not detected. Jul 6 21:06:39 murdock kernel: AVX instructions are not detected. Jul 6 21:06:39 murdock kernel: [ 28.454561] AVX or AES-NI instructions are not detected. Jul 6 21:06:39 murdock kernel: AVX or AES-NI instructions are not detected. Jul 6 21:06:39 murdock kernel: [ 28.457129] AVX or AES-NI instructions are not detected. Jul 6 21:06:39 murdock kernel: AVX or AES-NI instructions are not detected. Jul 6 21:06:39 murdock kernel: [ 28.593085] NET: Registered protocol family 15 Jul 6 21:06:39 murdock kernel: NET: Registered protocol family 15 Jul 6 21:06:39 murdock systemd: Started Internet Key Exchange (IKE) Protocol Daemon for IPsec. Jul 6 21:06:39 murdock sh: pluto: chdir() do dumpdir failed (2: No such file or directory) Jul 6 21:06:39 murdock pluto: pluto: chdir() do dumpdir failed (2: No such file or directory) Jul 6 21:06:40 murdock dbus-daemon: dbus[631]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd' Jul 6 21:06:40 murdock dbus[631]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd' Jul 6 21:06:42 murdock logger: initial_setup failed, keeping enabled Jul 6 21:06:42 murdock systemd: Received SIGRTMIN+20 from PID 1562 (kill). Jul 6 21:06:42 murdock systemd: Started Initial Setup configuration program (text mode). Jul 6 21:06:42 murdock abrt-hook-ccpp: File '/usr/sbin/plymouthd' seems to be deleted Jul 6 21:06:42 murdock setroubleshoot: Plugin Exception restorecon Jul 6 21:06:42 murdock setroubleshoot: Plugin Exception restorecon_source Jul 6 21:06:42 murdock setroubleshoot: SELinux is preventing /usr/bin/cat from open access on the file . For complete SELinux messages. run sealert -l 41a85e80-8e16-4768-94f4-257582a5a1c9 Jul 6 21:06:42 murdock python: SELinux is preventing /usr/bin/cat from open access on the file . ***** Plugin catchall (100. confidence) suggests ************************** If you believe that cat should be allowed open access on the file by default. Then you should report this as a bug. You can generate a local policy module to allow this access. Do allow this access for now by executing: # grep cat /var/log/audit/audit.log | audit2allow -M mypol # semodule -i mypol.pp Jul 6 21:06:42 murdock abrt-hook-ccpp: Saved core dump of pid 235 (/usr/sbin/plymouthd) to /var/tmp/abrt/ccpp-2014-07-06-21:06:42-235 (3694592 bytes) Jul 6 21:06:42 murdock systemd: plymouth-start.service: main process exited, code=dumped, status=6/ABRT Jul 6 21:06:42 murdock systemd: Unit plymouth-start.service entered failed state. Jul 6 21:06:42 murdock abrt-server: Generating core_backtrace Jul 6 21:06:42 murdock abrt-server: Generating backtrace Jul 6 21:06:42 murdock network: Bringing up interface p1p1: [ OK ] Jul 6 21:06:43 murdock kernel: [ 31.677124] IPv6: ADDRCONF(NETDEV_UP): p2p1: link is not ready Jul 6 21:06:43 murdock kernel: [ 31.677571] e100 0000:03:06.0 p2p1: NIC Link is Up 100 Mbps Full Duplex Jul 6 21:06:43 murdock kernel: IPv6: ADDRCONF(NETDEV_UP): p2p1: link is not ready Jul 6 21:06:43 murdock kernel: e100 0000:03:06.0 p2p1: NIC Link is Up 100 Mbps Full Duplex Jul 6 21:06:43 murdock kernel: [ 31.678319] IPv6: ADDRCONF(NETDEV_CHANGE): p2p1: link becomes ready Jul 6 21:06:43 murdock kernel: IPv6: ADDRCONF(NETDEV_CHANGE): p2p1: link becomes ready Jul 6 21:06:44 murdock abrt-server: Duplicate: core backtrace Jul 6 21:06:44 murdock abrt-server: DUP_OF_DIR: /var/tmp/abrt/ccpp-2014-05-24-15:59:42-235 Jul 6 21:06:44 murdock abrt-server: Deleting problem directory ccpp-2014-07-06-21:06:42-235 (dup of ccpp-2014-05-24-15:59:42-235) Jul 6 21:06:44 murdock dbus-daemon: dbus[631]: [system] Activating service name='org.freedesktop.problems' (using servicehelper) Jul 6 21:06:44 murdock dbus[631]: [system] Activating service name='org.freedesktop.problems' (using servicehelper) Jul 6 21:06:44 murdock dbus-daemon: dbus[631]: [system] Successfully activated service 'org.freedesktop.problems' Jul 6 21:06:44 murdock dbus[631]: [system] Successfully activated service 'org.freedesktop.problems' Jul 6 21:06:47 murdock NET[1767]: /etc/sysconfig/network-scripts/ifup-post : updated /etc/resolv.conf Jul 6 21:06:47 murdock network: Bringing up interface p2p1: [ OK ] Jul 6 21:06:47 murdock kernel: [ 36.353462] r8169 0000:02:00.0 p5p1: link down Jul 6 21:06:47 murdock kernel: [ 36.353511] IPv6: ADDRCONF(NETDEV_UP): p5p1: link is not ready Jul 6 21:06:47 murdock kernel: r8169 0000:02:00.0 p5p1: link down Jul 6 21:06:47 murdock kernel: IPv6: ADDRCONF(NETDEV_UP): p5p1: link is not ready Jul 6 21:06:47 murdock kernel: [ 36.358618] r8169 0000:02:00.0 p5p1: link down Jul 6 21:06:47 murdock kernel: r8169 0000:02:00.0 p5p1: link down Jul 6 21:06:50 murdock kernel: [ 38.765825] r8169 0000:02:00.0 p5p1: link up Jul 6 21:06:50 murdock kernel: [ 38.765843] IPv6: ADDRCONF(NETDEV_CHANGE): p5p1: link becomes ready Jul 6 21:06:50 murdock kernel: r8169 0000:02:00.0 p5p1: link up Jul 6 21:06:50 murdock kernel: IPv6: ADDRCONF(NETDEV_CHANGE): p5p1: link becomes ready Jul 6 21:06:51 murdock kernel: [ 40.314415] IPv4: martian destination 0.0.0.0 from 169.254.1.220, dev p5p1 Jul 6 21:06:51 murdock kernel: IPv4: martian destination 0.0.0.0 from 169.254.1.220, dev p5p1 Jul 6 21:06:52 murdock network: Bringing up interface p5p1: [ OK ] Jul 6 21:06:52 murdock kernel: [ 40.977729] PPP generic driver version 2.4.2 Jul 6 21:06:52 murdock kernel: PPP generic driver version 2.4.2 Jul 6 21:06:52 murdock pppd[1924]: pppd 2.4.5 started by root, uid 0 Jul 6 21:06:52 murdock pppd[1924]: Using interface ppp0 Jul 6 21:06:52 murdock pppd[1924]: Connect: ppp0 <--> /dev/pts/0 Jul 6 21:06:52 murdock dbus-daemon: string index out of range Jul 6 21:06:52 murdock dbus-daemon: 'list' object has no attribute 'split' Jul 6 21:06:52 murdock pppoe[1925]: PPP session is 9405 (0x24bd) Jul 6 21:06:53 murdock pppd[1924]: CHAP authentication succeeded Jul 6 21:06:53 murdock pppd[1924]: CHAP authentication succeeded Jul 6 21:06:53 murdock kernel: [ 42.319737] PPP BSD Compression module registered Jul 6 21:06:53 murdock kernel: PPP BSD Compression module registered Jul 6 21:06:53 murdock kernel: [ 42.353236] IPv4: martian destination 0.0.0.0 from 169.254.1.220, dev p5p1 Jul 6 21:06:53 murdock kernel: IPv4: martian destination 0.0.0.0 from 169.254.1.220, dev p5p1 Jul 6 21:06:53 murdock pppd[1924]: local IP address 216.160.0.218 Jul 6 21:06:53 murdock pppd[1924]: remote IP address 207.225.140.57 Jul 6 21:06:53 murdock pppd[1924]: primary DNS address 205.171.3.65 Jul 6 21:06:53 murdock pppd[1924]: secondary DNS address 205.171.2.65 Jul 6 21:06:55 murdock kernel: [ 44.403952] IPv4: martian destination 0.0.0.0 from 169.254.1.220, dev p5p1 Jul 6 21:06:55 murdock kernel: IPv4: martian destination 0.0.0.0 from 169.254.1.220, dev p5p1 Jul 6 21:06:57 murdock kernel: [ 46.447280] IPv4: martian destination 0.0.0.0 from 169.254.1.220, dev p5p1 Jul 6 21:06:57 murdock kernel: IPv4: martian destination 0.0.0.0 from 169.254.1.220, dev p5p1 Jul 6 21:06:58 murdock network: Bringing up interface ppp0: [ OK ] Jul 6 21:06:58 murdock systemd: Started LSB: Bring up/down networking. Jul 6 21:06:58 murdock systemd: Starting Network is Online. Jul 6 21:06:58 murdock systemd: Reached target Network is Online. # ifconfig -a lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 0 (Local Loopback) RX packets 992 bytes 112258 (109.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 992 bytes 112258 (109.6 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 p1p1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.0.2 netmask 255.255.255.0 broadcast 192.168.0.255 inet6 fe80::290:27ff:fe0e:30a5 prefixlen 64 scopeid 0x20<link> ether 00:90:27:0e:30:a5 txqueuelen 1000 (Ethernet) RX packets 11789531 bytes 13177980856 (12.2 GiB) RX errors 0 dropped 107 overruns 0 frame 0 TX packets 11147509 bytes 1279966765 (1.1 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 p2p1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.1 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::202:b3ff:fe90:7fc6 prefixlen 64 scopeid 0x20<link> ether 00:02:b3:90:7f:c6 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 14 bytes 900 (900.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 p5p1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.20.0.1 netmask 255.255.255.0 broadcast 10.20.0.255 inet6 fe80::6e62:6dff:fe52:f61e prefixlen 64 scopeid 0x20<link> ether 6c:62:6d:52:f6:1e txqueuelen 1000 (Ethernet) RX packets 11121352 bytes 1185134247 (1.1 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 11577914 bytes 13094561949 (12.1 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ppp0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1492 inet 216.160.0.218 netmask 255.255.255.255 destination 207.225.140.57 ppp txqueuelen 3 (Point-to-Point Protocol) RX packets 3985477 bytes 4445549714 (4.1 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 3548179 bytes 458618089 (437.3 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 /etc/named.conf (key portions) options { key-directory "/etc/named"; listen-on port 53 { 127.0.0.1; 10.20.0.1; 192.168.0.2; }; listen-on-v6 port 53 { ::1; }; allow-query { 127.0.0.1; 10.20.0/24; 10.20.1.0/24; 192.168.0.0/24; };
Reassigning to bind. They should either fix unit file and use *only* After=network-online.target or use IP_FREEBIND socket option when trying to bind socket to specific IP address.
Your log files list a bunch of possibly related, possibly unrelated problems. 1. plymouthd segfaults This is of course bad, but if this is on a server, you probably don't need plymouth, and might want to remove the package from the system and/or remove rhgb from the kernel command line 2. shorewall cannot start properly, becuase of selinux denial and possibly other issues. Are you sure this isn't screwing up your testing? 3. possible bind issues. Michal's comment deals with that. Please try to separate the issues.
Thank you for the response. I know there are several issues showing in the logs. These two servers were built a couple months ago, and have had several issues with them. As I see repeatable / non-stupid items I've been reporting them. Just haven't gotten to everything yet. The selinux issue with shorewall, while I haven't solved, isn't actually preventing the firewall from starting, but I haven't worked out what it is trying to do yet (trying to cat something). Anyway once I isolated the BIND9 named issue, and what the workaround could be, then I thought time to report.
The problem appears when upgrading systemd.x86_64 0:208-17.fc20 to systemd.x86_64 0:208-19.fc20 According to changelog, 208-18 contains interesting change: * Tue Jun 17 2014 Zbigniew Jędrzejewski-Szmek <zbyszek.pl> - 208-18 - Make SYSV $network be equivalent to network-online, not network target Probably this patch brokes habitual boot sequence: http://pkgs.fedoraproject.org/cgit/systemd.git/tree/0401-core-sysvcompat-network-should-be-equivalent-to-netw.patch?h=f20
The patch in question only delays the startup of services. It changes the generated services from: After=network.target to: After=network-online.target Wants=network-online.target But network-online.target is After=network.target, so the new requirements are a superset of the old ones.
So, as I don't have any SYSV services, something other change affects this? Anyway, manual changing of "After=network.target" to "After=network-online.target" in named.service fixes the situation.
It could be timing issue... Even systemd can queue jobs in a different order, etc. You can look at the dependency tree with: systemctl list-dependencies named.service # Wants/Requires systemctl list-dependencies --after named.service # After
*** Bug 1119815 has been marked as a duplicate of this bug. ***
(In reply to Michal Sekletar from comment #1) > Reassigning to bind. They should either fix unit file and use *only* > After=network-online.target or use IP_FREEBIND socket option when trying to > bind socket to specific IP address. Note that with systemd older than 208-18 everything works just fine ;) I think the changes systemd maintainer did in 208-18 broke things for a lot of services. I'm not going to use IP_FREEBIND as BIND has extensive logic to check which interface has specified address and if there is none, it does not even try to bind to it. Changing this would require non-trivial changes, which are not worth it ATM. However I'll discuss this with upstream if they want/plan to do anything about it. I'll change the network.target to network-online.target for the time being.
Fixed in: bind-9.9.5-8.P1.fc22 bind-9.9.5-8.P1.fc21 bind-9.9.4-14.P2.fc20
bind-9.9.4-15.P2.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/bind-9.9.4-15.P2.fc20
Package bind-9.9.4-15.P2.fc20: * should fix your issue, * was pushed to the Fedora 20 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing bind-9.9.4-15.P2.fc20' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-8477/bind-9.9.4-15.P2.fc20 then log in and leave karma (feedback).
bind-9.9.4-15.P2.fc20 works for me. Thanks, Corinna
bind-9.9.4-15.P2.fc20 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report.
The fix for this bug gives rise to a cyclic dependency when the glusterd service is also enabled on boot up, see /var/log/message extract below. As a result named doesn't start on boot up, requiring manual intervention on every reboot. Sep 1 08:20:25 test systemd: Found ordering cycle on glusterd.service/start Sep 1 08:20:25 test systemd: Found dependency on named.service/start Sep 1 08:20:25 test systemd: Found dependency on network-online.target/start Sep 1 08:20:25 test systemd: Found dependency on glusterd.service/start Sep 1 08:20:25 test systemd: Breaking ordering cycle by deleting job named.service/start Sep 1 08:20:25 test systemd: Job named.service/start deleted to break ordering cycle starting with glusterd.service/start Reverting the fix returns the system back to normal and named comes up as expected. Not sure if this is something that needs to be tackled in the named or glusterd unit files.
Filled new issue on glusterfs project, it seems cyclic dependency is some weirdness in their unit file. Before=network-online is used only by some services configuring network interfaces, NetworkManager-wait-online.service, systemd-networkd-wait-online.service and cloud-init.service. It does make sense on those. On glusterd.service I failed to found a reason for it. 1. https://github.com/gluster/glusterfs/issues/2778