Bug 1999334 - bind does not listen on all addresses over TCP when listen-on/listen-on-v6 has specific IPs or any listed
Summary: bind does not listen on all addresses over TCP when listen-on/listen-on-v6 ha...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: bind
Version: 34
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Petr Menšík
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1999857 1999691
TreeView+ depends on / blocked
 
Reported: 2021-08-30 22:47 UTC by Bojan Smojver
Modified: 2021-09-27 01:22 UTC (History)
10 users (show)

Fixed In Version: bind-9.16.21-1.fc35 bind-9.16.21-1.fc34
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1999691 (view as bug list)
Environment:
Last Closed: 2021-09-27 01:22:27 UTC
Type: Bug


Attachments (Terms of Use)
reproducer.sh (922 bytes, text/plain)
2021-08-31 15:06 UTC, Petr Menšík
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Internet Systems Consortium (ISC) isc-projects bind9 issues 2852 0 None None None 2021-08-31 07:21:38 UTC

Description Bojan Smojver 2021-08-30 22:47:48 UTC
Description of problem:
On boot, named will not listen on TCP sockets of all specified (or any) IP addresses. This appears to be an ordering problem during startup. When service is restarted by hand, it starts listening on TCP too.

Version-Release number of selected component (if applicable):
bind-9.16.20-3.fc34.x86_64

How reproducible:
Always.

Steps to Reproduce:
1. Configure specific IPs in listen-on/listen-on-v6 directives or any.
2. Boot machine with named-chroot service enabled.
3. Named only listening on all IPs over UDP.

Actual results:
Listens on all addresses over UDP only.

Expected results:
Should listen on TCP too, not just on the loopback address.

Additional info:
I tried adding:
-----
[Unit]
Wants=network-online.target
After=network-online.target
-----

in /etc/systemd/system/named-chroot.service.d/named-chroot.conf file and that did help.

Comment 1 Petr Špaček 2021-08-31 07:21:39 UTC
I think this is related to upstream issue #2852, please continue there.

Comment 2 Petr Menšík 2021-08-31 14:15:17 UTC
Indeed, Petr seems to be correct, it looks related to upstream issue linked.

Can you please check journalctl would contain additionally listening?

journalctl -xeu named-chroot | grep 'additionally listening'

I were unable to reproduce this issue on my test machine. My named listens on all IPs with both UDP and TCP, checked by:
lsof -n -p $(pidof named) | grep :domain

Is this issue reproducible on every reboot on your system?
Would "rndc scan" command fix missing listeners?

Comment 3 Petr Menšík 2021-08-31 15:06:24 UTC
Created attachment 1819450 [details]
reproducer.sh

I were able to reproduce this issue with reproducer script provided on upstream. It seems reliable when named starts and passes scanning of addresses in correct place.

Comment 4 Bojan Smojver 2021-08-31 22:29:18 UTC
(In reply to Petr Menšík from comment #2)

> Can you please check journalctl would contain additionally listening?
> 
> journalctl -xeu named-chroot | grep 'additionally listening'

No matches.

> I were unable to reproduce this issue on my test machine. My named listens
> on all IPs with both UDP and TCP, checked by:
> lsof -n -p $(pidof named) | grep :domain
> 
> Is this issue reproducible on every reboot on your system?

Yes, it was. I actually only discovered this because renewal of Let's Encrypt cert failed on this system, due to DNS errors. It took me a while to realise that Let's Encrypt's ACME uses TCP for DNS. So, my named-chroot was limping along using UDP for everyone and TCP on localhost for a while, which made it sort of semi-functional.

> Would "rndc scan" command fix missing listeners?

I have not tried that. But, after I ordered named-chroot after network-online.target, the problem went away. Or at least it did across a couple of reboots.

Comment 5 Petr Menšík 2021-09-01 13:20:49 UTC
Perhaps it is no longer visible at the end of logs, because you have found network-online.target workaround. Could you please try just journalctl -u named-chroot | grep 'additionally listening' and check, whether they were logged at time it did not listen on TCP properly? In case those logs are not already cleaned, of course. Do you know the date, when Let's Encrypt renewal occurred? Can you please check logs of named-chroot on that date?

Does grep 'additionally listening' -r /var/named/data/named.run* contain them at least?

Comment 6 Bojan Smojver 2021-09-01 14:02:42 UTC
First time this failed was on 30 Aug 2021 at 00:01 AEST (i.e. +10 TZ). My journal goes back to April. There is also nothing in that other location.

My named-chroot does run from /var/named, but my named.conf is very custom. So, that is the likely reason I cannot find these entries.

Comment 7 Fedora Update System 2021-09-18 12:14:40 UTC
FEDORA-2021-aebf5de259 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-aebf5de259

Comment 8 Fedora Update System 2021-09-18 12:18:12 UTC
FEDORA-2021-3215da7cce has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-3215da7cce

Comment 9 Fedora Update System 2021-09-19 01:49:09 UTC
FEDORA-2021-3215da7cce has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-3215da7cce`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-3215da7cce

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 10 Fedora Update System 2021-09-19 05:42:24 UTC
FEDORA-2021-aebf5de259 has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-aebf5de259`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-aebf5de259

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 11 Fedora Update System 2021-09-24 20:28:22 UTC
FEDORA-2021-3215da7cce has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 12 Bojan Smojver 2021-09-25 00:48:49 UTC
Unfortunately, this has not been fixed. I removed my workaround and after today's reboot, named is only listening on localhost again.

Comment 13 Bojan Smojver 2021-09-25 00:53:33 UTC
# netstat -tlnp | grep :53
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      876/named           
tcp6       0      0 ::1:53                  :::*                    LISTEN      876/named

# grep listen /etc/named.conf 
	listen-on { 127.0.0.1; <another-ip4>; };
        listen-on-v6 { ::1/128; <another-ip6>; };

Comment 14 Fedora Update System 2021-09-27 01:22:27 UTC
FEDORA-2021-aebf5de259 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.