Created attachment 1802981 [details] bash shell script that lists all listening or connected sockets Description of problem: Today I wrote a script, providing a 'sockets' function that lists processes listening on sockets (attached), and found that for some reason, systemd-journald had bound to the socket <IPADDR_ANY>:<port 53> : $ sockets ... /usr/lib/systemd/systemd-journald[1056]:5 udp: 0.0.0.0:53 0.0.0.0:0 00000000 00000000 I realize that systemd-journald is started before my named instance , which certainly MUST bind to port 53 on each interface - it does not actually bind to IPADDR_ANY, (0.0.0.0) but to the primary interface address of each active interface: /usr/sbin/named[773124]:100 udp: 127.0.0.1:53 0.0.0.0:0 00000000 00000000 /usr/sbin/named[773124]:133 udp: 192.168.122.1:53 0.0.0.0:0 00000000 00000000 /usr/sbin/named[773124]:96 tcp: 127.0.0.1:953 0.0.0.0:0 00000000 00000000 I was getting DNS lookup timeout issues at boot - now I know why. Maybe the parent of systemd-journald opened a port 53 socket and forgot to close it ? Is there any way I disable this listening of systemd / systemd-journald to port 53 ? Now, to stop my DNS timeouts at boot issue, I have to do: $ systemctl stop systemd-journald $ systemctl restart named $ systemctl start systemd-journald I use the ISC bind 9.6.18 named daemon for various essential tasks - caching-only local resolver, various home DMZ LAN hostnames, and a Response Policy AdWare / SpyWare RBL blocker. Having systemd / systemd-journald also listening on port 53 does seem get in the way of using ISC bind named at all. Version-Release number of selected component (if applicable): 248.4-1.fc34.x86_64 How reproducible: 100% Steps to Reproduce: Immediately after first boot, with both 'named' and 'systemd-journald' services enabled, and an /etc/resolv.conf containing: nameserver 127.0.0.1 , run the attached script: $ . ./sockets.sh $ sockets Actual results: Script might output includes the report of a UDP socket bound to 0.0.0.0:53 owned by systemd-journald or systemd . Expected results: The only process listening to port 53 should be named if named service is enabled. Additional info:
Oops - should have wrote: Actual results: Script output might include the report of a UDP socket bound to 0.0.0.0:53 owned by systemd-journald or systemd .
Actually, both systemd and systemd-journald have a socket bound to 0.0.0.0:53 . Restarting systemd-journald makes it drop its port 53 socket, but how do I make systemd drop its port 53 socket ? Surely, there has to be some way of telling systemd not to listen on this port. But it does not seem to be easy to find in the documentation.
Created attachment 1802996 [details] same script as a text file, for browsing Script that prints all processes' listening sockets - usage: $ . ./socket.sh $ sockets or $ sockets +U
# egrep -RIw 53 /etc/systemd /usr/lib/systemd/ /usr/lib/systemd/resolv.conf:nameserver 127.0.0.53 /usr/lib/systemd/system/ntpviz-daily.timer:OnCalendar=*:53:00 # { man systemd ; man systemd-journald ; } | egrep -iw 'dns|53' # There does not seem to be any documentation on why systemd is listening on port 53. The named systemd config file (system/multi-user.target.wants/named.service) is unchanged from RPM defaults and makes no mention of port 53 . Perhaps it should do so. There are no matches for port 53 anywhere in my /etc files , or under /usr/{lib,share}/systemd . So, please, is there a way to disable systemd'd listeing on port 53, or do I have to modify the source and build a custom systemd version that does not listen to it?
This may be something to do with my decision to $ systemctl disable systemd-resolved since my /etc/resolv.conf never changes ( it just contains: nameserver 127.0.0.1 ). To demonstrate the sort of problems this causes, consider what happens when a new interface address is brought up: [root@jvdspc:/etc/systemd [2968] 15:31:22 #:701!:8628]{0} # ip addr add dev lo 127.4.2.1/32 [root@jvdspc:/etc/systemd [2968] 15:32:40 #:702!:8629]{0} # ping 127.4.2.1 PING 127.4.2.1 (127.4.2.1) 56(84) bytes of data. 64 bytes from 127.4.2.1: icmp_seq=1 ttl=64 time=0.074 ms 64 bytes from 127.4.2.1: icmp_seq=2 ttl=64 time=0.060 ms ^C --- 127.4.2.1 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1058ms rtt min/avg/max/mdev = 0.060/0.067/0.074/0.007 ms [root@jvdspc:/etc/systemd [2968] 15:32:55 #:703!:8630]{0} # dig jvdspc @127.4.2.1 ; <<>> DiG 9.16.18-RH <<>> jvdspc @127.4.2.1 ;; global options: +cmd ;; connection timed out; no servers could be reached [root@jvdspc:/etc/systemd [2968] 15:33:32 #:704!:8631]{9} # dig jvdspc ; <<>> DiG 9.16.18-RH <<>> jvdspc ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39364 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ; COOKIE: 23b44848c004f9d20100000060f43bc81a5ed0d28c4808c1 (good) ;; QUESTION SECTION: ;jvdspc. IN A ;; ANSWER SECTION: jvdspc. 86400 IN A 127.0.0.1 ;; Query time: 0 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Sun Jul 18 15:33:44 IST 2021 ;; MSG SIZE rcvd: 79 'jvdspc' is a fake-TLD that my instance of named has defined, and is also an alias of 'loghost' and an alias of 'localhost' in my /etc/hosts. When a new interface address is configured, named will not be listening to '0.0.0.0:53', because systemd is, and new DNS requests will not be forwarded to my DNS server. This is very bad - please give systemd a way of being told not to listen on port 53.
Created attachment 1803009 [details] Browable version of script Same script - browsable this time?
Maybe something to do with -Ddefault-llmnr=resolve in systemd.spec ? I am changing that to -Ddefault-llmnr=no and rebuilding, after fixing: Bug 1983433 and Bug 1983450 .
Sorry, that should have been: 'after fixing: Bug 1983443 and Bug 1983450' .
Aha, yes, after building systemd-248.4-2 with patches as above all is OK - neither systemd nor systemd-journald now have a bound IPADDR_ANY socket to port 53 , when I changed systemd.spec to build with '-Ddefault_llmnr=no' instead of '-Ddefault_llmnr=resolve' - a lucky first time guess, aided by 'egrep -RIw resolv' ... Please consider making this a runtime rather than a compile time option.
(and after yum updating & rebooting, I should add
Output of sockets with patch: # sockets /usr/bin/emacs-27.2[12668]:16 tcp: 127.0.0.1:48202 127.0.0.1:34319 00000000 00000000 /usr/bin/sbcl[12680]:4 tcp: 127.0.0.1:34319 127.0.0.1:48202 00000000 00000000 /usr/lib64/firefox/firefox[12703]:136 tcp: 192.168.1.32:47452 34.216.185.123:443 00000000 00000000 /usr/lib64/firefox/firefox[12703]:198 udp: 0.0.0.0:55414 0.0.0.0:0 00000000 00000000 /usr/lib64/firefox/firefox[12703]:202 tcp: 192.168.1.32:51590 74.125.193.147:443 00000000 00000000 /usr/lib64/firefox/firefox[12703]:253 tcp: 192.168.1.32:39364 209.132.183.69:443 00000000 00000000 /usr/lib64/firefox/firefox[12703]:256 tcp: 192.168.1.32:39968 74.125.193.100:443 00000000 00000000 /usr/bin/qemu-system-x86_64[13097]:10 tcp: 127.0.0.1:5900 0.0.0.0:0 00000000 00000000 /usr/sbin/sendmail.sendmail[13123]:6 tcp: 127.0.0.1:25 0.0.0.0:0 00000000 00000000 /usr/sbin/avahi-daemon[1331]:15 udp: 0.0.0.0:5353 0.0.0.0:0 00000000 00000000 /usr/sbin/avahi-daemon[1331]:17 udp: 0.0.0.0:58469 0.0.0.0:0 00000000 00000000 /usr/sbin/chronyd[1392]:6 udp: 127.0.0.1:323 0.0.0.0:0 00000000 00000000 /usr/sbin/cupsd[1483]:10 tcp: 127.0.0.1:631 0.0.0.0:0 00000000 00000000 /usr/sbin/sshd[1497]:5 tcp: 0.0.0.0:22 0.0.0.0:0 00000000 00000000 /usr/sbin/named[1530]:100 udp: 127.0.0.1:53 0.0.0.0:0 00000000 00000000 /usr/sbin/named[1530]:134 udp: 192.168.122.1:53 0.0.0.0:0 00000000 00000000 /usr/sbin/named[1530]:96 tcp: 127.0.0.1:953 0.0.0.0:0 00000000 00000000 /usr/sbin/dhclient[1647]:7 udp: 0.0.0.0:68 0.0.0.0:0 00000000 00000000 [root@jvdspc:~ [2947] 19:38:07 #:8!:8707]{0} ie. there is NO socket bound to port 53 listened to by systemd / systemd-journald, when built with -Ddefault_llmnr=no' , after installing new systemd-248.4-2 RPMs resulting from build & rebooting & doing yum update .
Created attachment 1803114 [details] Patch to test-user-util.c to remove offending assert() .- fix for Bug 1983443
Created attachment 1803119 [details] Patch to systemd.spec that fixes the problem by adding /usr/sbin to $PATH - fix for Bug 1983450
-Ddefault-llmnr=no ^- oops, should have been '-', not '_', above
Created attachment 1803270 [details] bash shell script that lists all listening or connected sockets Better version of script - fix unix: socket listing.
It's most likely a problem with your script to list open ports. Please use a standard tool like lsof and reopen if can reproduce the issue.
Created attachment 1803502 [details] bash shell script that lists all listening or connected sockets Improved sockets.sh: # sockets '.*/named' /usr/sbin/named[1530]:100 udp: 127.0.0.1:53 0.0.0.0:0 00000000 00000000 /usr/sbin/named[1530]:134 udp: 192.168.122.1:53 0.0.0.0:0 00000000 00000000 /usr/sbin/named[1530]:512 udp: 0.0.0.0:53 0.0.0.0:0 00000000 00000000 /usr/sbin/named[1530]:96 tcp: 127.0.0.1:953 0.0.0.0:0 00000000 00000000 # sockets +u '.*/systemd' /usr/lib/systemd/systemd[1]:100 unix: /run/systemd/journal/stdout 00000003 STREAM /usr/lib/systemd/systemd[1]:16 unix: /run/systemd/private 00000002 STREAM /usr/lib/systemd/systemd[1]:17 unix: /run/systemd/userdb/io.systemd.DynamicUser 00000002 STREAM /usr/lib/systemd/systemd[1]:18 unix: /run/systemd/io.system.ManagedOOM 00000002 STREAM /usr/lib/systemd/systemd[1]:32 unix: /run/systemd/notify 00000004 DGRAM /usr/lib/systemd/systemd[1]:36 unix: /run/lvm/lvmpolld.socket 00000002 STREAM /usr/lib/systemd/systemd[1]:38 unix: /run/systemd/coredump 00000002 SEQPKT /usr/lib/systemd/systemd[1]:39 unix: /run/systemd/journal/dev-log 00000023 DGRAM /usr/lib/systemd/systemd[1]:41 unix: /run/systemd/journal/socket 00000011 DGRAM /usr/lib/systemd/systemd[1]:44 unix: /run/udev/control 00000002 SEQPKT /usr/lib/systemd/systemd[1]:59 unix: /run/avahi-daemon/socket 00000002 STREAM /usr/lib/systemd/systemd[1]:60 unix: /run/cups/cups.sock 00000002 STREAM /usr/lib/systemd/systemd[1]:62 unix: /run/dbus/system_bus_socket 00000002 STREAM /usr/lib/systemd/systemd[1]:65 unix: /run/libvirt/libvirt-sock 00000002 STREAM /usr/lib/systemd/systemd[1]:66 unix: /run/libvirt/libvirt-admin-sock 00000002 STREAM /usr/lib/systemd/systemd[1]:67 unix: /run/libvirt/libvirt-sock-ro 00000002 STREAM /usr/lib/systemd/systemd[1]:68 unix: /run/.heim_org.h5l.kcm-socket 00000002 STREAM /usr/lib/systemd/systemd[1]:69 unix: /run/libvirt/virtlockd-sock 00000002 STREAM /usr/lib/systemd/systemd[1]:70 unix: /run/libvirt/virtlogd-sock 00000002 STREAM /usr/lib/systemd/systemd[1]:99 unix: /run/libvirt/virtlogd-admin-sock 00000002 STREAM /usr/lib/systemd/systemd[2202]:12 unix: /run/user/666/bus 00000002 STREAM /usr/lib/systemd/systemd[2202]:16 unix: /run/user/666/systemd/notify 00000002 DGRAM /usr/lib/systemd/systemd[2202]:19 unix: /run/user/666/systemd/private 00000002 STREAM /usr/lib/systemd/systemd[2202]:26 unix: /run/user/666/pulse/native 00000002 STREAM /usr/lib/systemd/systemd[2202]:27 unix: /run/user/666/pipewire-0 00000002 STREAM
This bug has nothing to do with how I'm listing ports. When systemd is built with meson build option '-Ddefault-llmnr=resolve' , as it is in default .spec file, then, when systemd-resoled service is disabled, as it is on my machine since I run BIND named , then systemd listons on '0.0.0.0:53', which breaks my named setup , and it is impossible to obtain the 'sockets' listings in Comment #17 - named will not be able to listen on '0.0.0.0:53', because systemd and systemd-journald listen to it. This bug could be renamed 'please allow users to install and run the BIND named daemon (or any port 53 listener other than systemd-resolved)' .
ie. the listing in Comment #17, if on a system where systemd is built with '-Ddefault-llmnr=resolve', instead of '-Ddefault-llmnr=no', as my systemd now is, would show that both systemd and systemd-journald listen to 0.0.0.0 port 53 , when systemd-resolved service is disabled. This precludes correct use of named or any other daemon that listens to port 53. Please give users some option to disable systemd listening on port 53 during runtime, not compile time.
ie. please allow users not to run systemd-resolved and to run named instead.
It has to be a valid setup, to create /etc/dhcp/dhclient-${IFNAME}.conf files that contain supersede domain-name-servers 127.0.0.1; and to specify in /etc/NetworkManager/NetworkManager.conf: [main] dhcp=dhclient I also have nmcli options for each connection that specify the resolv.conf options and 'domain' setting, so that an identical /etc/resolv.conf is always produced - NetworkManager I think does notice that its new resolv.conf is identical to its old one and my /etc/resolv.conf never changes, so I have no need for systemd-resolved. Until this bug is fixed, I have to produce a build of systemd with the '-Ddefault-llmnr=no' option to enable me to disable systemd-resolved and just run named. In order to build systemd from SRPM, bug 1983443 and bug 1983450 must be fixed.
I did used to use an /sbin/dhclient-script that created an /etc/forwarders.conf file that was included by my /etc/named.conf , but this proved to be unnecessary & overcomplicated since I always know the nameservers in advance . Now, if I connect to a new interface for which there is no /etc/dhcp/dhclient-${IFNAME}.conf file , then the new nameservers do get added to /etc/resolv.conf so I can read them and add them manually to my forwarders list and create an /etc/dhcp/dhclient-${IFNAME}.conf file which does 'supersede domain-name-server 127.0.0.1' if necessary. I've been meaning to resurrect my contrib package to named that let NetworkManager send the new forwarders list to named over DBUS, but really that is overkill for my needs and I haven't got round to it yet.
ie. please either make systemd, when built with '-Ddefault-llmnr=resolve' , detect that systemd-resolved service has been disabled, and if so, do NOT bind any socket to port 53, OR allow all '-Ddefault-llmnr=resolve' processing to be disabled with some new systemd '--no-dns-listen' option .
It should be 100% reproducible, that downloading and running Attachment #1803502 [details] , on a system where systemd is built with -Ddefault-llmnr=resolve (FC34 spec default), and where systemd-resolved service has been stopped and disabled, will produce this output: $ . ./sockets.sh $ sockets '.*/systemd' '.*/systemd-journald' /usr/lib/systemd/[1]:? udp: 0.0.0.0:53 0.0.0.0:0 00000000 00000000 /usr/lib/systemd/systemd-journald[?]:? udp: 0.0.0.0:53 0.0.0.0:0 00000000 00000000 You can use 'lsof' or 'netstat -nautp' or whatever other tool you wish to verify this. This is very bad for BIND named users or users who want to disable systemd-resolved !
If you want to disable the llmnr listener in resolved, create '/etc/systemd/resolved.conf.d/llmnr.conf' with the following contents: ''' [Resolve] LLMNR=no ''' and restart systemd-resolved.service.
Thanks for the info ! But there is no 'man llmnr.conf' . Where is it documented ? I will test if that stops systemd listening on port 0.0.0.0:53 . And why must I run a service (systemd.resolved) that I have no use for ? I have got the number of running daemons on my system down to only those services that I actually need. I do not actually need to run systemd-resolved . Why am I forced to do so / prevented from running named by systemd ? Please answer: Are you saying that if I have the suggested llmnr.conf installed, and I do NOT run systemd.resolved , then systemd / systemd-journald will NOT bind to 0.0.0.0:53 ? Because if not, that would still not fix this bug. I will test this later, when next version of systemd comes out and this bug has not been marked fixed, and I will have to decide whether to rebuild with -Ddefault-llmnr=no after trying the new systemd.
Aha, I just discovered: https://docs.microsoft.com/en-us/previous-versions//bb878128(v=technet.10)?redirectedfrom=MSDN At least now I know what 'LLMNR' is , and that I definitely do not want to use it.
(In reply to Jason Vas Dias from comment #26) > Thanks for the info ! > But there is no 'man llmnr.conf' . Where is it documented ? resolved.conf(5) > And why must I run a service (systemd.resolved) that I have no use for ? You don't. I wrote about the option because you wrote about compiling systemd with '-Ddefault-llmnr=no' and asked about the runtime setting that does this. Setting LLMNR=no is the configuration setting to do this. (The setting can also be done at runtime over dbus, fwiw.) If you don't want systemd-resolved running, then disable the service. You don't need to have it running. https://fedoraproject.org/wiki/Changes/systemd-resolved#Upgrade.2Fcompatibility_impact describes various features and how to opt out of any or all of them. > I have got the number of running daemons on my system down to > only those services that I actually need. > I do not actually need to run systemd-resolved . > Why am I forced to do so / prevented from running named by systemd ? You are not. Please stop reopening bugs that have been closed. This is not a support forum. If the maintainer determines that there is no bug, please accept that unless you really have some new info.
OK, thanks, but please consider adding some documentation for this llmnr.conf file somewhere - I can't find any : # { man systemd; man systemd-resolved; } | egrep -w llmnr.conf # find /usr/share/man/man* -name '*llmnr*' # ... or where is it?