Bug 2061944 - Version dnsmasq-2.86-5 breaks properly processing /etc/NetworkManager/dnsmasq.d/
Summary: Version dnsmasq-2.86-5 breaks properly processing /etc/NetworkManager/dnsmasq.d/
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dnsmasq
Version: 35
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Petr Menšík
QA Contact: Fedora Extras Quality Assurance
URL: https://lists.thekelleys.org.uk/piper...
Whiteboard:
: 2064427 (view as bug list)
Depends On:
Blocks: 2148301
TreeView+ depends on / blocked
 
Reported: 2022-03-08 18:36 UTC by Jasper Siero
Modified: 2022-11-24 23:05 UTC (History)
10 users (show)

Fixed In Version: dnsmasq-2.86-9.fc37 dnsmasq-2.86-9.fc36 dnsmasq-2.86-6.fc35
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2148301 (view as bug list)
Environment:
Last Closed: 2022-06-22 00:47:49 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
candidate patch (2.50 KB, patch)
2022-06-17 10:07 UTC, Petr Menšík
no flags Details | Diff

Description Jasper Siero 2022-03-08 18:36:42 UTC
Description of problem:
Version dnsmasq-2.86-5 breaks properly processing /etc/NetworkManager/dnsmasq.d/

Version-Release number of selected component (if applicable):
dnsmasq-2.86-5

How reproducible:
100%

Steps to Reproduce:
1. dnf update
2. no static dns as before
3.

Actual results:
DNS resolving specific subdomains fails

Expected results:
DNS resolving specific subdomains succeeds

Additional info (obfuscated):
our static setup:
cat /etc/NetworkManager/dnsmasq.d/static.conf 
listen-address=127.0.0.35
server=/local.example.com/10.120.100.51
server=/local.example.com/10.120.100.52
server=/3.2.1.in-addr.arpa/192.168.0.1
server=/120.10.in-addr.arpa/10.120.100.51
server=/120.10.in-addr.arpa/10.120.100.52
addn-hosts=/usr/share/laptop/dnsmasq.hosts



journal logs:
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: DBus support enabled: connected to system bus
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 10.120.100.51#53 for domain local.example.com
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 10.120.100.52#53 for domain local.example.com
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 192.168.0.1#53 for domain 3.2.1.in-addr.arpa
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 10.120.100.51#53 for domain 120.10.in-addr.arpa
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 10.120.100.52#53 for domain 120.10.in-addr.arpa
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: read /usr/share/laptop/dnsmasq.hosts - 2 addresses
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: setting upstream servers from DBus
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 10.120.100.51#53 for domain local.example.com
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 10.120.100.52#53 for domain local.example.com
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 192.168.0.1#53 for domain 3.2.1.in-addr.arpa
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 10.120.100.51#53 for domain 120.10.in-addr.arpa
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 10.120.100.52#53 for domain 120.10.in-addr.arpa
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 192.168.2.1#53(via wlp2s0)
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 192.168.2.1#53 for domain 2.168.192.in-addr.arpa
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 192.168.2.1#53 for domain 123.2.168.192.in-addr.arpa
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: read /usr/share/laptop/dnsmasq.hosts - 2 addresses
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: setting upstream servers from DBus
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 192.168.2.1#53(via wlp2s0)
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 192.168.2.1#53 for domain 2.168.192.in-addr.arpa
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: using nameserver 192.168.2.1#53 for domain 123.2.168.192.in-addr.arpa
Mar 08 16:30:11 localhost.localdomain dnsmasq[1629]: read /usr/share/tb-laptop/dnsmasq.hosts - 2 addresses

Comment 1 Rolf Fokkens 2022-03-08 20:25:47 UTC
We built 2.87test8 as an RPM (dnsmasq-2.87-0.test8) and tried this: same issue. So the issue is in 2.87test and has been backported to the 2.86 package.

Comment 2 Andrew Watts 2022-05-27 19:43:22 UTC
This issue still appears to be present in Fedora 36, with dnsmasq-2.86-6.fc36.x86_64.

I use dnsmasq to refer the domains rn-default and orgs-default to refer to dnsmasq instances running inside Docker networks, so that I can access servers within the Docker networks by name.

My config in /etc/NetworkManager/dnsmasq.d/*.conf:

server=/rn-default/172.21.2.2
server=/orgs-default/172.22.2.2

As you can see from the logs below following a systemctl restart NetworkManager, the servers I've specified are present when the config is initially loaded, and even after the first batch of new servers come in from DBus, when my wired connection (enp0s20f0u4u3, through a USB-C docking station) gets activated. But the second time it gets an update from DBus, when my wireless connection (wlp4s0) is activated, the servers from the config file are no longer present. Then when IPv6 gets set up, the servers are still missing.

May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: started, version 2.86 cachesize 400
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: compile time options: IPv6 GNU-getopt DBus no-UBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth cryptohash DNSSEC loop-detect inotify dumpfile
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: chown of PID file /run/NetworkManager/dnsmasq.pid failed: Operation not permitted
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: DBus support enabled: connected to system bus
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: using nameserver 172.21.2.2#53 for domain rn-default
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: using nameserver 172.22.2.2#53 for domain orgs-default
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: cleared cache
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: setting upstream servers from DBus
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: using nameserver 172.21.2.2#53 for domain rn-default
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: using nameserver 172.22.2.2#53 for domain orgs-default
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53(via enp0s20f0u4u3)
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain example.net
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain 1.168.192.in-addr.arpa
May 27 15:24:52 ironic-mullet.example.com dnsmasq[134667]: cleared cache
May 27 15:24:55 ironic-mullet.example.com dnsmasq[134667]: setting upstream servers from DBus
May 27 15:24:55 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53(via enp0s20f0u4u3)
May 27 15:24:55 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain example.net
May 27 15:24:55 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain 1.168.192.in-addr.arpa
May 27 15:24:55 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53(via wlp4s0)
May 27 15:24:55 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain example.net
May 27 15:24:55 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain 1.168.192.in-addr.arpa
May 27 15:24:55 ironic-mullet.example.com dnsmasq[134667]: cleared cache
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: setting upstream servers from DBus
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53(via enp0s20f0u4u3)
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain example.net
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain 1.168.192.in-addr.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver fe80::hhhh:hhhh:hhhh:hhhh%wlp4s0#53
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver fe80::hhhh:hhhh:hhhh:hhhh%wlp4s0#53 for domain 5.7.5.a.7.0.6.6.0.8.0.6.3.0.6.2.ip6.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver fe80::hhhh:hhhh:hhhh:hhhh%wlp4s0#53 for domain 0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53(via wlp4s0)
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain example.net
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain 1.168.192.in-addr.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: cleared cache
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: setting upstream servers from DBus
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53(via enp0s20f0u4u3)
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain example.net
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain 1.168.192.in-addr.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver gggg:gggg:gggg:gggg::1#53(via wlp4s0)
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver gggg:gggg:gggg:gggg::1#53 for domain f.d.a.a.b.0.4.2.0.0.0.0.0.0.0.0.5.7.5.a.7.0.6.6.0.8.0.6.3.0.6.2.ip6.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver gggg:gggg:gggg:gggg::1#53 for domain 5.7.5.a.7.0.6.6.0.8.0.6.3.0.6.2.ip6.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver gggg:gggg:gggg:gggg::1#53 for domain 0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver fe80::hhhh:hhhh:hhhh:hhhh%wlp4s0#53
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver fe80::hhhh:hhhh:hhhh:hhhh%wlp4s0#53 for domain f.d.a.a.b.0.4.2.0.0.0.0.0.0.0.0.5.7.5.a.7.0.6.6.0.8.0.6.3.0.6.2.ip6.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver fe80::hhhh:hhhh:hhhh:hhhh%wlp4s0#53 for domain 5.7.5.a.7.0.6.6.0.8.0.6.3.0.6.2.ip6.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver fe80::hhhh:hhhh:hhhh:hhhh%wlp4s0#53 for domain 0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53(via wlp4s0)
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain example.net
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: using nameserver 192.168.1.1#53 for domain 1.168.192.in-addr.arpa
May 27 15:24:57 ironic-mullet.example.com dnsmasq[134667]: cleared cache

(some of the domain names and IP addresses have been changed here to anonymize things)

Comment 3 Petr Menšík 2022-06-04 20:44:50 UTC
So if I understand correctly, dnsmasq is integrated with Network Manager. Dbus configuration changes not only configuration previously set by DBus interface, but also static configuration set in file. Is that correct?

From comment #2 it seems static configuration survives first configuration via DBus, but disappears on the second. Or when two interfaces were used.

Comment 4 Jasper Siero 2022-06-07 08:47:42 UTC
The only solution for us is to remove Patch30 and rebuild the package for ourself but this is ofcourse a temporarily solution. Every new release of the package breaks our system until we fix it again, recently with the upgrade to Fedora 36. But we are getting the hang of it.

Comment 5 Andrew Watts 2022-06-08 19:36:28 UTC
@pem That appears to be correct. It's as if the static configuration is downgraded to a temporary configuration by the first DBus reconfiguration, and then the second DBus event replaces it. I haven't tried it with a single interface being reset multiple times, only with multiple interfaces / IPv6, but I would expect the same behavior in that case, too.

Comment 6 Petr Menšík 2022-06-17 09:59:32 UTC
I think I have found a reason for this. It was well hidden. The error is inside add_update_server function. There is a loop searching marked server with matching domain. But it does not update up pointer correctly. That then replaces the record following the marked server to beginning, which creates a memleak and forgets static servers defined by configuration file (which has flags == 0).

Solution is to update up pointer at the end of loop. Even latest master branch is affected, I do not think this were reported to upstream yet.

Comment 7 Petr Menšík 2022-06-17 10:03:20 UTC
*** Bug 2064427 has been marked as a duplicate of this bug. ***

Comment 8 Petr Menšík 2022-06-17 10:07:47 UTC
Created attachment 1890877 [details]
candidate patch

Comment 9 Renaud Métrich 2022-06-17 10:17:23 UTC
+1 because it's super annoying :)

Comment 10 Petr Menšík 2022-06-17 10:43:55 UTC
My test build above test8 tag works. I will prepare rawhide update, but this works to me now:

https://copr.fedorainfracloud.org/coprs/pemensik/dnsmasq/package/dnsmasq/

Comment 11 Petr Menšík 2022-06-17 10:45:07 UTC
Reported upstream with candidate patch:
https://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2022q2/016442.html

Comment 12 Petr Menšík 2022-06-17 11:08:08 UTC
Pushed fix to rawhide branch, build is running. Because it has no good workaround, not waiting for upstream response. Will replace the patch eventually.

Comment 13 Fedora Update System 2022-06-17 11:37:58 UTC
FEDORA-2022-12673da823 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-12673da823

Comment 14 Fedora Update System 2022-06-17 11:38:42 UTC
FEDORA-2022-73122722e6 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-73122722e6

Comment 15 Fedora Update System 2022-06-18 01:24:55 UTC
FEDORA-2022-73122722e6 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-73122722e6`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-73122722e6

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 16 Fedora Update System 2022-06-18 01:40:33 UTC
FEDORA-2022-12673da823 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-12673da823`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-12673da823

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 17 Andrew Watts 2022-06-19 00:40:48 UTC
Installing dnsmasq-2.86-9.fc36.x86_64 package from the updates-testing repository fixed the issue for me.

Comment 18 Renaud Métrich 2022-06-20 06:32:32 UTC
Seems to work for me as well.

Comment 19 Jasper Siero 2022-06-21 08:05:52 UTC
The new package dnsmasq-2.86-9.fc36.x86_64 in testing works for me and fixes the problem

Comment 20 Fedora Update System 2022-06-22 00:47:49 UTC
FEDORA-2022-12673da823 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 21 Fedora Update System 2022-07-03 01:19:45 UTC
FEDORA-2022-73122722e6 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 22 Andrew Watts 2022-11-22 03:18:10 UTC
This problem appears to have returned with Fedora 37 and dnsmasq-2.87-1.fc37.x86_64. It looks like the upstream patch fixing this issue, which I believe is this one: 

https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=930428fb970f4991e5c2933fd5a5d2504c18a551

is not in 2.87 and won't land until 2.88.


Note You need to log in before you can comment on or make changes to this bug.