Version-Release number of selected component: dnsmasq-2.86-1.fc34 Additional info: reporter: libreport-2.15.2 backtrace_rating: 4 cgroup: 0::/system.slice/libvirtd.service cmdline: /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper crash_function: lookup_domain environ: VIR_BRIDGE_NAME=virbr0 executable: /usr/sbin/dnsmasq journald_cursor: s=2fd37d9ede6548e7be192c534893de2e;i=3f0e4;b=a6f0ad74ba5643d39e274569878fca03;m=8bcb3c4c5;t=5cd4c1cd70b4b;x=bff9fa8c415a6046 kernel: 5.13.19-200.fc34.x86_64 rootdir: / runlevel: N 5 type: CCpp uid: 978 Truncated backtrace: Thread no. 1 (4 frames) #0 lookup_domain at /usr/src/debug/dnsmasq-2.86-1.fc34.x86_64/src/domain-match.c:234 #1 forward_query at /usr/src/debug/dnsmasq-2.86-1.fc34.x86_64/src/forward.c:258 #2 receive_query at /usr/src/debug/dnsmasq-2.86-1.fc34.x86_64/src/forward.c:1640 #3 check_dns_listeners at /usr/src/debug/dnsmasq-2.86-1.fc34.x86_64/src/dnsmasq.c:1817
Created attachment 1828366 [details] File: backtrace
Created attachment 1828367 [details] File: core_backtrace
Created attachment 1828368 [details] File: cpuinfo
Created attachment 1828369 [details] File: dso_list
Created attachment 1828370 [details] File: exploitable
Created attachment 1828371 [details] File: limits
Created attachment 1828372 [details] File: maps
Created attachment 1828373 [details] File: mountinfo
Created attachment 1828374 [details] File: open_fds
Created attachment 1828375 [details] File: proc_pid_status
Created attachment 1828376 [details] File: var_log_messages
I think this issue might be already fixed by bug #2006367 fix in dnsmasq-2.86-2 build. There were some memory corruption operation in some cases, which should be fixed already. But there is small chance this is another issue not yet fixed.
Hello. Are you able to reproduce this issue, Mustafa, more reliably? There are multiple reports of these issues. dnsmasq has been updated recently fixing similar error, which might have fixed also this one. Do you know how to trigger it? Could you please testing update [1], whether it helps to solve this issue or it remains the same? Could you please attach configuration /var/lib/libvirt/dnsmasq/default.conf? It would be very helpful if core dump could be attached. It is clear it crashed, because qdomain and qlen got very wrong values. Either it happened because dnsmasq_daemon->serverarray[0]->domain and dnsmasq_daemon->serverarray[0]->domain_len got corrupted. That might be fixed in update [1]. Another possibility is wrong algorithm when updating crop_query variable. It may rely on some implicit order, which may not be correct in all corner cases. qlen should have been between 0 and 27. domain-qdomain pointer gives 51515, which suggests negative substraction wrapped around 16 bit integer variable. I can only guess, crop_query variable is unfortunately optimized out. It would be very helpful, if you could attach core dump to this bug too or send it to me privately. Command "coredumpctl info dnsmasq" would point you to Storage with saved dump file. Alternatively, can you please at least list those data yourself? First ensure you have dnsmasq-debuginfo installed: dnf debuginfo-install dnsmasq dnf install gdb coredumpctl gdb dnsmasq Use this gdb snippet: set $i = 0 while ($i < dnsmasq_daemon->serverarraysz) p dnsmasq_daemon->serverarray[$i]->domain p dnsmasq_daemon->serverarray[$i]->domain_len p /x dnsmasq_daemon->serverarray[$i]->flags set $i = $i +1 end And please post what its prints. Query on which it failed can be issued by: dig @192.168.122.1 -x 192.168.122.18 But I expect some usual traffic have to be processed before it crashed dnsmasq. [1] https://bodhi.fedoraproject.org/updates/FEDORA-2021-03b9f525f0
(In reply to Petr Menšík from comment #13) > Hello. Are you able to reproduce this issue, Mustafa, more reliably? There > are multiple reports of these issues. dnsmasq has been updated recently > fixing similar error, which might have fixed also this one. Do you know how > to trigger it? Could you please testing update [1], whether it helps to > solve this issue or it remains the same? Hello, I can't reproduce this because I don't know the trigger, I saw it in ABRT and reported it. > > Could you please attach configuration /var/lib/libvirt/dnsmasq/default.conf? > Sure I will > It would be very helpful if core dump could be attached. It is clear it > crashed, because qdomain and qlen got very wrong values. Either it happened > because dnsmasq_daemon->serverarray[0]->domain and > dnsmasq_daemon->serverarray[0]->domain_len got corrupted. That might be > fixed in update [1]. > > Another possibility is wrong algorithm when updating crop_query variable. It > may rely on some implicit order, which may not be correct in all corner > cases. qlen should have been between 0 and 27. domain-qdomain pointer gives > 51515, which suggests negative substraction wrapped around 16 bit integer > variable. I can only guess, crop_query variable is unfortunately optimized > out. > > It would be very helpful, if you could attach core dump to this bug too or > send it to me privately. Command "coredumpctl info dnsmasq" would point you > to Storage with saved dump file. Sorry, the file is not there: coredumpctl info dnsmasq PID: 33788 (dnsmasq) UID: 978 (dnsmasq) GID: 978 (dnsmasq) Signal: 11 (SEGV) Timestamp: Fri 2021-10-01 18:21:08 +03 (3 days ago) Command Line: /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper Executable: /usr/sbin/dnsmasq Control Group: /system.slice/libvirtd.service Unit: libvirtd.service Slice: system.slice Boot ID: a6f0ad74ba5643d39e274569878fca03 Machine ID: bc109f649c554f339f46d5176c8eddc5 Hostname: localhost.localdomain Storage: /var/lib/systemd/coredump/core.dnsmasq.978.a6f0ad74ba5643d39e274569878fca03.33788.1633101668000000.zst (missing) Message: Process 33788 (dnsmasq) of user 978 dumped core. Stack trace of thread 33788: #0 0x0000557a9d7c7256 lookup_domain (dnsmasq + 0x53256) #1 0x0000557a9d797a3a forward_query.lto_priv.0 (dnsmasq + 0x23a3a) #2 0x0000557a9d79c5d0 check_dns_listeners (dnsmasq + 0x285d0) #3 0x0000557a9d780b00 main (dnsmasq + 0xcb00) #4 0x00007f38bce2cb75 __libc_start_main (libc.so.6 + 0x27b75) #5 0x0000557a9d78148e _start (dnsmasq + 0xd48e) > > Alternatively, can you please at least list those data yourself? First > ensure you have dnsmasq-debuginfo installed: > dnf debuginfo-install dnsmasq > dnf install gdb > coredumpctl gdb dnsmasq > > Use this gdb snippet: > > set $i = 0 > while ($i < dnsmasq_daemon->serverarraysz) > p dnsmasq_daemon->serverarray[$i]->domain > p dnsmasq_daemon->serverarray[$i]->domain_len > p /x dnsmasq_daemon->serverarray[$i]->flags > set $i = $i +1 > end > > And please post what its prints. > > Query on which it failed can be issued by: > dig @192.168.122.1 -x 192.168.122.18 > > But I expect some usual traffic have to be processed before it crashed > dnsmasq. > > [1] https://bodhi.fedoraproject.org/updates/FEDORA-2021-03b9f525f0 I think you can close this report assuming the update fixed it, and if this happens again, I'll try to get the core dump and open it again. Regards Mustafa
Created attachment 1829341 [details] The file /var/lib/libvirt/dnsmasq/default.conf
*** Bug 2012151 has been marked as a duplicate of this bug. ***
Okay, further reports confirms unfortunately fix of bug #2006367 did not fix this issue, since they reported it also on a fixed version. Would be any of new reporters able to provide coredump file of a crash? Should be listed by: coredumpctl list dnsmasq
Alternative command: abrt ls -x /usr/sbin/dnsmasq
Oh, this seems to be reported already and has different fix than I searched for. https://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2021q4/015849.html Seems like following commit should fix it: https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=d290630d31f4517ab26392d00753d1397f9a4114
According to above commit, the problem is caused by changed /etc/resolv.conf during runtime. dnsmasq_daemon->serverarray pointers get freed and some would point to invalid memory. Then searching for correct server may read memory outside of allocated space and crash dnsmasq. That would be fixed by above commit. I think some places require refresh of serverarray also, because changing servers via d-bus does not call serverarray rebuilding even on master branch. Proposed additional change to upstream [1], waiting for confirmation. 1. https://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2021q4/015852.html
*** Bug 2006367 has been marked as a duplicate of this bug. ***
Because upstream is not responding for some time, I would build just my proposals. They should improve reliability.
FEDORA-2021-bfb01154ce has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-bfb01154ce
FEDORA-2021-83554ef1c7 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-83554ef1c7
FEDORA-2021-bfb01154ce has been pushed to the Fedora 35 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-bfb01154ce` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-bfb01154ce See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2021-83554ef1c7 has been pushed to the Fedora 34 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-83554ef1c7` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-83554ef1c7 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2021-bfb01154ce has been pushed to the Fedora 35 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2021-83554ef1c7 has been pushed to the Fedora 34 stable repository. If problem still persists, please make note of it in this bug report.