Bug 1562594

Summary: Unbound 1.7.0 crashes with a buffer overflow
Product: [Fedora] Fedora Reporter: Tom Hughes <tom>
Component: unboundAssignee: Petr Menšík <pemensik>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 28CC: bill, dominik, fedora, johnh, pemensik, pj.pandit, pwouters, theo148, thozza
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://nlnetlabs.nl/bugs-script/show_bug.cgi?id=4189
Whiteboard:
Fixed In Version: unbound-1.7.3-10.fc28 unbound-1.7.3-10.fc27 unbound-1.8.1-1.fc29 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-11-03 00:00:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
stack trace from unbound-1.7.3-6.fc28.x86_64 none

Description Tom Hughes 2018-04-01 09:04:43 UTC 
Description of problem:

Unbound 1.7.0 from updates-testing crashes on a frequent basis with a buffer overflow error being logged:

Apr 01 09:46:56 arden.compton.nu unbound[15415]: [15415:0] notice: sendto failed: Invalid argument
Apr 01 09:46:56 arden.compton.nu unbound[15415]: [15415:0] notice: remote address is (inet_ntop error) port 45798
Apr 01 09:46:56 arden.compton.nu unbound[15415]: *** buffer overflow detected ***: /usr/sbin/unbound terminated
Apr 01 09:46:56 arden.compton.nu unbound[15415]: [15415:0] notice: sendto failed: Invalid argument
Apr 01 09:46:56 arden.compton.nu unbound[15415]: [15415:0] notice: remote address is (inet_ntop error) port 45798
Apr 01 09:46:56 arden.compton.nu unbound[15415]: [15415:0] notice: sendto failed: Invalid argument
Apr 01 09:46:56 arden.compton.nu unbound[15415]: [15415:0] notice: remote address is (inet_ntop error) port 45798
Apr 01 09:46:56 arden.compton.nu unbound[15415]: [15415:0] notice: sendto failed: Invalid argument
Apr 01 09:46:56 arden.compton.nu unbound[15415]: [15415:0] notice: remote address is (inet_ntop error) port 45798
Apr 01 09:46:56 arden.compton.nu unbound[15415]: [15415:0] notice: sendto failed: Invalid argument
Apr 01 09:46:56 arden.compton.nu unbound[15415]: [15415:0] notice: remote address is (inet_ntop error) port 45798
Apr 01 09:46:56 arden.compton.nu systemd[1]: unbound.service: Main process exited, code=killed, status=6/ABRT
Apr 01 09:46:56 arden.compton.nu systemd[1]: unbound.service: Unit entered failed state.
Apr 01 09:46:56 arden.compton.nu systemd[1]: unbound.service: Failed with result 'signal'.


Version-Release number of selected component (if applicable):

unbound-1.7.0-2.fc27.x86_64

How reproducible:

Run it. Wait five or ten minutes. Find DNS has stopped working.
Comment 1 William H. Haller 2018-04-11 16:09:04 UTC 
Also failing on fc26
Apr 11 08:48:56 ae-dirsrv unbound[1575]: *** buffer overflow detected ***: /usr/sbin/unbound terminated
Apr 11 08:48:56 ae-dirsrv audit[1575]: ANOM_ABEND auid=4294967295 uid=996 gid=994 ses=4294967295 subj=system_u:system_r:named_t:s0 pid=1575 comm="unbound" exe="/usr/sbin/unbound" sig=6 res=1
Apr 11 08:48:56 ae-dirsrv unbound[1575]: ======= Backtrace: =========
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /lib64/libc.so.6(+0x7cbac)[0x7f9133175bac]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /lib64/libc.so.6(__fortify_fail+0x37)[0x7f913321ca37]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /lib64/libc.so.6(+0x1217b0)[0x7f913321a7b0]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x83842)[0x563d2591f842]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x842b0)[0x563d259202b0]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(auth_xfer_probe_lookup_callback+0xa4)[0x563d25920464]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x46bc2)[0x563d258e2bc2]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47e1c)[0x563d258e3e1c]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47fdd)[0x563d258e3fdd]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x4889f)[0x563d258e489f]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x84105)[0x563d25920105]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x8427d)[0x563d2592027d]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(auth_xfer_probe_lookup_callback+0xa4)[0x563d25920464]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x46bc2)[0x563d258e2bc2]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47e1c)[0x563d258e3e1c]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47fdd)[0x563d258e3fdd]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x4889f)[0x563d258e489f]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x84105)[0x563d25920105]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x8427d)[0x563d2592027d]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(auth_xfer_probe_lookup_callback+0xa4)[0x563d25920464]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x46bc2)[0x563d258e2bc2]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47e1c)[0x563d258e3e1c]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47fdd)[0x563d258e3fdd]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x4889f)[0x563d258e489f]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x84105)[0x563d25920105]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x8427d)[0x563d2592027d]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(auth_xfer_probe_lookup_callback+0xa4)[0x563d25920464]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x46bc2)[0x563d258e2bc2]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47e1c)[0x563d258e3e1c]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47fdd)[0x563d258e3fdd]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x4889f)[0x563d258e489f]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x84105)[0x563d25920105]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x8427d)[0x563d2592027d]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(auth_xfer_probe_lookup_callback+0xa4)[0x563d25920464]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x46bc2)[0x563d258e2bc2]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47e1c)[0x563d258e3e1c]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47fdd)[0x563d258e3fdd]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x4889f)[0x563d258e489f]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x84105)[0x563d25920105]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x8427d)[0x563d2592027d]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(auth_xfer_probe_lookup_callback+0xa4)[0x563d25920464]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x46bc2)[0x563d258e2bc2]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47e1c)[0x563d258e3e1c]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x47fdd)[0x563d258e3fdd]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(worker_handle_service_reply+0x78)[0x563d258bf1b8]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0xc180a)[0x563d2595d80a]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(serviced_udp_callback+0xaf)[0x563d2595e46f]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(outnet_udp_cb+0x295)[0x563d2595c905]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(comm_point_udp_callback+0x89)[0x563d25955879]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /lib64/libevent-2.0.so.5(event_base_loop+0x7a9)[0x7f91340eb3f9]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0xb95ec)[0x563d259555ec]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x1be15)[0x563d258b7e15]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x178cc)[0x563d258b38cc]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /lib64/libc.so.6(__libc_start_main+0xea)[0x7f913311988a]
Apr 11 08:48:56 ae-dirsrv unbound[1575]: /usr/sbin/unbound(+0x1856a)[0x563d258b456a]
Comment 2 Paul Wouters 2018-04-12 14:26:52 UTC 
Please try 1.7.0-4
Comment 3 Tom Hughes 2018-04-12 14:49:16 UTC 
It's different, but it still crashes with a segmentation fault. In fact it seems to be even more reliable now:

Apr 12 15:46:23 arden.compton.nu unbound[5998]: [5998:0] info: start of service (unbound 1.7.0).
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure d.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure e.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure f.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure g.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure h.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure i.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure b.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure c.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure j.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure a.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure k.root-servers.net. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure k.root-servers.net. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure g.root-servers.net. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure f.root-servers.net. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure e.root-servers.net. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure c.root-servers.net. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure b.root-servers.net. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:3] info: validation failure wpad.compton.nu. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:3] info: validation failure wpad.compton.nu. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure wpad. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:0] info: validation failure wpad. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:2] info: validation failure detectportal.firefox.com. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:2] info: validation failure detectportal.firefox.com. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:2] info: validation failure location.services.mozilla.com. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:2] info: validation failure location.services.mozilla.com. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:2] info: validation failure webcache.compton.nu. AAAA IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:2] info: validation failure webcache.compton.nu. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:2] info: validation failure webcache.compton.nu.compton.nu. A IN
Apr 12 15:46:24 arden.compton.nu unbound[5998]: [5998:2] info: validation failure webcache.compton.nu.compton.nu. AAAA IN
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: service stopped (unbound 1.7.0).
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: server stats for thread 0: 12 queries, 10 answers from cache, 2 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: server stats for thread 0: requestlist max 1 avg 0.5 exceeded 0 jostled 0
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: average recursion processing time 0.016154 sec
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: histogram of recursion processing times
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: [25%]=0 median[50%]=0 [75%]=0
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: lower(secs) upper(secs) recursions
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info:    0.008192    0.016384 2
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: server stats for thread 1: 6 queries, 6 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: server stats for thread 1: requestlist max 0 avg 0 exceeded 0 jostled 0
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: server stats for thread 2: 8 queries, 0 answers from cache, 8 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: server stats for thread 2: requestlist max 1 avg 0.5 exceeded 0 jostled 0
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: average recursion processing time 0.016259 sec
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: histogram of recursion processing times
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: [25%]=0.00273067 median[50%]=0.004096 [75%]=0.032768
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: lower(secs) upper(secs) recursions
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info:    0.001024    0.002048 1
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info:    0.002048    0.004096 3
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info:    0.008192    0.016384 1
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info:    0.016384    0.032768 1
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info:    0.032768    0.065536 2
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: server stats for thread 3: 2 queries, 0 answers from cache, 2 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: server stats for thread 3: requestlist max 1 avg 0.5 exceeded 0 jostled 0
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: average recursion processing time 0.014153 sec
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: histogram of recursion processing times
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: [25%]=0 median[50%]=0 [75%]=0
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: lower(secs) upper(secs) recursions
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info:    0.008192    0.016384 2
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] notice: Restart of unbound 1.7.0.
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] notice: init module 0: ipsecmod
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] notice: init module 1: validator
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] notice: init module 2: iterator
Apr 12 15:46:25 arden.compton.nu unbound[5998]: [5998:0] info: start of service (unbound 1.7.0).
Apr 12 15:46:26 arden.compton.nu unbound[5998]: [5998:0] info: service stopped (unbound 1.7.0).
Apr 12 15:46:26 arden.compton.nu unbound[5998]: [5998:0] info: server stats for thread 0: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Apr 12 15:46:26 arden.compton.nu unbound[5998]: [5998:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
Apr 12 15:46:26 arden.compton.nu systemd[1]: unbound.service: Main process exited, code=killed, status=11/SEGV
Apr 12 15:46:26 arden.compton.nu systemd[1]: unbound.service: Unit entered failed state.
Apr 12 15:46:26 arden.compton.nu systemd[1]: unbound.service: Failed with result 'signal'.
Comment 4 Dominik 'Rathann' Mierzejewski 2018-06-03 21:59:39 UTC 
On F28 I'm getting an ABRT signal (unbound-1.7.0-4.fc28.x86_64):

Jun 03 23:10:17 localhost.localdomain unbound[1156]: [1156:1] info: validation failure 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.2.c.0.1.7.0.0.0.7.4.0.1.0.0.2.ip6.arpa. PTR IN
Jun 03 23:10:24 localhost.localdomain unbound[1156]: [1156:1] info: generate keytag query _ta-4a5c-4f66. NULL IN
Jun 03 23:10:38 localhost.localdomain unbound[1156]: *** buffer overflow detected ***: /usr/sbin/unbound terminated
Jun 03 23:10:38 localhost.localdomain systemd[1]: unbound.service: Main process exited, code=killed, status=6/ABRT
Jun 02 23:10:38 localhost.localdomain systemd[1]: unbound.service: Failed with result 'signal'.

Shall I open a separate bug?
Comment 5 Dominik 'Rathann' Mierzejewski 2018-06-20 12:46:57 UTC 
Actually, someone opened a separate bug 1590641. And I can reproduce with 1.7.2-1, too.
Comment 6 Petr Menšík 2018-07-13 08:05:32 UTC 
Is there coredump available for it? Is it listed by coredumpctl?

Note there was question on it on unbound list [1]. Could you provide more details with verbosity 4? Just use "unbound-control verbosity 4" command before waiting for crash.

It is strange abrt does not catch this error. It would be quite simple to solve with more details. But they are missing.

1. https://www.unbound.net/pipermail/unbound-users/2018-April/005128.html
Comment 7 Tomáš Hozza 2018-07-19 13:44:08 UTC 
Happens to me from time to time with unbound-1.7.3-3.fc28.x86_64
Comment 8 Dominik 'Rathann' Mierzejewski 2018-07-20 13:22:56 UTC 
No, coredumpctl doesn't even have these crashes in its list.
Comment 9 Tomáš Hozza 2018-07-21 08:27:31 UTC 
I was able to get more information:

Thread 1 "unbound" received signal SIGSEGV, Segmentation fault.
0x000055778c66aff9 in xfr_probe_send_probe ()

$ rpm -q unbound
unbound-1.7.3-3.fc28.x86_64

I turned off ipsecmod module, I'm running only with iterator and validator.

Hopefully I will have more information soon, as I'm running always with GDB attached to unbound process.
Comment 10 Dominik 'Rathann' Mierzejewski 2018-08-08 07:41:59 UTC 
unbound-1.7.3-6.fc28.x86_64 has just crashed on me. I wasn't running it under gdb, so no backtrace this time. I'll try doing the same as Tomáš.
Comment 11 Dominik 'Rathann' Mierzejewski 2018-08-28 22:50:48 UTC 
Created attachment 1479375 [details]
stack trace from unbound-1.7.3-6.fc28.x86_64

So, I was finally able to obtain a backtrace. I hope this helps.
Comment 12 Dominik 'Rathann' Mierzejewski 2018-09-05 09:05:50 UTC 
Ping?
Comment 13 Petr Menšík 2018-10-02 18:22:14 UTC 
Hi Dominik,

thanks for some backtrace finally. Backtrace reveals it iterates over and over. It seems to me it is looping until it crashes.

I think candidate upstream commit fixing this issue might be commit [1]. Do you still posses this crash dump? Are you able to reproduce this issue? Or does it occur to you often? I can prepare build with this patch, but I am not sure it really fixes this issue. It is strange it never happened to me.

1. https://github.com/NLnetLabs/unbound/commit/a31d45b13e2e496b2fdb6ce4715481681362472f
Comment 14 Petr Menšík 2018-10-02 18:30:51 UTC 
The answer to your question, whether new rebase (bug #1634771) should fix this issue, I think it would. Can you afford to try build in COPR repository [1]?

1. https://copr.fedorainfracloud.org/coprs/pemensik/unbound/
Comment 15 Petr Menšík 2018-10-02 18:45:58 UTC 
I have started scratch build with included fix [1], it uses source at my fork [2].

1. https://koji.fedoraproject.org/koji/taskinfo?taskID=30015769
2. https://src.fedoraproject.org/fork/pemensik/rpms/unbound/commits/f28
Comment 16 Petr Menšík 2018-10-03 09:41:13 UTC 
Reported upstream as bug https://nlnetlabs.nl/bugs-script/show_bug.cgi?id=4189
Comment 17 Dominik 'Rathann' Mierzejewski 2018-10-03 22:03:46 UTC 
(In reply to Petr Menšík from comment #16)
> Reported upstream as bug
> https://nlnetlabs.nl/bugs-script/show_bug.cgi?id=4189

I think you are mistaken that this doesn't happen on F27. This very bug report started on F27 and at least one reporter said (comment #2) that it happened on F26 as well. I'd appreciate it if you corrected your statement in the upstream bug report.

(In reply to Petr Menšík from comment #15)
> I have started scratch build with included fix [1], it uses source at my
> fork [2].
> 
> 1. https://koji.fedoraproject.org/koji/taskinfo?taskID=30015769
> 2. https://src.fedoraproject.org/fork/pemensik/rpms/unbound/commits/f28

I've installed your scratch build and I'll keep testing it for now. Thank you!
Comment 18 Petr Menšík 2018-10-08 17:43:52 UTC 
*** Bug 1590641 has been marked as a duplicate of this bug. ***
Comment 19 Petr Menšík 2018-10-08 17:47:16 UTC 
(In reply to Dominik 'Rathann' Mierzejewski from comment #17)
> 
> I think you are mistaken that this doesn't happen on F27. This very bug
> report started on F27 and at least one reporter said (comment #2) that it
> happened on F26 as well. I'd appreciate it if you corrected your statement
> in the upstream bug report.
> 
Ok, I take that statement back. It happened today to me on F27. It is somehow 
rare event, but can happen on Fedora 27 as well.

unbound-1.7.3-9.fc27.x86_64

Sep 12 16:37:42 menpad systemd[1]: Started Unbound recursive Domain Name Server.
Sep 12 16:37:42 menpad unbound[4710]: [4710:0] notice: init module 0: ipsecmod
Sep 12 16:37:42 menpad unbound[4710]: [4710:0] notice: init module 1: validator
Sep 12 16:37:42 menpad unbound[4710]: [4710:0] notice: init module 2: iterator
Sep 12 16:37:42 menpad unbound[4710]: [4710:0] info: start of service (unbound 1.7.3).
Sep 12 19:08:06 menpad unbound[4710]: [4710:0] error: can't bind socket: Permission denied for 0.0.0.0
Sep 13 04:19:26 menpad unbound[4710]: [4710:0] info: generate keytag query _ta-4a5c-4f66. NULL IN
Sep 13 09:39:39 menpad unbound[4710]: [4710:0] error: can't bind socket: Permission denied for ::
Sep 17 09:46:28 menpad unbound[4710]: [4710:3] info: generate keytag query _ta-4a5c-4f66. NULL IN
Sep 17 09:48:37 menpad unbound[4710]: [4710:0] notice: sendto failed: Address family not supported by protocol
Sep 17 09:48:37 menpad unbound[4710]: [4710:0] notice: remote address is (inet_ntop error) port 42790
Sep 17 09:48:37 menpad unbound[4710]: *** buffer overflow detected ***: /usr/sbin/unbound terminated
Sep 17 09:48:37 menpad systemd[1]: unbound.service: Main process exited, code=killed, status=6/ABRT
Sep 17 09:48:37 menpad systemd[1]: unbound.service: Unit entered failed state.
Sep 17 09:48:37 menpad systemd[1]: unbound.service: Failed with result 'signal'.
Comment 20 Fedora Update System 2018-10-08 17:49:22 UTC 
unbound-1.7.3-10.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-d0a1151499
Comment 21 Fedora Update System 2018-10-08 18:00:00 UTC 
unbound-1.7.3-10.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-b129e67fab
Comment 22 Fedora Update System 2018-10-08 18:00:13 UTC 
unbound-1.7.3-10.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-b129e67fab
Comment 23 Fedora Update System 2018-10-09 01:17:21 UTC 
unbound-1.7.3-10.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-b129e67fab
Comment 24 Fedora Update System 2018-10-09 02:30:14 UTC 
unbound-1.7.3-10.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-d0a1151499
Comment 25 Fedora Update System 2018-10-09 12:21:59 UTC 
netresolve-0.0.1-0.22.20160317git.fc29 getdns-1.4.2-4.fc29 unbound-1.8.1-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-7be27ff1d8
Comment 26 Fedora Update System 2018-10-09 20:03:19 UTC 
getdns-1.4.2-4.fc29, netresolve-0.0.1-0.22.20160317git.fc29, unbound-1.8.1-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-7be27ff1d8
Comment 27 Fedora Update System 2018-10-10 09:25:55 UTC 
getdns-1.4.2-4.fc29 gnutls-3.6.4-2.fc29 netresolve-0.0.1-0.22.20160317git.fc29 unbound-1.8.1-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-7be27ff1d8
Comment 28 Fedora Update System 2018-10-10 15:49:50 UTC 
getdns-1.4.2-4.fc29, gnutls-3.6.4-2.fc29, netresolve-0.0.1-0.22.20160317git.fc29, unbound-1.8.1-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-7be27ff1d8
Comment 29 Fedora Update System 2018-10-11 09:33:52 UTC 
getdns-1.4.2-4.fc29 gnutls-3.6.4-2.fc29 libreswan-3.27-1.fc29 netresolve-0.0.1-0.22.20160317git.fc29 unbound-1.8.1-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-7be27ff1d8
Comment 30 Fedora Update System 2018-10-11 18:59:26 UTC 
getdns-1.4.2-4.fc29, gnutls-3.6.4-2.fc29, libreswan-3.27-1.fc29, netresolve-0.0.1-0.22.20160317git.fc29, unbound-1.8.1-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-7be27ff1d8
Comment 31 Fedora Update System 2018-10-14 23:40:07 UTC 
unbound-1.7.3-10.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.
Comment 32 Dominik 'Rathann' Mierzejewski 2018-10-15 07:32:52 UTC 
(In reply to Fedora Update System from comment #31)
> unbound-1.7.3-10.fc28 has been pushed to the Fedora 28 stable repository. If
> problems still persist, please make note of it in this bug report.

Unfortunately, I've just had unbound-1.7.3-10.fc28 die with SIGABRT again:
Oct 15 03:56:02 unbound[1131]: [1131:0] notice: sendto failed: Address family not supported by protocol
Oct 15 03:56:02 unbound[1131]: [1131:0] notice: remote address is (inet_ntop error) port 12273
Oct 15 03:56:02 unbound[1131]: *** buffer overflow detected ***: /usr/sbin/unbound terminated
Oct 15 03:56:02 unbound[1131]: [1131:0] notice: sendto failed: Invalid argument
Oct 15 03:56:02 unbound[1131]: [1131:0] notice: remote address is (inet_ntop error) port 12273
Oct 15 03:56:02 systemd[1]: unbound.service: Main process exited, code=killed, status=6/ABRT
Oct 15 03:56:02 systemd[1]: unbound.service: Failed with result 'signal'.

So, not fixed.
Comment 33 Petr Menšík 2018-10-17 18:13:38 UTC 
Ok, I were able to find a way to make unbound dump be created by kernel and catched by abrt after that. It should help very much in uncovering the real case of this issue. As well as any possible crashes available.

It is required to change systemd service type to forking OR move changing of user to systemd. I would prefer the second variant.

Use these commands:

mkdir /etc/systemd/system/unbound.service.d
cat > /etc/systemd/system/unbound.service.d/user.conf << EOF
# vim: set ft=systemd
[Service]
User=unbound
CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_SYS_CHROOT CAP_SYS_RESOURCE CAP_NET_ADMIN CAP_DAC_READ_SEARCH
AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_SYS_CHROOT CAP_SYS_RESOURCE CAP_NET_ADMIN CAP_DAC_READ_SEARCH
EOF
systemctl daemon-reload
systemctl restart unbound
Comment 34 Petr Menšík 2018-10-17 18:48:22 UTC 
A simpler workaround would be to set sysctl fs.suid_dumpable = 1. Please use /etc/sysctl.conf to be permanent.
Comment 35 Fedora Update System 2018-10-18 03:53:29 UTC 
unbound-1.7.3-10.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.
Comment 36 Dominik 'Rathann' Mierzejewski 2018-10-18 10:46:57 UTC 
(In reply to Petr Menšík from comment #33)
> Ok, I were able to find a way to make unbound dump be created by kernel and
> catched by abrt after that. It should help very much in uncovering the real
> case of this issue. As well as any possible crashes available.

Great!

> It is required to change systemd service type to forking OR move changing of
> user to systemd. I would prefer the second variant.
> 
> Use these commands:
[...]
Thanks for the instructions. I did implement this on my machine. Let's see if it yields any new abrt reports.
Comment 37 Fedora Update System 2018-10-19 11:22:16 UTC 
asterisk-16.0.0-1.fc29 getdns-1.4.2-4.fc29 gnutls-3.6.4-2.fc29 libreswan-3.27-1.fc29 netresolve-0.0.1-0.22.20160317git.fc29 unbound-1.8.1-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-7be27ff1d8
Comment 38 Fedora Update System 2018-10-20 19:21:02 UTC 
asterisk-16.0.0-1.fc29, getdns-1.4.2-4.fc29, gnutls-3.6.4-4.fc29, libreswan-3.27-1.fc29, netresolve-0.0.1-0.22.20160317git.fc29, unbound-1.8.1-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-7be27ff1d8
Comment 39 Petr Menšík 2018-10-23 08:22:59 UTC 
Current update is reported insufficient. Not yet reported on F29 and F30 unbound 1.8 releases, but that does not mean it is fixed.
Comment 40 Fedora Update System 2018-11-03 00:00:30 UTC 
asterisk-16.0.0-1.fc29, getdns-1.4.2-4.fc29, gnutls-3.6.4-4.fc29, libreswan-3.27-1.fc29, netresolve-0.0.1-0.22.20160317git.fc29, unbound-1.8.1-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.