Bug 576379 - [abrt] hang in elfutils-0.145-1.fc12: Process /usr/bin/eu-unstrip enters endless loop
Summary: [abrt] hang in elfutils-0.145-1.fc12: Process /usr/bin/eu-unstrip enters endl...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: elfutils
Version: 12
Hardware: x86_64
OS: Linux
low
medium
Target Milestone: ---
Assignee: Roland McGrath
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: abrt_hash:429ca705ba55ed4d118d071b3a2...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-03-23 22:19 UTC by Doncho Gunchev
Modified: 2010-05-18 21:55 UTC (History)
2 users (show)

Fixed In Version: elfutils-0.147-1.fc12
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-04-28 03:07:57 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
File: backtrace (7.27 KB, text/plain)
2010-03-23 22:19 UTC, Doncho Gunchev
no flags Details

Description Doncho Gunchev 2010-03-23 22:19:29 UTC
abrt 1.0.8 detected a crash.

architecture: x86_64
Attached file: backtrace
cmdline: eu-unstrip --core=/var/cache/abrt/ccpp-1269380615-30557/coredump -n
component: elfutils
executable: /usr/bin/eu-unstrip
kernel: 2.6.32.9-70.fc12.x86_64
package: elfutils-0.145-1.fc12
rating: 4
reason: Process /usr/bin/eu-unstrip was killed by signal 11 (SIGSEGV)
release: Fedora release 12 (Constantine)

comment
-----
While clicking 2-3 times collapse/expand thread in thunderbird-3.0.3-1.fc12 it crashed. 5 minutes later eu-unstrip was still doing endless

open("\1", O_RDONLY)                    = -1 ENOENT (No such file or directory)

I killed it with SIGSEGV to get a trace. Trying to get backtrace with gdb resulted in similar problem (endless loop). Then I tried

sudo debuginfo-install thunderbird-3.0.3-1.fc12.x86_64

and so on till gdb stopped complaining about missing debuginfo (after Ctrl+C), but that did not help - both still start endless loop.

After Ctrl+C on the thunderbird's core file in gdb backtrace shows:
...
[New Thread 31445]
^CQuit
(gdb) bt
#0  0x0000003ed920efbb in ?? ()
#1  0x0000003b8781e952 in ?? ()
#2  0x0000000000000400 in ?? ()
#3  0x0000000000000000 in ?? ()
(gdb)

How to reproduce
-----
1. killall -SIGSEGV eu-unstrip while it is looping endlessly trying to do something with crashed thunderbird.

Comment 1 Doncho Gunchev 2010-03-23 22:19:32 UTC
Created attachment 402160 [details]
File: backtrace

Comment 2 Roland McGrath 2010-04-06 20:05:07 UTC
If you still have the file /var/cache/abrt/ccpp-1269380615-30557/coredump, please make it available for the elfutils maintainers to examine.

Comment 3 Doncho Gunchev 2010-04-06 23:38:41 UTC
I intentionally do not update my thunderbird and I still have it, but it contains parts of my company email and most likely passwords... which I can't expose to the whole world :(

The whole file is 300MiB, 16MiB zip btw. Any ideas?

I can give you tracebacks and debug it if instructed how/what to look for... recompile elfutils without optimization... The best result so far after installing all debuginfo packages is:

#0  0x0000003ed86d1380 in __open_nocancel () at ../sysdeps/unix/syscall-template.S:82
#1  0x0000003523a15204 in open64 (dwfl=0x60c510, name=0x7fffe9b24900 "\001", file_name=0x7fffe9b24900 "\001", fd=-1, base=140162847137600) at /usr/include/bits/fcntl2.h:86
#2  dwfl_report_elf (dwfl=0x60c510, name=0x7fffe9b24900 "\001", file_name=0x7fffe9b24900 "\001", fd=-1, base=140162847137600) at dwfl_report_elf.c:272
#3  0x0000003523a21bdf in report_r_debug (dwfl=0x7fffffffdae0, auxv=<value optimized out>, auxv_size=<value optimized out>, memory_callback=0x7fffffffdae8, memory_callback_arg=<value optimized out>) at link_map.c:413
#4  dwfl_link_map_report (dwfl=0x7fffffffdae0, auxv=<value optimized out>, auxv_size=<value optimized out>, memory_callback=0x7fffffffdae8, memory_callback_arg=<value optimized out>) at link_map.c:862
#5  0x0000003523a22216 in dwfl_core_file_report (dwfl=0x60c510, elf=0x60c580, ehdr=<value optimized out>) at core-file.c:469
#6  0x0000003523a19e82 in parse_opt (key=<value optimized out>, arg=0x7fffffffe520 "/var/cache/abrt/ccpp-1269380615-30557/coredump") at argp-std.c:229
#7  0x0000003ed86ec5e8 in group_parse (argp=<value optimized out>, argc=<value optimized out>, argv=<value optimized out>, flags=<value optimized out>, end_index=<value optimized out>, input=<value optimized out>) at argp-parse.c:257
#8  parser_parse_opt (argp=<value optimized out>, argc=<value optimized out>, argv=<value optimized out>, flags=<value optimized out>, end_index=<value optimized out>, input=<value optimized out>) at argp-parse.c:756
#9  parser_parse_next (argp=<value optimized out>, argc=<value optimized out>, argv=<value optimized out>, flags=<value optimized out>, end_index=<value optimized out>, input=<value optimized out>) at argp-parse.c:867
#10 __argp_parse (argp=<value optimized out>, argc=<value optimized out>, argv=<value optimized out>, flags=<value optimized out>, end_index=<value optimized out>, input=<value optimized out>) at argp-parse.c:921
#11 0x0000000000406e72 in main (argc=3, argv=0x7fffffffe208) at unstrip.c:2277

And it keeps trying to open this file "\001" forever.

Comment 4 Roland McGrath 2010-04-07 18:12:43 UTC
Unfortunately we'll really need some data file to reproduce the crash so we can debug it in detail, unless you just want to debug it yourself.  You can send me the compressed data file privately if you want, and I won't expose (or even look at) any of your data.  Any data like that will be irrelevant to debugging the problem.  You could edit the file and replace any strings of data with XXXX or whatever (just make sure not to delete any so the file offsets move around).  It's probably the case that any such data is in large data segments that you could just elide from the file, too (though probably replacing with 0 or X is simpler than adjusting the file so it's entirely absent).  From 'eu-readelf -l' output on the file we could guess which portions have the data we really need (which is just the saved text bits from the DSOs and executables, and a small bit of the dynamic linker's data that contains only details about DSOs and no user data).

Comment 5 Roland McGrath 2010-04-08 00:29:49 UTC
Waiting to see if reporter can get the data file to me privately.

Comment 6 Doncho Gunchev 2010-04-10 15:37:35 UTC
I removed my passwords (were both in ascii and UTF16 on several places) and replaced some more data with ghex2 and okteta, tried with gdb to make sure the bug is still reproducible and sent it compressed to Roland McGrath <XXXXXX>. Luck.

(tb-coredump.7z (application/x-7z-compressed) 10,390K)

Comment 7 Roland McGrath 2010-04-13 07:58:48 UTC
I reproduced the failure with the supplied data file, so I will be able to debug it now.

Comment 8 Roland McGrath 2010-04-14 20:00:04 UTC
The link_map list in the core file was clobbered (presumably part of the underlying thunderbird bug) in such a way that it became a circular list.  I've fixed libdwfl upstream to bail out in this case.  The "\1" file name is authentically what appears in a link_map.l_name field (as clobbered), so it is "normal" to try to look that up.  The new loop detection only kicks in at a very high upper bound of what could be valid, so in this file we produce 4257 loops on the bad element and thus 4257 open("\1", O_RDONLY) calls that fail.  But it's no longer unbounded, so it completes as it should.

Comment 9 Fedora Update System 2010-04-22 00:24:52 UTC
elfutils-0.146-1.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/elfutils-0.146-1.fc12

Comment 10 Fedora Update System 2010-04-22 00:25:02 UTC
elfutils-0.146-1.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/elfutils-0.146-1.fc11

Comment 11 Fedora Update System 2010-04-22 00:25:12 UTC
elfutils-0.146-1.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/elfutils-0.146-1.fc13

Comment 12 Roland McGrath 2010-04-22 00:52:13 UTC
This should be fixed by 0.146, now in updates-testing.

Comment 13 Fedora Update System 2010-04-22 22:32:15 UTC
elfutils-0.146-1.fc11 has been pushed to the Fedora 11 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update elfutils'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/elfutils-0.146-1.fc11

Comment 14 Fedora Update System 2010-04-22 22:44:13 UTC
elfutils-0.146-1.fc13 has been pushed to the Fedora 13 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update elfutils'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/elfutils-0.146-1.fc13

Comment 15 Fedora Update System 2010-04-22 22:49:34 UTC
elfutils-0.146-1.fc12 has been pushed to the Fedora 12 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update elfutils'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/elfutils-0.146-1.fc12

Comment 16 Doncho Gunchev 2010-04-26 21:55:27 UTC
Yep, this fixes eu-unstrip - it quits.

Unfortunately gdb is still hanging, but I updated my system so I can not gdb thunderbird-bin coredumpm against the same binary (thunderbird-bin)...

Comment 17 Roland McGrath 2010-04-27 01:32:29 UTC
Thanks for the verification.  I know a GDB maintainer was looking into its version of this problem, but I don't know if there is already a bugzilla report for that.  If you don't find one already open, you can open a new bug against gdb for that.  Please add a comment here mentioning that bug number for future reference.

Comment 18 Roland McGrath 2010-04-27 08:22:48 UTC
FYI, bug 578136 covers this issue in GDB.

Comment 19 Fedora Update System 2010-04-28 03:07:46 UTC
elfutils-0.146-1.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 20 Fedora Update System 2010-05-18 21:50:11 UTC
elfutils-0.147-1.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 21 Fedora Update System 2010-05-18 21:55:45 UTC
elfutils-0.147-1.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.