Created attachment 619619 [details] backtrace Description of problem: The attached backtrace cannot be parsed: [snippet from event_log] 2012-09-30-21:50:18 All debuginfo files are available 2012-09-30-21:50:18 Generating backtrace 2012-09-30-21:50:27 Backtrace is generated and saved, 218280 bytes 2012-09-30-21:50:27 Backtrace parsing failed for . 2012-09-30-21:50:27 11:0: "Thread" header expected Version-Release number of selected component (if applicable): abrt-2.0.13-1.fc17.x86_64 btparser-0.19-1.fc17.x86_64 gdb-7.4.50.20120120-50.fc17.x86_64 How reproducible: Only tried this once. Steps to Reproduce: 1. Crash qemu-kvm. 2. Retrace service failed because the coredump is too large. 3. Quit abrt. 3. Download some symbols manually. 4. Restart abrt. 5. Choose local debugging. Actual results: backtrace cannot be parsed. Expected results: backtrace can be parsed. Additional info: The coredump is huge: $ ls -l coredump -rw-r--r--. 1 stephent stephent 2376052736 Sep 30 21:09 coredump
Created attachment 619620 [details] event_log
$ cat cmdline qemu-kvm -m 2048 -hda f18-test-2.img -vga qxl
Created attachment 619637 [details] gdb log showing output from "t a a bt full" $ gdb qemu-kvm ../ccpp-2012-09-30-21:09:00-2240/coredump GNU gdb (GDB) Fedora (7.4.50.20120120-50.fc17) Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/bin/qemu-kvm...Reading symbols from /usr/lib/debug/usr/bin/qemu-kvm.debug...done. done. [New LWP 2240] [New LWP 2430] [New LWP 2244] [New LWP 2243] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `qemu-kvm -m 2048 -hda f18-test-2.img -vga qxl'. Program terminated with signal 11, Segmentation fault. #0 __memcpy_sse2 () at ../sysdeps/x86_64/memcpy.S:202 202 movq %rax, (%rdi) Missing separate debuginfos, use: debuginfo-install celt051-0.5.1.3-4.fc17.x86_64 cryptopp-5.6.1-7.fc17.x86_64 dbus-libs-1.4.10-5.fc17.x86_64 flac-1.2.1-9.fc17.x86_64 gsm-1.0.13-6.fc17.x86_64 json-c-0.9-4.fc17.x86_64 keyutils-libs-1.5.5-2.fc17.x86_64 krb5-libs-1.10.2-6.fc17.x86_64 libICE-1.0.8-1.fc17.x86_64 libSM-1.2.1-1.fc17.x86_64 libXau-1.0.6-3.fc17.x86_64 libXcursor-1.1.13-1.fc17.x86_64 libXext-1.3.1-1.fc17.x86_64 libXfixes-5.0-2.fc17.x86_64 libXi-1.6.1-1.fc17.x86_64 libXrandr-1.3.1-3.fc17.x86_64 libXrender-0.9.7-1.fc17.x86_64 libXtst-1.2.0-3.fc17.x86_64 libasyncns-0.8-3.fc17.x86_64 libcom_err-1.42.3-2.fc17.x86_64 libdb-5.2.36-5.fc17.x86_64 libgcc-4.7.2-2.fc17.x86_64 libgcrypt-1.5.0-3.fc17.x86_64 libgpg-error-1.10-2.fc17.x86_64 libidn-1.24-1.fc17.x86_64 libogg-1.3.0-1.fc17.x86_64 libselinux-2.1.10-3.fc17.x86_64 libsndfile-1.0.25-2.fc17.x86_64 libssh2-1.4.1-2.fc17.x86_64 libstdc++-4.7.2-2.fc17.x86_64 libtasn1-2.12-1.fc17.x86_64 libvorbis-1.3.3-1.fc17.x86_64 libxcb-1.8.1-1.fc17.x86_64 nss-softokn-freebl-3.13.5-1.fc17.x86_64 openldap-2.4.32-2.fc17.x86_64 openssl-1.0.0j-2.fc17.x86_64 p11-kit-0.12-1.fc17.x86_64 pixman-0.24.4-2.fc17.x86_64 tcp_wrappers-libs-7.6-69.fc17.x86_64 (gdb) set logging on Copying output to gdb.txt. (gdb) t a a bt full ...
See also: Bug 852897 - qemu-kvm crash in validate_surface Created attachment 608006 [details] Backtrace generated by abrt, which it fails to parse
This seems to be a bug in backtrace generation and not in backtrace parsing. Btparser assumes that GDB uses the Thread line to introduce a new thread in a backtrace, and comment #3 indicates this is true also for this bug when the backtrace is generated manually. Why the backtrace is different when generated by ABRT?
(In reply to comment #5) > This seems to be a bug in backtrace generation and not in backtrace parsing. > Btparser assumes that GDB uses the Thread line to introduce a new thread in > a backtrace, and comment #3 indicates this is true also for this bug when > the backtrace is generated manually. Why the backtrace is different when > generated by ABRT? The only difference between those backtraces is that GDB can't find debuginfo if you run the gdb manually, the rest seems the same (at least to me...)
Please disregard comment#6 I didn't notice the missing [Thread ..] line. ABRT uses gdb to generate the backtrace (it runs gdb as a child) so I suspect there is a bug in GDB - is there some option which could make GDB to not output these lines?
As was already told parsing GDB output will never work as GDB output changes. What is the expecte and actual output? When loading a core file GDB prints: [New LWP 30612] [New LWP 30611] This seems to always have been in recent history, the threads are then shown in 'info threads': (gdb) info threads Id Target Id Frame 2 Thread 0x7f02d9ec6700 (LWP 30611) 0x37e4609080 in pthread_join [...] * 1 Thread 0x7f02d9ec4700 (LWP 30612) 0x0000000000 in ?? () With live processes GDB prints: [New Thread 0x7ffff7fd3700 (LWP 21540)] GDB probably could print [New Thread...] even for core files but I do not think it ever was that way, was it?
This recent bug was reported with abrt 2.0.16, and the backtrace has: Thread 6 (Thread 0x7f4729e22700 (LWP 10964)): Bug 868974 - [abrt] nautilus-3.6.1-1.fc18: g_type_check_instance_cast: Process /usr/libexec/nautilus-shell-search-provider was killed by signal 11 (SIGSEGV) What is different about the qemu-kvm examples? The coredump was huge (~2GB): $ ls -l coredump -rw-r--r--. 1 stephent stephent 2376052736 Sep 30 21:09 coredump Thread 3, Frame 1 is huge: gdb log showing output from "t a a bt full" Attachment 619637 [details]
Here's a reproducer: Crash qemu-kvm with 2048MB memory running a Live CD. 1. $ qemu-kvm -m 2048 -cdrom ~/xfr/fedora/F18/F18-Beta/TC6/Fedora-18-Beta-TC6-x86_64-Live-Desktop.iso -usb -vga qxl -usbdevice mouse 2. Boot to the Live CD desktop. 3. From another terminal: $ kill -ABRT <pid-of-qemu-kvm-process> NB: The retrace server will reject the coredump because it is too large. I had to download one debuginfo package with yum, because the connection was so slow. ===== snippet from event_log ===== ... 2012-10-22-09:05:13 Retrace server can not be used, because the crash is too large. Try local retracing. 2012-10-22-09:05:13 The size of your crash is 2391605326 bytes, but the retrace server only accepts crashes smaller or equal to 1342177280 bytes. ... 2012-10-22-09:23:34 All debuginfo files are available 2012-10-22-09:23:34 Generating backtrace 2012-10-22-09:23:45 Backtrace is generated and saved, 217572 bytes 2012-10-22-09:23:45 Backtrace parsing failed for . 2012-10-22-09:23:45 10:0: "Thread" header expected ===== head of backtrace ===== [New LWP 4537] [New LWP 4540] [New LWP 4541] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `qemu-kvm -m 2048 -cdrom /home/stephent/xfr/fedora/F18/F18-Beta/TC6/Fedora-18-Be'. Program terminated with signal 6, Aborted. #0 0x00007f6aed7ac9e3 in select () at ../sysdeps/unix/syscall-template.S:82 82 ../sysdeps/unix/syscall-template.S: No such file or directory. #0 0x00007f6aed7ac9e3 in select () at ../sysdeps/unix/syscall-template.S:82 No locals. #1 0x00007f6af30516f8 in main_loop_wait (nonblocking=<optimized out>) at main-loop.c:456 rfds = {fds_bits = {32, 0 <repeats 15 times>}} wfds = {fds_bits = {0 <repeats 16 times>}} xfds = {fds_bits = {0 <repeats 16 times>}} ret = <optimized out> nfds = 28 tv = {tv_sec = 0, tv_usec = 998107} timeout = 1000 #2 0x00007f6af2f92729 in main_loop () at /usr/src/debug/qemu-kvm-1.0.1/vl.c:1482 nonblocking = <optimized out> last_io = 1
Created attachment 631634 [details] backtrace from reproducer $ rpm -q abrt libreport btparser gdb | sort abrt-2.0.13-1.fc17.x86_64 btparser-0.19-1.fc17.x86_64 gdb-7.4.50.20120120-50.fc17.x86_64 libreport-2.0.14-1.fc17.x86_64
Created attachment 631635 [details] event_log from reproducer
Here's a reproducer that can use either the retrace server or the local debugger: 1. Specify 64 MB of memory and boot to the syslinux menu that is first displayed when booting from the installer DVD. For consistency, use the up arrow to select the first menu item. This also stops the 60 second countdown. $ qemu-kvm -m 64 -cdrom ~/xfr/fedora/F18/F18-Beta/TC6/Fedora-18-Beta-TC6-x86_64-DVD.iso -usb -vga qxl -usbdevice mouse 2. $ kill -ABRT <pid-of-qemu-kvm-process> Summary: With retrace server: Thread header present With local debugger: Thread header absent
This message is a reminder that Fedora 17 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 17. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '17'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 17's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 17 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior to Fedora 17's end of life. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
This message is a reminder that Fedora 18 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 18. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '18'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 18's end of life. Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 18 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior to Fedora 18's end of life. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
ABRT should not depend on GDB output, it should use the GDB Python scripting.
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days