Bug 1001606

Summary: abrt / gdb goes into some sort of infinite loop generating backtrace locally
Product: [Fedora] Fedora Reporter: Jonathan Kamens <jik>
Component: satyrAssignee: Richard Marko <rmarko>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 19CC: abrt-devel-list, ars_1, dvlasenk, iprikryl, jberan, jfilak, jmoskovc, mmilata, mtoman, rmarko
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 0.10 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-10-31 14:46:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
GDB output none

Description Jonathan Kamens 2013-08-27 11:40:54 UTC
Created attachment 790920 [details]
GDB output

I was trying to report a crash in /usr/bin/ld.gold, generating the backtrace locally. It was taking a very long time, so I used "ps auxw" to find out what gdb command was being run and then ran it myself in an Emacs buffer to figure out what was taking so long.

Here's the command that I ran:

gdb -batch -ex 'set debug-file-directory /usr/lib/debug:/var/cache/abrt-di/usr/lib/debug' -ex 'file /usr/bin/ld.gold' -ex 'core-file ./coredump' -ex 'thread apply all backtrace 1024 full' -ex 'info sharedlib' -ex 'print (char*)__abort_msg' -ex 'print (char*)__glib_assert_msg' -ex 'info registers' -ex disassemble

It appears to have gone into some sort of infinite, or at least many-times-repeated, loop processing the same data over and over again. It seems to have done it something like 11 times before I gave up and ctrl-c'd it. I've attached the output of the command (near the end where it says "Quit" is where I interrupted it).

In the meantime, ABRT hummed merrily along trying to generate a backtrace. Eventually it logged this and failed:

Backtrace is too big (2113473 bytes), reducing depth to 512
Backtrace is too big (1742281 bytes), reducing depth to 256
Backtrace is too big (2462985 bytes), reducing depth to 128
Backtrace is too big (2103328 bytes), reducing depth to 64
Backtrace is too big (2475360 bytes), reducing depth to 64
Backtrace is generated and saved, 6318 bytes
Backtrace parsing failed for .
7:0: "Thread" header expected
Looking for similar problems in bugzilla

I did some experimentation and discovered that a usable backtrace would have been generated if ABRT had either used a depth of 16 or not specified "full" to the gdb backtrace command.

I'm not sure what the right fix is here, but something is clearly wrong.

Comment 1 Jonathan Kamens 2013-08-27 11:41:58 UTC
abrt-desktop-2.1.6-3.fc19.x86_64
abrt-addon-vmcore-2.1.6-3.fc19.x86_64
abrt-addon-kerneloops-2.1.6-3.fc19.x86_64
abrt-addon-xorg-2.1.6-3.fc19.x86_64
abrt-plugin-bodhi-2.1.6-3.fc19.x86_64
abrt-2.1.6-3.fc19.x86_64
abrt-addon-uefioops-2.1.6-3.fc19.x86_64
abrt-addon-ccpp-2.1.6-3.fc19.x86_64
abrt-gui-2.1.6-3.fc19.x86_64
abrt-libs-2.1.6-3.fc19.x86_64
abrt-retrace-client-2.1.6-3.fc19.x86_64
abrt-addon-python-2.1.6-3.fc19.x86_64
abrt-dbus-2.1.6-3.fc19.x86_64
gdb-7.6-34.fc19.x86_64

Comment 2 Richard Marko 2013-10-02 13:50:44 UTC
Your observations are correct. ABRT reduces length to 64 and then it disables 'thread apply all' and 'full' if 'thread apply all' is not enough.

The problem is in our backtrace parser - satyr, which refuses to parse backtrace without 'Thread' headers.

Comment 3 Richard Marko 2013-10-31 14:46:43 UTC
commit 8ed25963408353b3d17db5b29c0caf18cb8fd5bf
Author: Richard Marko <rmarko>
Date:   Wed Oct 2 15:42:30 2013 +0200

    parse backtrace without Thread header
    
    GDB output may not contain all threads if backtrace
    is too large which leads to failures during parsing.
    
    This patch makes Thread header optional and creates
    single threaded stacktrace if header is missing.
    
    Closes: rhbz#1001606.
    
    Signed-off-by: Richard Marko <rmarko>



Included in satyr-0.10 release.