Bug 1001606 - abrt / gdb goes into some sort of infinite loop generating backtrace locally
abrt / gdb goes into some sort of infinite loop generating backtrace locally
Product: Fedora
Classification: Fedora
Component: satyr (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Richard Marko
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2013-08-27 07:40 EDT by Jonathan Kamens
Modified: 2016-01-31 21:23 EST (History)
10 users (show)

See Also:
Fixed In Version: 0.10
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-10-31 10:46:43 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
GDB output (1.51 MB, text/plain)
2013-08-27 07:40 EDT, Jonathan Kamens
no flags Details

  None (edit)
Description Jonathan Kamens 2013-08-27 07:40:54 EDT
Created attachment 790920 [details]
GDB output

I was trying to report a crash in /usr/bin/ld.gold, generating the backtrace locally. It was taking a very long time, so I used "ps auxw" to find out what gdb command was being run and then ran it myself in an Emacs buffer to figure out what was taking so long.

Here's the command that I ran:

gdb -batch -ex 'set debug-file-directory /usr/lib/debug:/var/cache/abrt-di/usr/lib/debug' -ex 'file /usr/bin/ld.gold' -ex 'core-file ./coredump' -ex 'thread apply all backtrace 1024 full' -ex 'info sharedlib' -ex 'print (char*)__abort_msg' -ex 'print (char*)__glib_assert_msg' -ex 'info registers' -ex disassemble

It appears to have gone into some sort of infinite, or at least many-times-repeated, loop processing the same data over and over again. It seems to have done it something like 11 times before I gave up and ctrl-c'd it. I've attached the output of the command (near the end where it says "Quit" is where I interrupted it).

In the meantime, ABRT hummed merrily along trying to generate a backtrace. Eventually it logged this and failed:

Backtrace is too big (2113473 bytes), reducing depth to 512
Backtrace is too big (1742281 bytes), reducing depth to 256
Backtrace is too big (2462985 bytes), reducing depth to 128
Backtrace is too big (2103328 bytes), reducing depth to 64
Backtrace is too big (2475360 bytes), reducing depth to 64
Backtrace is generated and saved, 6318 bytes
Backtrace parsing failed for .
7:0: "Thread" header expected
Looking for similar problems in bugzilla

I did some experimentation and discovered that a usable backtrace would have been generated if ABRT had either used a depth of 16 or not specified "full" to the gdb backtrace command.

I'm not sure what the right fix is here, but something is clearly wrong.
Comment 1 Jonathan Kamens 2013-08-27 07:41:58 EDT
Comment 2 Richard Marko 2013-10-02 09:50:44 EDT
Your observations are correct. ABRT reduces length to 64 and then it disables 'thread apply all' and 'full' if 'thread apply all' is not enough.

The problem is in our backtrace parser - satyr, which refuses to parse backtrace without 'Thread' headers.
Comment 3 Richard Marko 2013-10-31 10:46:43 EDT
commit 8ed25963408353b3d17db5b29c0caf18cb8fd5bf
Author: Richard Marko <rmarko@redhat.com>
Date:   Wed Oct 2 15:42:30 2013 +0200

    parse backtrace without Thread header
    GDB output may not contain all threads if backtrace
    is too large which leads to failures during parsing.
    This patch makes Thread header optional and creates
    single threaded stacktrace if header is missing.
    Closes: rhbz#1001606.
    Signed-off-by: Richard Marko <rmarko@redhat.com>

Included in satyr-0.10 release.

Note You need to log in before you can comment on or make changes to this bug.