Bug 1638798

Summary: gdb does crash randomly on loading symbols or setting a breakpoint
Product: [Fedora] Fedora Reporter: Victor Stinner <vstinner>
Component: gdbAssignee: Keith Seitz <keiths>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: jan.kratochvil, jpokorny, keiths, kevinb, kloczko.tomasz, mikhail.v.gavrilov, pmuldoon, sergiodj
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: gdb-8.2.50.20190120-13.fc30 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1667919 (view as bug list) Environment:
Last Closed: 2019-01-21 20:44:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1541967, 1634781    
Attachments:
Description Flags
Python ELF binary including debug symbols (not stripped), compressed by bzip2
none
Patch to fix the bug by Keith Seitz none

Description Victor Stinner 2018-10-12 13:13:42 UTC
Description of problem:
On Fedora Rawhide, gdb does crash on loading a program with debug symbols, or on setting a breakpoint. gdb 8.2.50.20181006-4.fc30.


Version-Release number of selected component (if applicable):
8.2.50.20181006-4.fc30

How reproducible:


Steps to Reproduce:
1. Install Fedora rawhide. Example using mock:

(a) sudo mock -r fedora-rawhide-x86_64 --init
(b) sudo mock -r fedora-rawhide-x86_64 --install vim dnf
(b) sudo mock -r fedora-rawhide-x86_64 --shell
<vstinner> i cannot survive without vim :-(

2. Install gdb and gdb debug symbols:

* dnf install gdb
* dnf install 'dnf-command(debuginfo-install)'
* dnf debuginfo-install gdb

3. Run "gdb gdb". If it doesn't crash, run "b main".

Actual results:

segfault

Expected results:

Joy.

Additional info:

It seems to be a recent regressions, I'm quite sure that Fedora Rawhide was fine 1 month ago.

Comment 1 Victor Stinner 2018-10-12 13:17:29 UTC
Commands to get a core dump in a mock container:

mount -o rw,remount /proc/sys
echo '%e.%p' >| /proc/sys/kernel/core_pattern
gdb gdb
(... do something to make it crash)

But... loading the coredump does crash gdb as well:

<mock-chroot> sh-4.4# gdb /usr/bin/gdb -c gdb.166
GNU gdb (GDB) Fedora 8.2.50.20181006-4.fc30
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/gdb...
Reading symbols from /usr/lib/debug/usr/libexec/gdb-8.2.50.20181006-4.fc30.x86_64.debug...
[New LWP 166]
[New LWP 173]
[New LWP 167]
[New LWP 169]
[New LWP 170]
[New LWP 172]
[New LWP 168]
[New LWP 171]

warning: Loadable section ".note.gnu.property" outside of ELF segments
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

warning: Loadable section ".note.gnu.property" outside of ELF segments

warning: Loadable section ".note.gnu.property" outside of ELF segments

warning: Loadable section ".note.gnu.property" outside of ELF segments

warning: Loadable section ".note.gnu.property" outside of ELF segments

warning: Loadable section ".note.gnu.property" outside of ELF segments

warning: Loadable section ".note.gnu.property" outside of ELF segments

warning: Loadable section ".note.gnu.property" outside of ELF segments
Core was generated by `gdb gdb'.
Program terminated with signal SIGSEGV, Segmentation fault.
Segmentation fault (core dumped)

Comment 2 Victor Stinner 2018-10-12 13:40:32 UTC
I also tried 8.2.50.20181010-5.fc30: it doesn't fix the bug.

Comment 3 Victor Stinner 2018-10-12 13:49:45 UTC
* gdb-8.2.50.20180917-1: still crash (package built at 2018-09-17)
* gdb-8.1.90.20180727-44: works (package built at 2018-08-20)

Comment 4 Victor Stinner 2018-10-12 15:23:53 UTC
Created attachment 1493354 [details]
Python ELF binary including debug symbols (not stripped), compressed by bzip2

Comment 5 Victor Stinner 2018-10-12 15:25:38 UTC
I built Python 3.7rc1 using the following command:

CFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv '
export CFLAGS
CXXFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv'
export CXXFLAGS
FFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -I/usr/lib64/gfortran/modules'
export FFLAGS
FCFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -I/usr/lib64/gfortran/modules'
export FCFLAGS
LDFLAGS='-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -g '
export LDFLAGS

./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --enable-ipv6 --with-computed-gotos=yes --with-dbmliborder=gdbm:ndbm:bdb --with-system-expat --with-system-ffi --enable-loadable-sqlite-extensions --with-dtrace --with-lto --with-ssl-default-suites=openssl  --without-ensurepip --with-pydebug
# --with-valgrind
make
./python -m test -v test_gdb

Comment 6 Victor Stinner 2018-10-12 15:55:59 UTC
You can reproduce the bug by loading the attached "Python ELF binary including debug symbols (not stripped), compressed by bzip2".


The crash has been introduced by this following commit. I identified it using git bisect, but keiths already found it before me :-)

commit 804d27291b2922f01fe5cc2d9ae759533d4a2f6f
Author: Tom Tromey <tom>
Date:   Tue May 22 15:22:53 2018 -0600

    Convert the DWARF reader to new-style buildysm

Comment 7 Victor Stinner 2018-10-12 16:15:02 UTC
Created attachment 1493362 [details]
Patch to fix the bug by Keith Seitz

Keith Seitz provided me attached gdb.patch (written for the master branch of gdb) which fixes my issue! He is working on pushing this fix upstream.

Comment 8 Sergio Durigan Junior 2018-10-12 18:58:17 UTC
I'm tempted to close this bug as a duplicate of Bug 1560010.  Keith, what do you think?

Comment 9 Keith Seitz 2018-10-12 19:26:46 UTC
(In reply to Sergio Durigan Junior from comment #8)
> I'm tempted to close this bug as a duplicate of Bug 1560010.  Keith, what do
> you think?

No, this is an entirely different bug that crept in with the buildsym_compunit C++-ification in late May or so. Please don't dup it. [NOTE: It is *very*
likely that someone hitting symtab/23010 (aka 1560010) will also hit this bug, but only if he has my patch for that bug, too. ;-)

I'll have a patch submitted upstream by EOD, so I'll also assign this to myself.

Comment 10 Keith Seitz 2018-10-12 21:22:45 UTC
Upstream patch submitted:

https://sourceware.org/ml/gdb-patches/2018-10/msg00299.html

Comment 11 Sergio Durigan Junior 2018-10-13 18:41:28 UTC
*** Bug 1638974 has been marked as a duplicate of this bug. ***

Comment 12 Sergio Durigan Junior 2018-10-13 20:34:05 UTC
*** Bug 1638989 has been marked as a duplicate of this bug. ***

Comment 13 Victor Stinner 2018-11-08 16:12:42 UTC
Keith Seitz submitted a different patch, still waiting for a review:
[RFC PATCH] Introduce dwarf2_cu::get_builder
https://sourceware.org/ml/gdb-patches/2018-10/msg00520.html

Comment 14 Sergio Durigan Junior 2018-12-20 16:49:06 UTC
*** Bug 1661199 has been marked as a duplicate of this bug. ***

Comment 15 Victor Stinner 2019-01-07 11:58:57 UTC
Happy new year! Any update on this crash?

Comment 16 Sergio Durigan Junior 2019-01-07 16:09:47 UTC
Relaying the question to Keith.  IIRC the patch has been approved, so once he pushes it I can rebase Rawhide GDB.

Comment 17 Keith Seitz 2019-01-16 19:37:20 UTC
(In reply to Sergio Durigan Junior from comment #16)

Apologies for the delay. RHEL/DTS has kept me a little too busy the past ten weeks...

I should be pushing the patch for this today.

Comment 18 Victor Stinner 2019-01-21 09:02:02 UTC
I guess that the fix is:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=c24bdb023c8e1fa969d6eb945059fa8ed0d490c7

Is a bugfix release scheduled or a Fedora backport?

Note: gdb on python is broken for 1 year: bz#1541967.

Comment 19 Sergio Durigan Junior 2019-01-21 12:31:57 UTC
(In reply to Victor Stinner from comment #18)
> I guess that the fix is:
> https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;
> h=c24bdb023c8e1fa969d6eb945059fa8ed0d490c7
> 
> Is a bugfix release scheduled or a Fedora backport?

Yes, I'm working on a new Fedora GDB Rawhide release right now.  Sorry about the delay, I was sick last week.

> Note: gdb on python is broken for 1 year: bz#1541967.

I hope this improves the situation.  BTW, would you like me to backport this patch to F29 GDB as well?  If so, I'd kindly ask you to clone this bug.  Thanks.

Comment 20 Victor Stinner 2019-01-21 12:58:42 UTC
> BTW, would you like me to backport this patch to F29 GDB as well?  If so, I'd kindly ask you to clone this bug.

Pleeeeease 🐱 I created https://bugzilla.redhat.com/show_bug.cgi?id=1667919

Note: I don't bother about F28, sorry. Moreover, I don't know if F28 is affected.

Comment 21 Sergio Durigan Junior 2019-01-21 20:44:33 UTC
Fixed.  Thanks for the report, and sorry about the (long) delay.