Bug 911712 - [abrt] gdb-7.5.1-32.fc18: bsearch_cmp: Process /usr/bin/gdb was killed by signal 11 (SIGSEGV)
Summary: [abrt] gdb-7.5.1-32.fc18: bsearch_cmp: Process /usr/bin/gdb was killed by sig...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: gdb
Version: 18
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Gary Benson
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: abrt_hash:7503148881a4cd7de392564d7e8...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-02-15 17:43 UTC by Ilyes Gouta
Modified: 2013-05-15 12:41 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-04-01 03:27:25 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
File: backtrace (18.22 KB, text/plain)
2013-02-15 17:43 UTC, Ilyes Gouta
no flags Details
File: cgroup (128 bytes, text/plain)
2013-02-15 17:43 UTC, Ilyes Gouta
no flags Details
File: core_backtrace (4.57 KB, text/plain)
2013-02-15 17:43 UTC, Ilyes Gouta
no flags Details
File: dso_list (4.34 KB, text/plain)
2013-02-15 17:43 UTC, Ilyes Gouta
no flags Details
File: environ (3.10 KB, text/plain)
2013-02-15 17:43 UTC, Ilyes Gouta
no flags Details
File: limits (1.29 KB, text/plain)
2013-02-15 17:43 UTC, Ilyes Gouta
no flags Details
File: maps (23.93 KB, text/plain)
2013-02-15 17:43 UTC, Ilyes Gouta
no flags Details
File: open_fds (309 bytes, text/plain)
2013-02-15 17:43 UTC, Ilyes Gouta
no flags Details
File: proc_pid_status (914 bytes, text/plain)
2013-02-15 17:43 UTC, Ilyes Gouta
no flags Details
File: var_log_messages (876 bytes, text/plain)
2013-02-15 17:43 UTC, Ilyes Gouta
no flags Details
core dump (282.91 KB, application/octet-stream)
2013-02-15 18:38 UTC, Ilyes Gouta
no flags Details
core dump 2 (5.37 MB, application/octet-stream)
2013-02-15 18:40 UTC, Ilyes Gouta
no flags Details
DirectFB test case (source code) (1.78 KB, text/plain)
2013-02-15 18:45 UTC, Ilyes Gouta
no flags Details
backtrace of a reproducible crash with gdb-7.5.1-36.fc18.x86_64 (5.50 KB, text/plain)
2013-03-21 10:13 UTC, Stephan Bergmann
no flags Details
valgrind log of a reproducible crash with gdb-7.5.1-36.fc18.x86_64 (67.41 KB, text/plain)
2013-03-21 10:14 UTC, Stephan Bergmann
no flags Details

Description Ilyes Gouta 2013-02-15 17:43:22 UTC
Version-Release number of selected component:
gdb-7.5.1-32.fc18

Additional info:
backtrace_rating: 4
cmdline:        gdb ./internal
crash_function: bsearch_cmp
executable:     /usr/bin/gdb
kernel:         3.7.5-201.fc18.x86_64
remote_result:  607043
uid:            1000

Truncated backtrace:
Thread no. 1 (10 frames)
 #0 bsearch_cmp at ../../gdb/objfiles.c:1280
 #1 bsearch at bsearch.c:37
 #2 find_pc_section at ../../gdb/objfiles.c:1320
 #3 lookup_minimal_symbol_by_pc_section at ../../gdb/minsyms.c:708
 #4 find_pc_sect_symtab at ../../gdb/symtab.c:2076
 #5 find_pc_symtab at ../../gdb/symtab.c:2174
 #6 select_frame at ../../gdb/frame.c:1451
 #7 get_selected_frame at ../../gdb/frame.c:1389
 #8 evaluate_subexp_standard at ../../gdb/eval.c:1132
 #9 evaluate_subexp_c at ../../gdb/c-lang.c:719

Comment 1 Ilyes Gouta 2013-02-15 17:43:25 UTC
Created attachment 697904 [details]
File: backtrace

Comment 2 Ilyes Gouta 2013-02-15 17:43:28 UTC
Created attachment 697905 [details]
File: cgroup

Comment 3 Ilyes Gouta 2013-02-15 17:43:30 UTC
Created attachment 697906 [details]
File: core_backtrace

Comment 4 Ilyes Gouta 2013-02-15 17:43:32 UTC
Created attachment 697907 [details]
File: dso_list

Comment 5 Ilyes Gouta 2013-02-15 17:43:34 UTC
Created attachment 697908 [details]
File: environ

Comment 6 Ilyes Gouta 2013-02-15 17:43:36 UTC
Created attachment 697909 [details]
File: limits

Comment 7 Ilyes Gouta 2013-02-15 17:43:39 UTC
Created attachment 697910 [details]
File: maps

Comment 8 Ilyes Gouta 2013-02-15 17:43:42 UTC
Created attachment 697911 [details]
File: open_fds

Comment 9 Ilyes Gouta 2013-02-15 17:43:44 UTC
Created attachment 697912 [details]
File: proc_pid_status

Comment 10 Ilyes Gouta 2013-02-15 17:43:46 UTC
Created attachment 697913 [details]
File: var_log_messages

Comment 11 Jan Kratochvil 2013-02-15 17:50:07 UTC
Is it reproducible?  Could you provide the core file from crashed GDB?

Comment 12 Ilyes Gouta 2013-02-15 18:02:33 UTC
Hi Jan,

> Is it reproducible?  Could you provide the core file from crashed GDB?

Not really, I'll try to though. I'm debugging a program which is somehow corrupting memory and stuff isn't stable..

Ilyes

Comment 13 Ilyes Gouta 2013-02-15 18:38:32 UTC
Created attachment 697941 [details]
core dump

Comment 14 Ilyes Gouta 2013-02-15 18:40:26 UTC
Created attachment 697942 [details]
core dump 2

Comment 15 Ilyes Gouta 2013-02-15 18:45:22 UTC
Created attachment 697943 [details]
DirectFB test case (source code)


The test case has to be compiled against (the library) DirectFB-1.6.3. It exposes a memory corruption (within DirectFB) which is quite difficult to debug.

Comment 16 Jan Kratochvil 2013-02-15 19:13:26 UTC
(gdb) p *pspace_info->sections[370]
$11 = {the_bfd_section = 0x0, objfile = 0x0, ovly_mapped = 0}
Those should not be zeroes...

(gdb) p pspace_info->num_sections 
$28 = 741

Just to be sure, have you run memtest86+ at least over a night on that box?

Comment 17 Ilyes Gouta 2013-02-15 19:20:43 UTC
Hi Jan,

(In reply to comment #16)
> (gdb) p *pspace_info->sections[370]
> $11 = {the_bfd_section = 0x0, objfile = 0x0, ovly_mapped = 0}
> Those should not be zeroes...
> 
> (gdb) p pspace_info->num_sections 
> $28 = 741
> 
> Just to be sure, have you run memtest86+ at least over a night on that box?

Nope. I'm running this on my laptop which is well functioning.

I guess it's the program which is corrupting the process' memory.

Ilyes

Comment 18 Jan Kratochvil 2013-02-15 19:23:26 UTC
Debugged program really cannot (accidentally) corrupt debugger's memory.

I can only recommend running "valgrind gdb [...]" but that is unusably slow, it would be suitable only if you can crash GDB reproducibly.

With the provided GDB core file I cannot fix it.

Comment 19 Stephan Bergmann 2013-03-21 10:13:52 UTC
Created attachment 713725 [details]
backtrace of a reproducible crash with gdb-7.5.1-36.fc18.x86_64

Comment 20 Stephan Bergmann 2013-03-21 10:14:39 UTC
Created attachment 713727 [details]
valgrind log of a reproducible crash with gdb-7.5.1-36.fc18.x86_64

Comment 21 Stephan Bergmann 2013-03-21 10:16:22 UTC
I happen to be able to reproduce this when using gdb-7.5.1-36.fc18.x86_64 to debug a local LibreOffice build, see attached backtrace and valgrind log.

Comment 22 Jan Kratochvil 2013-03-21 18:14:38 UTC
I have found in the "core file 2" from Comment 14:
(gdb) p *pspace_info
$2 = {objfiles_changed_p = 1, sections = 0x2ca4ae0, num_sections = 741,
      inhibit_updates = 1}
      ^^^^^^^^^^^^^^^^^^^

One may guess it also happened due to free_objfile as seen on the valgrind log of Comment 20.

Gary, there should be at least this assertion; I do not know yet how but apparently it can happen.
--- ./gdb/objfiles.c-orig	2013-03-21 18:51:00.141957331 +0100
+++ ./gdb/objfiles.c	2013-03-21 19:09:54.531962495 +0100
@@ -615,6 +615,7 @@ free_objfile (struct objfile *objfile)
   obstack_free (&objfile->objfile_obstack, 0);
 
   /* Rebuild section map next time we need it.  */
+  gdb_assert (!get_objfile_pspace_data (objfile->pspace)->inhibit_updates);
   get_objfile_pspace_data (objfile->pspace)->objfiles_changed_p = 1;
 
   xfree (objfile);

Comment 23 Jan Kratochvil 2013-03-21 19:02:40 UTC
I have tried now this patch:
http://pkgs.fedoraproject.org/cgit/gdb.git/plain/gdb-dlopen-stap-probe-inhibit.patch?h=f18

Does it fix the problem?
gdb-7.5.1-37.fc18
https://koji.fedoraproject.org/koji/buildinfo?buildID=404757

Comment 24 Stephan Bergmann 2013-03-22 08:01:06 UTC
(In reply to comment #23)
> Does it fix the problem?
> gdb-7.5.1-37.fc18
> https://koji.fedoraproject.org/koji/buildinfo?buildID=404757

Yes, it does!  Thanks.

Comment 25 Fedora Update System 2013-03-22 09:35:22 UTC
gdb-7.5.1-37.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/gdb-7.5.1-37.fc18

Comment 26 Jan Kratochvil 2013-03-22 09:38:38 UTC
(In reply to comment #22)
> Gary, there should be at least this assertion; I do not know yet how but
> apparently it can happen.

The bug was different, but the assertion is still useful IMO.


(In reply to comment #24)
> Yes, it does!  Thanks.

Thanks for the great valgrind output.

Comment 27 Fedora Update System 2013-03-22 21:16:49 UTC
Package gdb-7.5.1-37.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing gdb-7.5.1-37.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-4269/gdb-7.5.1-37.fc18
then log in and leave karma (feedback).

Comment 28 Jacek Wielemborek 2013-03-28 18:33:03 UTC
Was trying to debug my own build of Darkplaces game engine.

backtrace_rating: 4
cmdline:        /usr/bin/gdb -nx -fullname -quiet -args bin/Debug/Darkplaces-0
crash_function: bsearch_cmp
executable:     /usr/bin/gdb
kernel:         3.8.4-202.fc18.x86_64
package:        gdb-7.5.1-36.fc18
reason:         Process /usr/bin/gdb was killed by signal 11 (SIGSEGV)
uid:            1000
ureports_counter: 3

Comment 29 Fedora Update System 2013-04-01 03:27:28 UTC
gdb-7.5.1-37.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 30 Gary Benson 2013-04-04 10:54:43 UTC
FWIW this fix is not required for the latest version (in archer git)

Comment 31 Gary Benson 2013-05-15 12:41:26 UTC
The fix in gdb-7.5.1-37.fc18 prevents the crash, but makes GDB considerably slower when debugging inferiors with large numbers of shared objects.  I have updated gbenson/rtld-probes on archer.git with code that should avoid this crash without losing performance.


Note You need to log in before you can comment on or make changes to this bug.