Bug 509124

Event posted on 06-30-2009 06:00pm EDT by woodard

Received: from mail-2.llnl.gov ([unix socket])
         by mail-2.llnl.gov (Cyrus v2.2.12) with LMTPA;
         Mon, 29 Jun 2009 16:33:28 -0700
Received: from nspiron-2.llnl.gov (nspiron-2.llnl.gov [128.115.41.82])
        by mail-2.llnl.gov (8.13.1/8.12.3/LLNL evision: 1.7 $) with ESMTP id n5TNXP0D003170;
        Mon, 29 Jun 2009 16:33:28 -0700
Date: Mon, 29 Jun 2009 16:33:28 -0700
Message-Id: <7u32oi$1eucri.gov>
X-Attachments: None
Received: from auk21.llnl.gov ([134.9.94.176])
  by nspiron-2.llnl.gov with ESMTP; 29 Jun 2009 16:33:28 -0700
From: Matt Wolfe <mwolfe>
To: woodard9
Subject: readelf: Error: Range lists in .debug_info section aren't in ascending order!

Ben,

Enjoy!  This one is definitely preventing the user from seeing local
variables in a debugger (though these have been mostly omitted below).
Let me know if you find any fault with my reproducer or if it works in
g++ 4.4.

-Matt
------------------------------------------------------------------------
ssh alastor
cat > readelf_error2.c << "CAT_INPUT"
class A { public: A(); void f(){} };

class C { C(); };

C::C() {A* p = new A; p->f();}
"CAT_INPUT"
g++-4.3.2 -g -O0 -c readelf_error2.c
readelf -a -w -W readelf_error2.o > readelf_error2.o.readelf-a-w-W.stdout
readelf: Warning: There is a hole [0x10 - 0x4c] in .debug_loc section.
readelf: Warning: There is a hole [0x5c - 0x98] in .debug_loc section.
readelf: Error: Range lists in .debug_info section aren't in ascending order!
readelf: Warning: Range lists in .debug_ranges section start at 0x60
readelf: Warning: There is an overlap [0x70 - 0x0] in .debug_ranges section.
readelf: Warning: There is a hole [0x10 - 0x30] in .debug_ranges section.

------------

Seems to reproduce with 4.4 as well

[ben@apbptr TV-bugs]$ g++44 -g -O0 -c ascend.C
[ben@apbptr TV-bugs]$ readelf -a -w -W ascend.o >/dev/null 
readelf: Warning: There is a hole [0x10 - 0x4c] in .debug_loc section.
readelf: Warning: There is a hole [0x5c - 0x98] in .debug_loc section.
readelf: Error: Range lists in .debug_info section aren't in ascending order!
readelf: Warning: Range lists in .debug_ranges section start at 0x60
readelf: Warning: There is an overlap [0x70 - 0x0] in .debug_ranges section.
readelf: Warning: There is a hole [0x10 - 0x30] in .debug_ranges section.
[ben@apbptr TV-bugs]$ rpm -q gcc44
gcc44-4.4.0-3.el5

This event sent from IssueTracker by kbaxley  [LLNL (HPC)]
 issue 313140

Event posted on 06-30-2009 07:17pm EDT by woodard

http://sourceware.org/ml/binutils/2007-05/msg00359.html
http://sourceware.org/ml/binutils/2007-05/msg00367.html
[ben@apbptr tmp]$ rpm -q binutils
binutils-2.17.50.0.6-12.el5

Message-ID: <4A4A9644.7050809>
Date: Tue, 30 Jun 2009 15:48:36 -0700
From: Ben Woodard <bwoodard>
Reply-To: bwoodard
Organization: Red Hat Inc.
User-Agent: Thunderbird 2.0.0.21 (X11/20090310)
MIME-Version: 1.0
To: Matt Wolfe <mwolfe>
Subject: Re: readelf: Error: Range lists in .debug_info section aren't
in
 ascending order!
References: <7u32oi$1eucri.gov>
In-Reply-To: <7u32oi$1eucri.gov>
X-Enigmail-Version: 0.95.6
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The problem still reproduces with gcc 4.4. I'm not sure the significance
of the fact that these holes. I'm not sure if that is valid DWARF but
unusual or even if it is really a problem.
http://sourceware.org/ml/binutils/2007-05/msg00359.html &
http://sourceware.org/ml/binutils/2007-05/msg00367.html suggests to me
that we just need a newer version of readelf from binutils. The one in
5.3 binutils-2.17.50.0.6-12.el5 might be up to handling gcc-4.1.2-44.el5
but we might need a newer version to handle gcc-4.3.2 and
gcc44-4.4.0-3.el5.

A work around that might be acceptable to you might be using the
eu-readelf from elfutils it doesn't seem to have a problem:

[ben@apbptr TV-bugs]$ eu-readelf  -a -w ascend.o >/dev/null
[ben@apbptr TV-bugs]$

I just removed the -W since that wasn't an available option.

- -ben


woodard assigned to issue for LLNL (HPC).
Status set to: Waiting on Tech
Version set to: '2.1'

This event sent from IssueTracker by kbaxley  [LLNL (HPC)]
 issue 313140

Event posted on 06-30-2009 07:47pm EDT by woodard

Talked with Matt for about half an hour. 

The original problem report was 4.1.2 and part of the problem appears to
exist with that too:

[ben@apbptr TV-bugs]$ readelf -a -w -W ascend.o >/dev/null
readelf: Warning: There is a hole [0x10 - 0x4c] in .debug_loc section.
readelf: Warning: There is a hole [0x5c - 0x98] in .debug_loc section.
[ben@apbptr TV-bugs]$ rpm -q gcc
gcc-4.1.2-44.el5

Though the problem reported here is with readelf, that just happens to be
his debugging tool. The problem he's chasing is that in certain cases the
debug information is missing from a resulting binary and that is making it
impossible for the user to do the debugging that they need to do. The
correct debug information is in the .o but it is missing from the
executable. So it looks like ld might be stripping out the debug info.
Since ld is also part of binutils like readelf it might be that readelf is
manifesting a problem in code that is shared with ld and by fixing the
problem in readelf we can also fix the problem with the missing debug
information after linking. 


This event sent from IssueTracker by kbaxley  [LLNL (HPC)]
 issue 313140

Event posted on 06-30-2009 07:49pm EDT by woodard

Since this is a secondary problem and his user is heating up he asked me if
I could make this a high priority issue. 

It might also be related to:
https://enterprise.redhat.com/issue-tracker/?module=issues&action=view&tid=291087

here's the corresponding bz:

https://bugzilla.redhat.com/show_bug.cgi?id=499164

Severity set to: High

This event sent from IssueTracker by kbaxley  [LLNL (HPC)]
 issue 313140

Created attachment 350131 [details]
sample code

Event posted on 07-08-2009 05:14pm CDT by jbastian

Note: Fedora 11 doesn't appear to have this problem.

$ rpm -q binutils gcc-c++
binutils-2.19.51.0.2-17.fc11.i586
gcc-c++-4.4.0-4.i586

$ g++ -g -O0 -c ascend.C

$ readelf -a -w -W ascend.o >/dev/null
readelf: Error: Range lists in .debug_info section aren't in ascending
order!
readelf: Warning: Range lists in .debug_ranges section start at 0x30
readelf: Warning: There is an overlap [0x58 - 0x0] in .debug_ranges
section.



This event sent from IssueTracker by jbastian 
 issue 313140

Event posted on 07-08-2009 06:16pm CDT by jbastian

As an experiment, I built binutils-2.19.51.0.2-17 from Fedora 11 on RHEL
5.3 with gcc44-c++-4.4.0-6.el5, but it still has holes.

$ PATH=/opt/binutils-2.19.51.0.2-17/usr/bin:${PATH} \
  LD_LIBRARY_PATH=/opt/binutils-2.19.51.0.2-17/usr/lib64 \
  g++44 -g -O0 -L/opt/binutils-2.19.51.0.2-17/usr/lib64 -c ascend.C

$ /opt/binutils-2.19.51.0.2-17/usr/bin/readelf -a -w -W ascend.o
>/dev/null
readelf: Warning: There is a hole [0x10 - 0x4c] in .debug_loc section.
readelf: Warning: There is a hole [0x5c - 0x98] in .debug_loc section.
...


I also ran it through strace to make sure it was actually using the new
binutils. :)

$ PATH=/opt/binutils-2.19.51.0.2-17/usr/bin:${PATH} \
  LD_LIBRARY_PATH=/opt/binutils-2.19.51.0.2-17/usr/lib64 \
  strace -f -e trace=process -o g++44.out \
  g++44 -g -O0 -L/opt/binutils-2.19.51.0.2-17/usr/lib64 -c ascend.C

$ grep binutils g++44.out
...
1449  execve("/opt/binutils-2.19.51.0.2-17/usr/bin/as", ["as",
"-Qy", "-o", "ascend.o", "/tmp/cc1GW1lf.s"], [/* 36 vars */]) = 0



This event sent from IssueTracker by jbastian 
 issue 313140

(1) [->readelf CVS HEAD fix]
One problem is a deficiency in readelf only which produces these messages:

readelf: Error: Range lists in .debug_info section aren't in ascending order!
readelf: Warning: Range lists in .debug_ranges section start at 0x60

It is due to this readelf code (which is still present even in its CVS HEAD):
  if (!use_debug_info)
    /* FIXME: Should we handle this case?  */
    error (_("Range lists in .debug_info section aren't in ascending order!\n"));

DWARF3 says nothing such ordering would be required.

Currently readelf expects DW_AT_ranges are sorted in ascending order in .debug_info.  Just in this case gcc does not produce it such way:

Contents of the .debug_info section:
 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit)
    <29>   DW_AT_ranges      : 0x60
 <1><dc>: Abbrev Number: 15 (DW_TAG_subprogram)
 <2><102>: Abbrev Number: 17 (DW_TAG_lexical_block)
    <103>   DW_AT_ranges      : 0x0
 <1><111>: Abbrev Number: 19 (DW_TAG_subprogram)
 <2><133>: Abbrev Number: 17 (DW_TAG_lexical_block)
    <134>   DW_AT_ranges      : 0x30

Due to this readelf problem it even cannot display .debug_ranges:

Contents of the .debug_ranges section:
    Offset   Begin    End
    00000060 <End of list>
    00000000 <End of list>
    00000030 <End of list>

One should check if the gcc fix of (2) will not make the gcc output compatible with existing readelf.  Such GCC change would no longer require readelf to be extended to be able to parse the current unusual target binary format from gcc.



(2) [->gcc]
Another problem is with the content of .debug_ranges offset 0x60 (DW_TAG_compile_unit).

eu-readelf displays these cases correctly - shown on an executable (not an object file) as the relocations get resolved there and it gets more visible:

DWARF section [35] '.debug_ranges' at offset 0x1190:
 [     0]  0x00000000004006bd <_ZN1CC2Ev+0x11>..0x00000000004006e5 <_ZN1CC2Ev+0x39>
           0x00000000004006f6 <_ZN1CC2Ev+0x4a>..0x000000000040070c <_ZN1CC1Ev>
 [    30]  0x000000000040071d <_ZN1CC1Ev+0x11>..0x0000000000400745 <_ZN1CC1Ev+0x39>
           0x0000000000400756 <_ZN1CC1Ev+0x4a>..0x000000000040076c <_ZN1A1fEv>
 [    60]  0x00000000004006ac <_ZN1CC2Ev>..0x000000000040076c <_ZN1A1fEv>
           0x000000000040076c <_ZN1A1fEv>..0x0000000000400776
           0x00000000004006ac <_ZN1CC2Ev>..0x000000000040070c <_ZN1CC1Ev>
           0x000000000040070c <_ZN1CC1Ev>..0x000000000040076c <_ZN1A1fEv>


One can see a real target binary problem in this case - the offset 0x60 has overlapping entries.  This is forbidden by DWARF3:
    http://dwarf.freestandards.org/Dwarf3.pdf 2.17.3
    Address range entries in a range list may not overlap.

This problem is reproducible on: gcc43-4.3.2-7.el5.x86_64
But it is not reproducible on: gcc-4.1.2-44.el5.x86_64

GCC produces:
        .quad   .Ltext0 # Offset 0x60
        .quad   .Letext0
        .quad   .LFB2   # Offset 0x70
        .quad   .LFE2
        .quad   .LFB5   # Offset 0x80
        .quad   .LFE5
        .quad   .LFB4   # Offset 0x90
        .quad   .LFE4
        .quad   0x0
        .quad   0x0

but .LFB5 .. .LFE5 and .LFB4 .. .LFE4 lay inside .text (.Ltext0 .. .Letext0).

This problem produces the readelf message:
readelf: Warning: There is an overlap [0xb0 - 0x0] in .debug_ranges section.


(3) [->readelf backport]
readelf from binutils-2.17.50.0.6-9.el5.x86_64 also produces:

readelf: Warning: There is a hole [0x10 - 0x4c] in .debug_loc section.
readelf: Warning: There is a hole [0x5c - 0x98] in .debug_loc section.

+ displays invalid output:

Contents of the .debug_loc section:
    Offset   Begin    End      Expression
    00000000 <End of list>
    0000004c <End of list>
    00000098 <End of list>

But this looks to be fixed in recent readelf versions:

Contents of the .debug_loc section:
    Offset   Begin    End      Expression
    00000000 0000000000000000 0000000000000001 (DW_OP_breg7: 8)
    00000000 0000000000000001 0000000000000004 (DW_OP_breg7: 16)
    00000000 0000000000000004 000000000000000a (DW_OP_breg6: 16)
    00000000 <End of list>
    0000004c 0000000000000000 0000000000000001 (DW_OP_breg7: 8)
    0000004c 0000000000000001 0000000000000004 (DW_OP_breg7: 16)
    0000004c 0000000000000004 0000000000000064 (DW_OP_breg6: 16)
    0000004c <End of list>
    00000098 0000000000000064 0000000000000065 (DW_OP_breg7: 8)
    00000098 0000000000000065 0000000000000068 (DW_OP_breg7: 16)
    00000098 0000000000000068 00000000000000c8 (DW_OP_breg6: 16)
    00000098 <End of list>

and the fix just should be backported for RHEL-5.

GCC problem (2) verified as present also on:
gcc44-c++-4.4.0-6.el5.x86_64
GNU C++ 4.5.0 20090709 (experimental)

(1) readelf fix now posted upstream:
    http://sourceware.org/ml/binutils/2009-07/msg00123.html
QA: New testcase: binutils-all/testranges.s

(2) gcc has been fixed based on this bugreport by Jakub upstream:
    http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40713
    gcc44 backport request for RHEL-5.5 is: Bug 510958

(3) Going to be backported for RHEL-5.5 readelf.
    It has been already fixed by Nick Clifton upstream:
    http://sourceware.org/ml/binutils/2009-06/msg00418.html
    http://sourceware.org/ml/binutils-cvs/2009-06/msg00158.html
QA: RHEL-only new testcase: binutils-all/testloc.S
    as a part of binutils-all/readelf.exp (not shown when PASSing)

(3) <http://sourceware.org/ml/binutils/2009-06/msg00359.html>

There was a fix of my testcase by H.J.Lu specifically for ia64:
http://sourceware.org/ml/binutils/2009-07/msg00172.html
requiring a fix of my testcase by Dave Korn:
http://sourceware.org/ml/binutils/2009-07/msg00160.html

Some other updates were for non-RH arches.

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0304.html