Bug 1366145 - dwz applied to a dts-compiled binary complains about section offsets not monotonically increasing
Summary: dwz applied to a dts-compiled binary complains about section offsets not mono...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Developer Toolset
Classification: Red Hat
Component: binutils
Version: DTS 6.0 RHEL 7
Hardware: Unspecified
OS: Linux
medium
medium
Target Milestone: beta1
: 6.0
Assignee: Nick Clifton
QA Contact: Sergey Kolosov
URL:
Whiteboard:
Depends On:
Blocks: 1431255
TreeView+ depends on / blocked
 
Reported: 2016-08-11 08:00 UTC by Martin Cermak
Modified: 2017-03-10 17:59 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Improvements to the linker's ability to create compressed debug sections meant that the name of a debug section was not fully known until very late on in the linking process. (Since compression will only be applied if it saves space, and this can only be determined once the contents of the section have been finalized, which only happens just before the sections are written to diskl. If the section *is* compressed it will be renamed to .zdebug_... but if it is not compressed then it will be left as .debug_...). A consequence of this change meant that the generation of the section name table was delayed until after all the other sections had been created and so it was moved to the end of the output file. But, ther code to generate the entries in the section header table was not updated to reflect this fact. Consequence: The .shstrtab section would appear in the middle of the section header table, but at the end of the actual file. This meant that the file offsets of the entries in the section header table were no longer in a monotonically increasing order. Whilst this property is not a requirement of the ELF standard, it is expected by a variety of other tools which process ELF binaries. Fix: Move the placement of the .shstrtab section to the end of the section header table. Result: Entries in the section header table now appear in increasing order of file offset.
Clone Of:
: 1430255 1431255 (view as bug list)
Environment:
Last Closed: 2016-11-15 10:01:02 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
binutils-rh1366145.patch (2.04 KB, patch)
2016-08-16 20:03 UTC, Jakub Jelinek
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:2721 0 normal SHIPPED_LIVE new packages: devtoolset-6-binutils 2016-11-15 14:29:21 UTC

Description Martin Cermak 2016-08-11 08:00:55 UTC
When dwz is run on a simple binary compiled with devtoolset-6 toolchain, it complains about section offsets not monotonically increasing:

=======
# cat test.c 
int main() { return 0; }

# rpm -qf `which gcc objdump dwz`
devtoolset-6-gcc-6.1.1-4.1.el7.x86_64
devtoolset-6-binutils-2.27-3.el7.x86_64
devtoolset-6-dwz-0.12-1.el7.x86_64
# 
# gcc test.c -g -o test
# ./test
# 
# md5sum test
dcafda5ca2f21fe103f83f50f5f49622  test
# dwz ./test
dwz: Section offsets in ./test not monotonically increasing
# md5sum test
dcafda5ca2f21fe103f83f50f5f49622  test
#
=======

When this source gets compiled with the baseos toolchain (gcc-4.8.5-9.el7, binutils-2.25.1-20.base.el7), then devtoolset-6-dwz works fine on the resulting binary.

This reminds me the prelink bug 1357016 in some way.  Applies consistently to both rhel-6 and rhel-7, all supported arches.

Although this might not be a dwz bug, I'm filing it against dwz as a start.

Comment 1 Jakub Jelinek 2016-08-12 14:38:55 UTC
Looks like binutils bug to me.
If I link with the DTS ld, I get:
...
  [25] .bss              NOBITS          0000000000601024 001024 000004 00  WA  0   0  4
  [26] .comment          PROGBITS        0000000000000000 001024 000059 01  MS  0   0  1
  [27] .debug_aranges    PROGBITS        0000000000000000 00107d 000030 00      0   0  1
  [28] .debug_info       PROGBITS        0000000000000000 0010ad 000054 00      0   0  1
  [29] .debug_abbrev     PROGBITS        0000000000000000 001101 000037 00      0   0  1
  [30] .debug_line       PROGBITS        0000000000000000 001138 000039 00      0   0  1
  [31] .debug_str        PROGBITS        0000000000000000 001171 00005a 01  MS  0   0  1
  [32] .shstrtab         STRTAB          0000000000000000 001a28 000148 00      0   0  1
  [33] .symtab           SYMTAB          0000000000000000 0011d0 000678 18     34  53  8
  [34] .strtab           STRTAB          0000000000000000 001848 0001e0 00      0   0  1
in the test binary (note the sh_offset for .shstrtab being out of order).
Either .shstrtab should be emitted after .strtab instead of .symtab, because it is placed there, or it should use sh_offset 0x11d0 or better 0x11cb, then .symtab 0x1318 and finally .strtab 0x1990.
If I link with the system ld instead, I get sane ordering:
  [25] .bss              NOBITS          0000000000600834 000834 000004 00  WA  0   0  4
  [26] .comment          PROGBITS        0000000000000000 000834 000059 01  MS  0   0  1
  [27] .debug_aranges    PROGBITS        0000000000000000 00088d 00002f 00      0   0  1
  [28] .debug_info       PROGBITS        0000000000000000 0008bc 000042 00      0   0  1
  [29] .debug_abbrev     PROGBITS        0000000000000000 0008fe 000037 00      0   0  1
  [30] .debug_line       PROGBITS        0000000000000000 000935 000039 00      0   0  1
  [31] .debug_str        PROGBITS        0000000000000000 00096e 00005a 01  MS  0   0  1
  [32] .shstrtab         STRTAB          0000000000000000 0009c8 000148 00      0   0  1
  [33] .symtab           SYMTAB          0000000000000000 0013d0 000678 18     34  50  8
  [34] .strtab           STRTAB          0000000000000000 001a48 000234 00      0   0  1
-fuse-ld=gold with DTS binutils also works sanely:
  [26] .bss              NOBITS          0000000000402010 001010 000004 00  WA  0   0  4
  [27] .comment          PROGBITS        0000000000000000 001010 00005a 01  MS  0   0  1
  [28] .debug_info       PROGBITS        0000000000000000 00106a 000054 00      0   0  1
  [29] .debug_abbrev     PROGBITS        0000000000000000 0010be 000037 00      0   0  1
  [30] .debug_aranges    PROGBITS        0000000000000000 0010f5 000030 00      0   0  1
  [31] .debug_line       PROGBITS        0000000000000000 001125 000039 00      0   0  1
  [32] .debug_str        PROGBITS        0000000000000000 00115e 000068 01  MS  0   0  1
  [33] .note.gnu.gold-version NOTE            0000000000000000 0011c8 00001c 00      0   0  4
  [34] .symtab           SYMTAB          0000000000000000 0011e8 0003a8 18     35  22  8
  [35] .strtab           STRTAB          0000000000000000 001590 000234 00      0   0  1
  [36] .shstrtab         STRTAB          0000000000000000 0017c4 00016f 00      0   0  1
dwz isn't the only tool that expects sanely working linker.

Comment 2 Jakub Jelinek 2016-08-12 16:15:07 UTC
I think it is 3e19fb8f990e4ce8a08f9cf2817cd9e9398648d5 https://sourceware.org/bugzilla/show_bug.cgi?id=18277 change that broke this.

Comment 6 Jakub Jelinek 2016-08-16 20:03:59 UTC
Created attachment 1191363 [details]
binutils-rh1366145.patch

E.g. following patch IMHO should work (with minor needed adjustments in dozen or so tests that expect particular section order).

Comment 7 Nick Clifton 2016-08-17 10:45:14 UTC
Hi Jakub,

> E.g. following patch IMHO should work (with minor needed adjustments in
> dozen or so tests that expect particular section order).

Doh!  That is a much better solution than the one that I was working on.

It does have one deficiency however - it does not reorder relocation sections, so these still appear out of order in the resultant binary.  Eg from the ld-x86-64/pr13947 test:

  % ../binutils/readelf -S tmpdir/dump --wide
There are 16 section headers, starting at offset 0x5b8:

Section Headers:
  [Nr] Name         Type        Address          Off    Size   ES Flg Lk Inf Al
...
  [ 6] .text        PROGBITS    0000000000000210 000210 000005 00  AX  0   0  1
  [ 7] .rela.text   RELA        0000000000000000 000530 000018 18   I 13   6  8
readelf: Warning: [ 8]: Section offsets not in a monotonically increasing order.
  [ 8] .eh_frame    PROGBITS    0000000000000218 000218 000000 00   A  0   0  8

I do not know if this will be a problem for prelink/dwz.  I am investigating to see whether I can safely extend your patch to cover these sections.

Cheers
  Nick

Comment 8 Jakub Jelinek 2016-08-17 11:02:41 UTC
dwz/prelink only support non-ET_REL, so for those tools .rel* sections aren't a problem, and unlike the .shstrtab case, it seems as/ld has been doing that since like forever, at least looking at ET_REL files in RHEL4 confirms it.
Of course, if it isn't too hard on the linker side, adjusting even the .rel* sections wouldn't hurt, but it is much lower urgency.
Actually reading the code I wrote many years ago, it seems prelink actually has code to reorder the sections in the section table first so that it is monotonically increasing unless there are overlaps or similar bogosities, and then verifies this invariant before writing a new ELF object.
dwz doesn't have this code though, all linkers just have done a good job (until recently).  In prelink it isn't just sorting, one needs to update all the section references after the sorting.  And the comments indicate also various versions of libelf (the non-elfutils ones) have issues with non-monotonically increasing sh_offset.

Comment 9 Nick Clifton 2016-08-18 07:32:46 UTC
Fixed in binutils-2.27-6.el7

Comment 13 errata-xmlrpc 2016-11-15 10:01:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2721.html


Note You need to log in before you can comment on or make changes to this bug.