Bug 1236699 - eu-unstrip too strict sanity check for file with separate .shstrtab/.strtab in main/debug file.
Summary: eu-unstrip too strict sanity check for file with separate .shstrtab/.strtab i...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: elfutils
Version: 23
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Mark Wielaard
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-29 18:23 UTC by H.J. Lu
Modified: 2016-02-05 21:51 UTC (History)
8 users (show)

Fixed In Version: elfutils-0.165-2.fc23 elfutils-0.165-2.fc22
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-05 21:51:20 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
A testcase (724 bytes, application/x-xz)
2015-06-29 18:23 UTC, H.J. Lu
no flags Details
A testcase compiled with clang 3.7 (2.09 KB, application/octet-stream)
2015-06-30 10:52 UTC, H.J. Lu
no flags Details
A ET_DYN testcase (2.39 KB, application/octet-stream)
2015-06-30 10:59 UTC, H.J. Lu
no flags Details

Description H.J. Lu 2015-06-29 18:23:57 UTC
Created attachment 1044436 [details]
A testcase

[hjl@gnu-6 strtab]$ cat foo.c
int
main(void)
{
  return 0;
}
[hjl@gnu-6 strtab]$ make
cc -O -g -c foo.c
eu-strip -f debug.o foo.o -o stripped.o
readelf -s debug.o

Symbol table '.symtab' contains 14 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS foo.c
     2: 0000000000000000     0 SECTION LOCAL  DEFAULT    1 
     3: 0000000000000000     0 SECTION LOCAL  DEFAULT    2 
     4: 0000000000000000     0 SECTION LOCAL  DEFAULT    3 
     5: 0000000000000000     0 SECTION LOCAL  DEFAULT    4 
     6: 0000000000000000     0 SECTION LOCAL  DEFAULT    6 
     7: 0000000000000000     0 SECTION LOCAL  DEFAULT    7 
     8: 0000000000000000     0 SECTION LOCAL  DEFAULT    9 
     9: 0000000000000000     0 SECTION LOCAL  DEFAULT   11 
    10: 0000000000000000     0 SECTION LOCAL  DEFAULT   13 
    11: 0000000000000000     0 SECTION LOCAL  DEFAULT   14 
    12: 0000000000000000     0 SECTION LOCAL  DEFAULT   12 
    13: 0000000000000000     6 FUNC    GLOBAL DEFAULT    1 main
readelf -s foo.o

Symbol table '.symtab' contains 14 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS foo.c
     2: 0000000000000000     0 SECTION LOCAL  DEFAULT    1 
     3: 0000000000000000     0 SECTION LOCAL  DEFAULT    2 
     4: 0000000000000000     0 SECTION LOCAL  DEFAULT    3 
     5: 0000000000000000     0 SECTION LOCAL  DEFAULT    4 
     6: 0000000000000000     0 SECTION LOCAL  DEFAULT    6 
     7: 0000000000000000     0 SECTION LOCAL  DEFAULT    7 
     8: 0000000000000000     0 SECTION LOCAL  DEFAULT    9 
     9: 0000000000000000     0 SECTION LOCAL  DEFAULT   11 
    10: 0000000000000000     0 SECTION LOCAL  DEFAULT   13 
    11: 0000000000000000     0 SECTION LOCAL  DEFAULT   14 
    12: 0000000000000000     0 SECTION LOCAL  DEFAULT   12 
    13: 0000000000000000     6 FUNC    GLOBAL DEFAULT    1 main
readelf -s stripped.o

Symbol table '.symtab' contains 8 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND <corrupt>
     1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS <corrupt>
     2: 0000000000000000     0 SECTION LOCAL  DEFAULT    1 <corrupt>

Comment 1 H.J. Lu 2015-06-29 23:56:53 UTC
Clang 3.7 doesn't generate .shstrtab section:

clang -c  -O foo.c 
[hjl@gnu-6 strtab]$ readelf -SW foo.o
There are 10 section headers, starting at offset 0x1bc:

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .strtab           STRTAB          00000000 000164 000055 00      0   0  1
  [ 2] .text             PROGBITS        00000000 000040 000003 00  AX  0   0 16
  [ 3] .data             PROGBITS        00000000 000044 000000 00  WA  0   0  4
  [ 4] .bss              NOBITS          00000000 000044 000000 00  WA  0   0  4
  [ 5] .comment          PROGBITS        00000000 000044 0000a6 01  MS  0   0  1
  [ 6] .note.GNU-stack   PROGBITS        00000000 0000ea 000000 00      0   0  1
  [ 7] .eh_frame         PROGBITS        00000000 0000ec 00002c 00   A  0   0  4
  [ 8] .rela.eh_frame    RELA            00000000 000158 00000c 0c      9   7  4
  [ 9] .symtab           SYMTAB          00000000 000118 000040 10      1   3  4
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)
[hjl@gnu-6 strtab]$

Comment 2 Mark Wielaard 2015-06-30 07:56:59 UTC
I think this might be a real bug. eu-strip probably doesn't correctly handle reusing the symtab .strtab as section header string table. Although that is certainly a valid thing to do. So lets fix that.

But your testcase is confusing. Although eu-strip handles kernel modules, which are a special kind of ET_REL files, eu-strip doesn't handle generic ET_REL files. And your example also uses old GNU style .zdebug sections in ET_REL files, which are not well defined because relocations will be applied wrongly. So it will go wrong for various other reasons.

Do you have an example that is just a regular ET_EXEC or ET_DYN file with just the section header string table index pointing to the .strtab section instead of having a separate .shstrtab section?

Comment 3 H.J. Lu 2015-06-30 10:52:08 UTC
Created attachment 1044662 [details]
A testcase compiled with clang 3.7

Comment 4 Mark Wielaard 2015-06-30 10:56:07 UTC
(In reply to H.J. Lu from comment #3)
> Created attachment 1044662 [details]
> A testcase compiled with clang 3.7

Thanks. If you could provide a normal ET_EXEC or ET_DYN file as requested in comment #2 that would be great.

Comment 5 H.J. Lu 2015-06-30 10:59:29 UTC
Created attachment 1044664 [details]
A ET_DYN  testcase

Comment 6 Mark Wielaard 2015-06-30 11:18:04 UTC
(In reply to H.J. Lu from comment #5)
> Created attachment 1044664 [details]
> A ET_DYN  testcase

Thanks. So how does that example fail for you?

If I run "eu-strip -o foo-stripped.so -f foo-debug.so foo.so" it seems to do the right thing. The foo-debug.so (re)uses the .strtab section as section header name table and for foo-stripped.so eu-strip generates a new .shstrtab section to hold the section header names.

Are you running it differently?
Or does it only happen for those ET_REL files?

Comment 7 H.J. Lu 2015-06-30 11:23:57 UTC
(In reply to Mark Wielaard from comment #6)
> (In reply to H.J. Lu from comment #5)
> > Created attachment 1044664 [details]
> > A ET_DYN  testcase
> 
> Thanks. So how does that example fail for you?

clang -O -g -m32 -c foo.c
ld -shared -m elf_i386 -o foo.so foo.o
eu-strip -g -f debug.so foo.so -o stripped.so
./unstrip debug.so stripped.so -o unstripped.so
./unstrip: more sections in stripped file than debug file -- arguments reversed?
make: *** [unstripped.so] Error 1
[hjl@gnu-6 strtab]$

Comment 8 Mark Wielaard 2015-06-30 11:28:37 UTC
(In reply to H.J. Lu from comment #7)
> (In reply to Mark Wielaard from comment #6)
> > (In reply to H.J. Lu from comment #5)
> > > Created attachment 1044664 [details]
> > > A ET_DYN  testcase
> > 
> > Thanks. So how does that example fail for you?
> 
> clang -O -g -m32 -c foo.c
> ld -shared -m elf_i386 -o foo.so foo.o
> eu-strip -g -f debug.so foo.so -o stripped.so
> ./unstrip debug.so stripped.so -o unstripped.so
> ./unstrip: more sections in stripped file than debug file -- arguments
> reversed?
> make: *** [unstripped.so] Error 1
> [hjl@gnu-6 strtab]$

OK, so the problem isn't with eu-strip, but with eu-unstrip?
Or is this some other unstrip program you are running?

Comment 9 H.J. Lu 2015-06-30 11:33:24 UTC
(In reply to Mark Wielaard from comment #8)
> 
> OK, so the problem isn't with eu-strip, but with eu-unstrip?
> Or is this some other unstrip program you are running?

eu-strip generated the debug output which eu-unstrip can't handle:

[hjl@gnu-6 strtab]$ readelf -SW debug.so
There are 17 section headers, starting at offset 0x554:

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .hash             NOBITS          000000b4 0000b4 000028 04   A  2   0  4
  [ 2] .dynsym           NOBITS          000000dc 0000b4 000050 10   A  3   1  4
  [ 3] .dynstr           NOBITS          0000012c 0000b4 00001e 00   A  0   0  1
  [ 4] .text             NOBITS          00000150 0000c0 000003 00  AX  0   0 16
  [ 5] .eh_frame         NOBITS          00000154 0000c0 000000 00   A  0   0  4
  [ 6] .dynamic          NOBITS          00001154 0000c0 000058 08  WA  3   0  4
  [ 7] .comment          NOBITS          00000000 0000c0 0000a5 01  MS  0   0  1
  [ 8] .debug_pubnames   PROGBITS        00000000 0000c0 00001b 00      0   0  1
  [ 9] .debug_info       PROGBITS        00000000 0000db 000043 00      0   0  1
  [10] .debug_abbrev     PROGBITS        00000000 00011e 00003f 00      0   0  1
  [11] .debug_line       PROGBITS        00000000 00015d 000037 00      0   0  1
  [12] .debug_frame      PROGBITS        00000000 000194 000024 00      0   0  4
  [13] .debug_str        PROGBITS        00000000 0001b8 0000d6 01  MS  0   0  1
  [14] .debug_pubtypes   PROGBITS        00000000 00028e 00001a 00      0   0  1
  [15] .strtab           STRTAB          00000000 0002a8 0000e9 00      0   0  1
  [16] .symtab           SYMTAB          00000000 000394 0001c0 10     15  24  4
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)
[hjl@gnu-6 strtab]$

Comment 10 Mark Wielaard 2015-06-30 11:39:40 UTC
OK, in that case the sanity check in eu-unstrip probably just needs to be relaxed a bit for the case the stripped file uses a separate section header table, while the debug file (re)uses the .strtab section.

Just commenting out the sanity check seems to work just fine:

diff --git a/src/unstrip.c b/src/unstrip.c
index 4a8e5fa..3929853 100644
--- a/src/unstrip.c
+++ b/src/unstrip.c
@@ -1239,9 +1239,11 @@ copy_elided_sections (Elf *unstripped, Elf *stripped,
   ELF_CHECK (elf_getshdrnum (stripped, &stripped_shnum) == 0,
             _("cannot get section count: %s"));
 
+/*
   if (unlikely (stripped_shnum > unstripped_shnum))
     error (EXIT_FAILURE, 0, _("\
 more sections in stripped file than debug file -- arguments reversed?"));
+*/
 
   /* Cache the stripped file's section details.  */
   struct section sections[stripped_shnum - 1];

LD_LIBRARY_PATH=backends:libelf:libdw src/unstrip foo-stripped.so foo-debug.so -o foo-unstripped.so

Produces a foo-unstripped.so that is identical to the orginal foo.so:
$ eu-elfcmp /tmp/foo.so /tmp/foo-unstripped.so
$ echo $?
0

Does that also work for?
Then we just need to make that sanity check a little smarter.

Comment 11 H.J. Lu 2015-06-30 12:00:17 UTC
Remove sanity check leads to

(gdb) r  debug.so stripped.so -o unstripped.so
Starting program: /export/home/hjl/bugs/misc/strtab/eu-unstrip debug.so stripped.so -o unstripped.so
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x000000000040b304 in ebl_strtaboffset (se=0x0)
    at /export/gnu/import/git/elfutils/libebl/eblstrtab.c:348
348	  return se->offset;
Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-9.fc20.x86_64 glibc-2.18-19.2.fc20.x86_64 xz-libs-5.1.2-12alpha.fc20.x86_64 zlib-1.2.8-3.fc20.x86_64
(gdb) bt
#0  0x000000000040b304 in ebl_strtaboffset (se=0x0)
    at /export/gnu/import/git/elfutils/libebl/eblstrtab.c:348
#1  0x000000000040772b in copy_elided_sections (unstripped=0x6107d0, 
    stripped=0x60f060, stripped_ehdr=0x7fffffffde00, bias=0)
    at /export/gnu/import/git/elfutils/src/unstrip.c:1509
#2  0x0000000000409732 in handle_file (
    output_file=0x7fffffffe3ad "unstripped.so", create_dirs=false, 
    stripped=0x60f060, stripped_ehdr=0x7fffffffde00, unstripped=0x60fdb0)
    at /export/gnu/import/git/elfutils/src/unstrip.c:1938
#3  0x0000000000409b0c in handle_explicit_files (
    output_file=0x7fffffffe3ad "unstripped.so", create_dirs=false, force=false)
    at /export/gnu/import/git/elfutils/src/unstrip.c:2003
#4  0x000000000040a98c in main (argc=5, argv=0x7fffffffe078)
    at /export/gnu/import/git/elfutils/src/unstrip.c:2337
(gdb)

Comment 12 H.J. Lu 2015-06-30 13:12:09 UTC
(In reply to H.J. Lu from comment #11)
> Remove sanity check leads to
> 
> (gdb) r  debug.so stripped.so -o unstripped.so
> Starting program: /export/home/hjl/bugs/misc/strtab/eu-unstrip debug.so
> stripped.so -o unstripped.so
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/libthread_db.so.1".
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0x000000000040b304 in ebl_strtaboffset (se=0x0)
>     at /export/gnu/import/git/elfutils/libebl/eblstrtab.c:348
> 348	  return se->offset;

Never mind. Pilot error.

Comment 13 Jan Kurik 2015-07-15 13:53:34 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle.
Changing version to '23'.

(As we did not run this process for some time, it could affect also pre-Fedora 23 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23

Comment 14 Mark Wielaard 2015-10-05 11:20:06 UTC
The following patch adds proper support for eu-strip and eu-unstrip to handle merged .shstrtab/.strtab sections:
https://lists.fedorahosted.org/pipermail/elfutils-devel/2015-October/005207.html

Comment 15 Mark Wielaard 2016-01-07 12:40:48 UTC
The patch mentioned in comment #14 is now part of git master and will be in elfutils 0.165 when it will be released in a couple of days.

Comment 16 Fedora Update System 2016-01-18 14:39:27 UTC
elfutils-0.165-2.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-a6cf456e31

Comment 17 Fedora Update System 2016-01-20 03:55:04 UTC
elfutils-0.165-2.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-a6cf456e31

Comment 18 Fedora Update System 2016-01-20 21:51:28 UTC
elfutils-0.165-2.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2016-01-25 11:30:50 UTC
elfutils-0.165-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-35e3877a89

Comment 20 Fedora Update System 2016-01-26 04:23:12 UTC
elfutils-0.165-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-35e3877a89

Comment 21 Fedora Update System 2016-02-05 21:51:09 UTC
elfutils-0.165-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.