Bug 1716380

Summary: elfutils (eu-strip) segmentation fault in handle_elf (strip.c:1978)
Product: [Fedora] Fedora Reporter: Damian Wrobel <dwrobel>
Component: elfutilsAssignee: Mark Wielaard <mjw>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 30CC: fche, fweimer, jakub, me, mjw
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: elfutils-0.176-3.fc30 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-22 01:02:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1659055    

Description Damian Wrobel 2019-06-03 11:22:05 UTC
Description of problem:
eu-strip crashes while striping debug symbols

$ rpm -qv elfutils gcc binutils
elfutils-0.176-2.fc30.x86_64
gcc-9.1.1-1.fc30.x86_64
binutils-2.31.1-29.fc30.x86_64

How reproducible:
100%

Steps to Reproduce:
Please either recompile the whole kernel available here: https://dwrobel.fedorapeople.org/projects/rpmbuild/SRPMS/kernel-5.0.19-300.rt11.3.fc30.src.rpm
using:
$ mock -r fedora-30-x86_64 --rebuild kernel-5.0.19-300.rt11.3.fc30.src.rpm

or (probably faster) just download https://dwrobel.fedorapeople.org/projects/bugz/eu-strip/qat_c3xxx.ko and execute:
$ eu-strip --remove-comment --reloc-debug-sections -f qat_c3xxx.ko.debug qat_c3xxx.ko

Actual results:

Program received signal SIGSEGV, Segmentation fault.
0x000055555555cb0d in handle_elf (fd=<optimized out>, elf=<optimized out>, fname=<optimized out>, mode=<optimized out>, tvp=<optimized out>, prefix=0x0) at strip.c:1978
1978			    sec = shdr_info[sidx].idx;
Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-29.fc30.x86_64 xz-libs-5.2.4-5.fc30.x86_64 zlib-1.2.11-15.fc30.x86_64
(gdb) bt
#0  0x000055555555cb0d in handle_elf (fd=<optimized out>, elf=<optimized out>, fname=<optimized out>, mode=<optimized out>, tvp=<optimized out>, prefix=0x0) at strip.c:1978
#1  0x000055555555e504 in process_file (fname=0x7fffffffe772 "/home/dwrobel/projects/rpmbuild/BUILDROOT/kernel-5.0.19-300.rt11.3.fc30.x86_64/lib/modules/5.0.19-300.rt11.3.fc30.x86_64+rt/kernel/drivers/crypto/qat/qat_c3xxx/qat_c3xxx.ko") at strip.c:769
#2  0x0000555555558d94 in main (argc=6, argv=0x7fffffffe418) at strip.c:272
(gdb) l
1973			    if (sym->st_shndx == SHN_XINDEX)
1974			      elf_assert (shndxdata != NULL
1975					  && shndxdata->d_buf != NULL);
1976			    size_t sidx = (sym->st_shndx != SHN_XINDEX
1977					   ? sym->st_shndx : xshndx);
1978			    sec = shdr_info[sidx].idx;
1979	
1980			    if (sec != 0)
1981			      {
1982				GElf_Section nshndx;

(gdb) p sidx
$7 = <optimized out>
(gdb) p /x sym->st_shndx
$8 = 0x7833
(gdb) p (sym->st_shndx != 0xffff  ? sym->st_shndx : xshndx)
$9 = 30771
(gdb) p shdr_info[30771]
Cannot access memory at address 0x8000004368c0

(gdb) p shdr_info[62]
$16 = {scn = 0x36333b31303d7870, shdr = {sh_name = 2016291386, sh_type = 1030123635, sh_flags = 4827922805003333936, sh_addr = 3620141271245737039, sh_offset = 4417277267235844401, sh_size = 6004706148239111288, sh_link = 1681740357, sh_info = 1651470967, sh_addralign = 6218424706814536805, sh_entsize = 6865531301700648268}, data = 0x5f454d49544e5552, debug_data = 0x6e75722f3d524944, name = 0x32322f726573752f <error: Cannot access memory at address 0x32322f726573752f>, idx = 1342191157, old_sh_link = 1028150337, symtab_idx = 1920169263, version_idx = 1668246575, group_idx = 1932487777, group_cnt = 980314466, newscn = 0x636f6c2f7273752f, se = 0x2f3a6e69622f6c61, newsymidx = 0x6e6962732f727375}
(gdb) p shdr_info[63]
Cannot access memory at address 0x7ffffffff000
(gdb) 

So it looks that using index bigger than 62 (in particular 30771) generates segfaults.


Expected results:
eu-strip shouldn't segfaults.

Additional info:
The kernel I'm compiling is based on vanilla Fedora kernel with kernel-rt patch with disabled all CONFIG_DEBUG_* options from kernel configuration file:
$ grep -e '^CONFIG_DEBUG' .config  | sed 's/\(.*\)=.*/# \1 is not set/g'
as they are unwanted for the real-time kernel.

It appears that disabling all CONFIG_DEBUG settings triggers eu-strip to start crashing.

Recompiling the kernel --without_debuginfo works without any problem.

Comment 1 Mark Wielaard 2019-06-03 12:44:37 UTC
(In reply to Damian Wrobel from comment #0)
> or (probably faster) just download
> https://dwrobel.fedorapeople.org/projects/bugz/eu-strip/qat_c3xxx.ko and
> execute:
> $ eu-strip --remove-comment --reloc-debug-sections -f qat_c3xxx.ko.debug
> qat_c3xxx.ko

Thanks. I can replicate the crash with that.
It certainly shouldn't crash, so I'll look into that.

But... It does look like the https://dwrobel.fedorapeople.org/projects/bugz/eu-strip/qat_c3xxx.ko is already corrupted.
If you inspect the section headers, even before stripping:

$ eu-readelf -S qat_c3xxx.ko
There are 30 section headers, starting at offset 0x4198:

Section Headers:
[Nr] Name                 Type         Addr             Off      Size     ES Flags Lk Inf Al
[ 0] symtab               NULL         0000000000000000 00000000 00000000  0        0   0  0
[ 1] ote.gnu.build-id     NOTE         0000000000000000 00000040 00000024  0 A      0   0  4
[ 2] ote.Linux            NOTE         0000000000000000 00000064 00000034  0 A      0   0  4
[ 3] ext                  PROGBITS     0000000000000000 000000a0 0000095d  0 AX     0   0 16
[ 4] ela.text             RELA         0000000000000000 00002900 00000c18 24 I     27   3  8
[ 5] ext.unlikely         PROGBITS     0000000000000000 000009fd 0000000d  0 AX     0   0  1
[ 6] ela.text.unlikely    RELA         0000000000000000 00003518 00000030 24 I     27   5  8
[ 7] nit.text             PROGBITS     0000000000000000 00000a0a 00000046  0 AX     0   0  1
[ 8] ela.init.text        RELA         0000000000000000 00003548 000000d8 24 I     27   7  8
[ 9] xit.text             PROGBITS     0000000000000000 00000a50 0000000c  0 AX     0   0  1
[10] ela.exit.text        RELA         0000000000000000 00003620 00000030 24 I     27   9  8
[11] odata                PROGBITS     0000000000000000 00000a60 00000058  0 A      0   0 32
[12] mcount_loc           PROGBITS     0000000000000000 00000ab8 000000a0  0 A      0   0  1
[13] ela__mcount_loc      RELA         0000000000000000 00003650 000001e0 24 I     27  12  8
[14] odata.str1.1         PROGBITS     0000000000000000 00000b58 00000127  1 AMS    0   0  1
[15] odata.str1.8         PROGBITS     0000000000000000 00000c80 000000a0  1 AMS    0   0  8
[16] odinfo               PROGBITS     0000000000000000 00000d20 00000158  0 A      0   0 16
[17] rc_unwind_ip         PROGBITS     0000000000000000 00000e78 00000140  0 A      0   0  1
[18] ela.orc_unwind_ip    RELA         0000000000000000 00003830 00000780 24 I     27  17  8
[19] rc_unwind            PROGBITS     0000000000000000 00000fb8 000001e0  0 A      0   0  1
[20] ata                  PROGBITS     0000000000000000 000011a0 00000140  0 WA     0   0 32
[21] ela.data             RELA         0000000000000000 00003fb0 00000090 24 I     27  20  8
[22] nu.linkonce.this_module PROGBITS     0000000000000000 00001300 00000380  0 WA     0   0 64
[23] ela.gnu.linkonce.this_module RELA         0000000000000000 00004040 00000030 24 I     27  22  8
[24] ss                   NOBITS       0000000000000000 00001680 00000000  0 WA     0   0  1
[25] omment               PROGBITS     0000000000000000 00001680 00000087  1 MS     0   0  1
[26] ote.GNU-stack        PROGBITS     0000000000000000 00001707 00000000  0        0   0  1
[27] ymtab                SYMTAB       0000000000000000 00001708 00000b28 24       28  60  8
[28] trtab                STRTAB       0000000000000000 00002230 000006cf  0        0   0  1
[29] hstrtab              STRTAB       0000000000000000 00004070 00000122  0        0   0  1

There is clearly some off-by-2 going on. All names have their . and first letter chopped off.

This might be some other process adjusting the ELF .ko file earlier.
Which might be rpm debugedit. If so, then this might be:
https://github.com/rpm-software-management/rpm/issues/423

Which turned out to actually also being a elfutils/libelf bug:
https://sourceware.org/ml/elfutils-devel/2019-q2/msg00077.html

I can backport that fix to the fedora (rawhide) package.
Would it be possible to do a rebuild using that?

Comment 2 Damian Wrobel 2019-06-03 13:12:36 UTC
(In reply to Mark Wielaard from comment #1)
> I can backport that fix to the fedora (rawhide) package.
> Would it be possible to do a rebuild using that?

Yes, certainly. It should be just a matter of doing:
$ mock -r fedora-rawhide-x86_64 --rebuild kernel-5.0.19-300.rt11.3.fc30.src.rpm
when the backport will be available in the rawhide.

Comment 3 Mark Wielaard 2019-06-03 17:39:03 UTC
(In reply to Damian Wrobel from comment #2)
> (In reply to Mark Wielaard from comment #1)
> > I can backport that fix to the fedora (rawhide) package.
> > Would it be possible to do a rebuild using that?
> 
> Yes, certainly. It should be just a matter of doing:
> $ mock -r fedora-rawhide-x86_64 --rebuild
> kernel-5.0.19-300.rt11.3.fc30.src.rpm
> when the backport will be available in the rawhide.

Thanks. If you could try with elfutils-0.176-3.fc31 that would be appreciated.
https://koji.fedoraproject.org/koji/buildinfo?buildID=1278740

Comment 4 Damian Wrobel 2019-06-05 07:44:34 UTC
> Thanks. If you could try with elfutils-0.176-3.fc31 that would be
> appreciated.
> https://koji.fedoraproject.org/koji/buildinfo?buildID=1278740

Mark,

I can confirm that 0.176-3 version fixes the issue and I was able to successfully build kernel package with debuginfo.
Thank you for the support.

I think the issue can be closed unless there is a plan to backport the fix to f30 as well.

Comment 5 Mark Wielaard 2019-06-05 08:16:10 UTC
(In reply to Damian Wrobel from comment #4)
> I can confirm that 0.176-3 version fixes the issue and I was able to
> successfully build kernel package with debuginfo.
> Thank you for the support.

Thanks for testing. I am both happy and slightly concerned this fixed the issue. This bug has been in the code for a very long time. It is slightly surprising it has hit multiple times now recently (I also did get a report from a suse and a gentoo user). Maybe in the past people hadn't noticed or didn't report it.

> I think the issue can be closed unless there is a plan to backport the fix
> to f30 as well.

I'll backport it to f30 too.

Comment 6 Fedora Update System 2019-06-05 09:28:52 UTC
FEDORA-2019-3c0523b33c has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-3c0523b33c

Comment 7 Fedora Update System 2019-06-06 01:00:12 UTC
elfutils-0.176-3.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-3c0523b33c

Comment 8 Mark Wielaard 2019-06-14 09:05:14 UTC
*** Bug 1659047 has been marked as a duplicate of this bug. ***

Comment 9 Mark Wielaard 2019-06-20 09:41:12 UTC
(In reply to Mark Wielaard from comment #1)
> (In reply to Damian Wrobel from comment #0)
> > or (probably faster) just download
> > https://dwrobel.fedorapeople.org/projects/bugz/eu-strip/qat_c3xxx.ko and
> > execute:
> > $ eu-strip --remove-comment --reloc-debug-sections -f qat_c3xxx.ko.debug
> > qat_c3xxx.ko
> 
> Thanks. I can replicate the crash with that.
> It certainly shouldn't crash, so I'll look into that.

And I finally did. The crash was fixed upstream by:

commit f03ac75239e0981deaf4aa18f66f423bcc5ce051
Author: Mark Wielaard <mark>
Date:   Wed Mar 27 21:54:06 2019 +0100

    strip: Files with symbols referring to non-existing sections are illformed
    
    The check added in commit 4540ea98c "strip: Fix check test for SHN_XINDEX
    symbol" was not complete. The (extended) section index should also exist.
    If it doesn't exist, mark the file as illformed.
    
    https://sourceware.org/bugzilla/show_bug.cgi?id=24385
    
    Signed-off-by: Mark Wielaard <mark>

Which isn't in Fedora yet. But the underlying bug fix, that created the bad ELF file in the first place now is.

Comment 10 Fedora Update System 2019-06-22 01:02:43 UTC
elfutils-0.176-3.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.