Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: eu-strip crashes while striping debug symbols $ rpm -qv elfutils gcc binutils elfutils-0.176-2.fc30.x86_64 gcc-9.1.1-1.fc30.x86_64 binutils-2.31.1-29.fc30.x86_64 How reproducible: 100% Steps to Reproduce: Please either recompile the whole kernel available here: https://dwrobel.fedorapeople.org/projects/rpmbuild/SRPMS/kernel-5.0.19-300.rt11.3.fc30.src.rpm using: $ mock -r fedora-30-x86_64 --rebuild kernel-5.0.19-300.rt11.3.fc30.src.rpm or (probably faster) just download https://dwrobel.fedorapeople.org/projects/bugz/eu-strip/qat_c3xxx.ko and execute: $ eu-strip --remove-comment --reloc-debug-sections -f qat_c3xxx.ko.debug qat_c3xxx.ko Actual results: Program received signal SIGSEGV, Segmentation fault. 0x000055555555cb0d in handle_elf (fd=<optimized out>, elf=<optimized out>, fname=<optimized out>, mode=<optimized out>, tvp=<optimized out>, prefix=0x0) at strip.c:1978 1978 sec = shdr_info[sidx].idx; Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-29.fc30.x86_64 xz-libs-5.2.4-5.fc30.x86_64 zlib-1.2.11-15.fc30.x86_64 (gdb) bt #0 0x000055555555cb0d in handle_elf (fd=<optimized out>, elf=<optimized out>, fname=<optimized out>, mode=<optimized out>, tvp=<optimized out>, prefix=0x0) at strip.c:1978 #1 0x000055555555e504 in process_file (fname=0x7fffffffe772 "/home/dwrobel/projects/rpmbuild/BUILDROOT/kernel-5.0.19-300.rt11.3.fc30.x86_64/lib/modules/5.0.19-300.rt11.3.fc30.x86_64+rt/kernel/drivers/crypto/qat/qat_c3xxx/qat_c3xxx.ko") at strip.c:769 #2 0x0000555555558d94 in main (argc=6, argv=0x7fffffffe418) at strip.c:272 (gdb) l 1973 if (sym->st_shndx == SHN_XINDEX) 1974 elf_assert (shndxdata != NULL 1975 && shndxdata->d_buf != NULL); 1976 size_t sidx = (sym->st_shndx != SHN_XINDEX 1977 ? sym->st_shndx : xshndx); 1978 sec = shdr_info[sidx].idx; 1979 1980 if (sec != 0) 1981 { 1982 GElf_Section nshndx; (gdb) p sidx $7 = <optimized out> (gdb) p /x sym->st_shndx $8 = 0x7833 (gdb) p (sym->st_shndx != 0xffff ? sym->st_shndx : xshndx) $9 = 30771 (gdb) p shdr_info[30771] Cannot access memory at address 0x8000004368c0 (gdb) p shdr_info[62] $16 = {scn = 0x36333b31303d7870, shdr = {sh_name = 2016291386, sh_type = 1030123635, sh_flags = 4827922805003333936, sh_addr = 3620141271245737039, sh_offset = 4417277267235844401, sh_size = 6004706148239111288, sh_link = 1681740357, sh_info = 1651470967, sh_addralign = 6218424706814536805, sh_entsize = 6865531301700648268}, data = 0x5f454d49544e5552, debug_data = 0x6e75722f3d524944, name = 0x32322f726573752f <error: Cannot access memory at address 0x32322f726573752f>, idx = 1342191157, old_sh_link = 1028150337, symtab_idx = 1920169263, version_idx = 1668246575, group_idx = 1932487777, group_cnt = 980314466, newscn = 0x636f6c2f7273752f, se = 0x2f3a6e69622f6c61, newsymidx = 0x6e6962732f727375} (gdb) p shdr_info[63] Cannot access memory at address 0x7ffffffff000 (gdb) So it looks that using index bigger than 62 (in particular 30771) generates segfaults. Expected results: eu-strip shouldn't segfaults. Additional info: The kernel I'm compiling is based on vanilla Fedora kernel with kernel-rt patch with disabled all CONFIG_DEBUG_* options from kernel configuration file: $ grep -e '^CONFIG_DEBUG' .config | sed 's/\(.*\)=.*/# \1 is not set/g' as they are unwanted for the real-time kernel. It appears that disabling all CONFIG_DEBUG settings triggers eu-strip to start crashing. Recompiling the kernel --without_debuginfo works without any problem.
(In reply to Damian Wrobel from comment #0) > or (probably faster) just download > https://dwrobel.fedorapeople.org/projects/bugz/eu-strip/qat_c3xxx.ko and > execute: > $ eu-strip --remove-comment --reloc-debug-sections -f qat_c3xxx.ko.debug > qat_c3xxx.ko Thanks. I can replicate the crash with that. It certainly shouldn't crash, so I'll look into that. But... It does look like the https://dwrobel.fedorapeople.org/projects/bugz/eu-strip/qat_c3xxx.ko is already corrupted. If you inspect the section headers, even before stripping: $ eu-readelf -S qat_c3xxx.ko There are 30 section headers, starting at offset 0x4198: Section Headers: [Nr] Name Type Addr Off Size ES Flags Lk Inf Al [ 0] symtab NULL 0000000000000000 00000000 00000000 0 0 0 0 [ 1] ote.gnu.build-id NOTE 0000000000000000 00000040 00000024 0 A 0 0 4 [ 2] ote.Linux NOTE 0000000000000000 00000064 00000034 0 A 0 0 4 [ 3] ext PROGBITS 0000000000000000 000000a0 0000095d 0 AX 0 0 16 [ 4] ela.text RELA 0000000000000000 00002900 00000c18 24 I 27 3 8 [ 5] ext.unlikely PROGBITS 0000000000000000 000009fd 0000000d 0 AX 0 0 1 [ 6] ela.text.unlikely RELA 0000000000000000 00003518 00000030 24 I 27 5 8 [ 7] nit.text PROGBITS 0000000000000000 00000a0a 00000046 0 AX 0 0 1 [ 8] ela.init.text RELA 0000000000000000 00003548 000000d8 24 I 27 7 8 [ 9] xit.text PROGBITS 0000000000000000 00000a50 0000000c 0 AX 0 0 1 [10] ela.exit.text RELA 0000000000000000 00003620 00000030 24 I 27 9 8 [11] odata PROGBITS 0000000000000000 00000a60 00000058 0 A 0 0 32 [12] mcount_loc PROGBITS 0000000000000000 00000ab8 000000a0 0 A 0 0 1 [13] ela__mcount_loc RELA 0000000000000000 00003650 000001e0 24 I 27 12 8 [14] odata.str1.1 PROGBITS 0000000000000000 00000b58 00000127 1 AMS 0 0 1 [15] odata.str1.8 PROGBITS 0000000000000000 00000c80 000000a0 1 AMS 0 0 8 [16] odinfo PROGBITS 0000000000000000 00000d20 00000158 0 A 0 0 16 [17] rc_unwind_ip PROGBITS 0000000000000000 00000e78 00000140 0 A 0 0 1 [18] ela.orc_unwind_ip RELA 0000000000000000 00003830 00000780 24 I 27 17 8 [19] rc_unwind PROGBITS 0000000000000000 00000fb8 000001e0 0 A 0 0 1 [20] ata PROGBITS 0000000000000000 000011a0 00000140 0 WA 0 0 32 [21] ela.data RELA 0000000000000000 00003fb0 00000090 24 I 27 20 8 [22] nu.linkonce.this_module PROGBITS 0000000000000000 00001300 00000380 0 WA 0 0 64 [23] ela.gnu.linkonce.this_module RELA 0000000000000000 00004040 00000030 24 I 27 22 8 [24] ss NOBITS 0000000000000000 00001680 00000000 0 WA 0 0 1 [25] omment PROGBITS 0000000000000000 00001680 00000087 1 MS 0 0 1 [26] ote.GNU-stack PROGBITS 0000000000000000 00001707 00000000 0 0 0 1 [27] ymtab SYMTAB 0000000000000000 00001708 00000b28 24 28 60 8 [28] trtab STRTAB 0000000000000000 00002230 000006cf 0 0 0 1 [29] hstrtab STRTAB 0000000000000000 00004070 00000122 0 0 0 1 There is clearly some off-by-2 going on. All names have their . and first letter chopped off. This might be some other process adjusting the ELF .ko file earlier. Which might be rpm debugedit. If so, then this might be: https://github.com/rpm-software-management/rpm/issues/423 Which turned out to actually also being a elfutils/libelf bug: https://sourceware.org/ml/elfutils-devel/2019-q2/msg00077.html I can backport that fix to the fedora (rawhide) package. Would it be possible to do a rebuild using that?
(In reply to Mark Wielaard from comment #1) > I can backport that fix to the fedora (rawhide) package. > Would it be possible to do a rebuild using that? Yes, certainly. It should be just a matter of doing: $ mock -r fedora-rawhide-x86_64 --rebuild kernel-5.0.19-300.rt11.3.fc30.src.rpm when the backport will be available in the rawhide.
(In reply to Damian Wrobel from comment #2) > (In reply to Mark Wielaard from comment #1) > > I can backport that fix to the fedora (rawhide) package. > > Would it be possible to do a rebuild using that? > > Yes, certainly. It should be just a matter of doing: > $ mock -r fedora-rawhide-x86_64 --rebuild > kernel-5.0.19-300.rt11.3.fc30.src.rpm > when the backport will be available in the rawhide. Thanks. If you could try with elfutils-0.176-3.fc31 that would be appreciated. https://koji.fedoraproject.org/koji/buildinfo?buildID=1278740
> Thanks. If you could try with elfutils-0.176-3.fc31 that would be > appreciated. > https://koji.fedoraproject.org/koji/buildinfo?buildID=1278740 Mark, I can confirm that 0.176-3 version fixes the issue and I was able to successfully build kernel package with debuginfo. Thank you for the support. I think the issue can be closed unless there is a plan to backport the fix to f30 as well.
(In reply to Damian Wrobel from comment #4) > I can confirm that 0.176-3 version fixes the issue and I was able to > successfully build kernel package with debuginfo. > Thank you for the support. Thanks for testing. I am both happy and slightly concerned this fixed the issue. This bug has been in the code for a very long time. It is slightly surprising it has hit multiple times now recently (I also did get a report from a suse and a gentoo user). Maybe in the past people hadn't noticed or didn't report it. > I think the issue can be closed unless there is a plan to backport the fix > to f30 as well. I'll backport it to f30 too.
FEDORA-2019-3c0523b33c has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-3c0523b33c
elfutils-0.176-3.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-3c0523b33c
*** Bug 1659047 has been marked as a duplicate of this bug. ***
(In reply to Mark Wielaard from comment #1) > (In reply to Damian Wrobel from comment #0) > > or (probably faster) just download > > https://dwrobel.fedorapeople.org/projects/bugz/eu-strip/qat_c3xxx.ko and > > execute: > > $ eu-strip --remove-comment --reloc-debug-sections -f qat_c3xxx.ko.debug > > qat_c3xxx.ko > > Thanks. I can replicate the crash with that. > It certainly shouldn't crash, so I'll look into that. And I finally did. The crash was fixed upstream by: commit f03ac75239e0981deaf4aa18f66f423bcc5ce051 Author: Mark Wielaard <mark> Date: Wed Mar 27 21:54:06 2019 +0100 strip: Files with symbols referring to non-existing sections are illformed The check added in commit 4540ea98c "strip: Fix check test for SHN_XINDEX symbol" was not complete. The (extended) section index should also exist. If it doesn't exist, mark the file as illformed. https://sourceware.org/bugzilla/show_bug.cgi?id=24385 Signed-off-by: Mark Wielaard <mark> Which isn't in Fedora yet. But the underlying bug fix, that created the bad ELF file in the first place now is.
elfutils-0.176-3.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.