Description of problem: rpminspect debuginfo check failure due to the following message, the below is an example: BAD /usr/lib/udev/stratis-str-cmp in stratisd-3.5.0-1.fc37 on aarch64 contains debugging symbols Anyone Contains: .symtab Version-Release number of selected component (if applicable): The version of rpmbuild used to make this build: https://koji.fedoraproject.org/koji/buildinfo?buildID=2138891 . How reproducible: .symtab does seem to be present in the artifacts in the builds where it is declared to be present. Steps to Reproduce: 1. View stratisd f37 update: https://bodhi.fedoraproject.org/updates/FEDORA-2023-11d44a0dbc 2. Note failing test, follow link: https://bodhi.fedoraproject.org/updates/FEDORA-2023-11d44a0dbc` 3. Choose rpminspect test: https://artifacts.dev.testing-farm.io/d15f5242-9d0e-4b4d-aca4-f834040702e3/ 4. Select debuginfo check and view results. Actual results: Failed to pass rpminspect checks due to presence of .symtab in specified Rust executables. Have verified that .symtab is present in the generated executables. Expected results: Expected that rpmbuild would have removed the .symtab entry along with other debuginfo matter in a later step of the build process, so that rpminspect debuginfo check would not have failed in this way. Additional info: The specified executables are special; they are statically compiled as far as is possible with Rust. See the stratisd spec file: https://src.fedoraproject.org/rpms/stratisd/blob/rawhide/f/stratisd.spec#_100 (and 101).
The ".symtab" file was created for these architectures: aarch64, ppc64le, s390x. (i.e.: not x86_64.)
Looking at some of the build.logs there seem to be various issues when find-debuginfo is invoked. Could you provide a binary from before find-debuginfo is run? That way we can inspect what causes these issues. + /usr/bin/find-debuginfo -j8 --strict-build-id -m -i --build-id-seed 3.5.0-1.fc37 --unique-debug-suffix -3.5.0-1.fc37.ppc64le --unique-debug-src-base stratisd-3.5.0-1.fc37.ppc64le --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 50000000 -S debugsourcefiles.list /builddir/build/BUILD/stratisd-3.5.0 extracting debug info from /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/bin/stratis-predict-usage extracting debug info from /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/lib/udev/stratis-str-cmp extracting debug info from /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/lib/udev/stratis-base32-decode extracting debug info from /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/bin/stratis-min extracting debug info from /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/libexec/stratisd-min extracting debug info from /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/libexec/stratisd warning: Unsupported auto-load script at offset 0 in section .debug_gdb_scripts of file /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/lib/udev/stratis-str-cmp. Use `info auto-load python-scripts [REGEXP]' to list them. warning: Unsupported auto-load script at offset 0 in section .debug_gdb_scripts of file /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/lib/udev/stratis-base32-decode. Use `info auto-load python-scripts [REGEXP]' to list them. nm: /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/lib/udev/stratis-str-cmp: no symbols nm: /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/lib/debug/usr/lib/udev/stratis-str-cmp-3.5.0-1.fc37.ppc64le.debug: no symbols xz: /tmp/tmp.AgdeaM53XL: No such file or directory objcopy: cannot open: /tmp/tmp.AgdeaM53XL.xz: No such file or directory nm: /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/lib/udev/stratis-base32-decode: no symbols nm: /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/lib/debug/usr/lib/udev/stratis-base32-decode-3.5.0-1.fc37.ppc64le.debug: no symbols xz: /tmp/tmp.Q3FJBghs18: No such file or directory objcopy: cannot open: /tmp/tmp.Q3FJBghs18.xz: No such file or directory warning: Unsupported auto-load script at offset 0 in section .debug_gdb_scripts of file /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/bin/stratis-predict-usage. Use `info auto-load python-scripts [REGEXP]' to list them. warning: Unsupported auto-load script at offset 0 in section .debug_gdb_scripts of file /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/libexec/stratisd-min. Use `info auto-load python-scripts [REGEXP]' to list them. warning: Unsupported auto-load script at offset 0 in section .debug_gdb_scripts of file /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/bin/stratis-min. Use `info auto-load python-scripts [REGEXP]' to list them. warning: Unsupported auto-load script at offset 0 in section .debug_gdb_scripts of file /builddir/build/BUILDROOT/stratisd-3.5.0-1.fc37.ppc64le/usr/libexec/stratisd. Use `info auto-load python-scripts [REGEXP]' to list them. It looks like you embed some gdb script in the binaries? Apparently gdb-add-index complains about that. Then nm is unable to find the symtab and/or dynsym symbols (maybe you don't have any because you don't have any caused by the special cargo flags?) That makes it impossible to generate the .gnu_debugdata section. Although why that leaves .symtab around is a bit of a mystery. Lets start by capturing one of these binaries, say stratis-str-cmp on ppc64le, before find-debuginfo tries to mangle it to see what it really looks like.
> It looks like you embed some gdb script in the binaries? > Apparently gdb-add-index complains about that. It has always complained about Rust's .debug_gdb_scripts, but AFAIK they do still work as intended, so I don't think that's the problem here. > Then nm is unable to find the symtab and/or dynsym symbols (maybe you don't have any because you don't have any caused by the special cargo flags?) I think that's specifically dynsym, since these are static binaries. > That makes it impossible to generate the .gnu_debugdata section. > > Although why that leaves .symtab around is a bit of a mystery. It *does* generate .gnu_debugdata though, and looking at "eu-readelf --elf-section" it seems correct, AFAICT. > Lets start by capturing one of these binaries, say stratis-str-cmp on ppc64le, before find-debuginfo tries to mangle it to see what it really looks like. I have a local mock build for aarch64 -- I'll upload the before and after binaries for you to look at.
Created attachment 1941885 [details] aarch64 stratis-str-cmp binaries
> I think that's specifically dynsym, since these are static binaries. Just to clarify -- we build Rust executables with static linking to Rust libraries, but usually still dynamically linking for FFI, especially libc etc. In this particular case though, they're using the "crt-static" feature to get a completely static executable.
So most of the errors/warnings are indeed related to not having a .dynsym section, the find-debuginfo script doesn't handle that very nicely. I would suggest setting %undefine _include_minidebuginfo in the spec file. But that doesn't seem to be the reason the symtab section isn't (re)moved. The reason is that there is a [4] .rela.plt section (for the [21] .got section) that references the [39] .symtab section. Since eu-strip believes the .symtab is used, it won't remove it. Section Headers: [Nr] Name Type Addr Off Size ES Flags Lk Inf Al [ 0] NULL 0000000000000000 00000000 00000000 0 0 0 0 [...] [ 4] .rela.plt RELA 0000000000400308 00000308 000000a8 24 AI 39 21 8 [...] [21] .got PROGBITS 000000000050fab8 000ffab8 00000548 8 WA 0 0 8 [...] [39] .symtab SYMTAB 0000000000000000 00160330 000375c0 24 40 7798 8 [40] .strtab STRTAB 0000000000000000 001978f0 0002b719 0 0 0 1 [41] .shstrtab STRTAB 0000000000000000 001c3009 000001eb 0 0 0 1 "Normally" .rela.plt would refer the the .dynsym section. Since these relocations don't seem to actually refer to a symbol I believe the symbol table (sh_link) reference can be zero. I think the linker should do this (setting sh_link to zero) in this case (a static binary without .dynsym) for .rela.plt.
So what is the status here? Have you tried using %undefine _include_minidebuginfo in your spec file? Which linker are you using? You'll have to figure out why it is setting the sh_link reference on the .rela.plt entry to point at the .symtab even though it isn't actually referencing any symbols from that table.
* Have you tried using %undefine _include_minidebuginfo in your spec file? No. I don't know what the consequences will be. It will affect all the executables, not just these ones. Here is the one relevant quotation I found in https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/rawhide/f/buildflags.md. "By default, a compressed symbol table is preserved in the .gnu_debugdata section. To disable that, undefine _include_minidebuginfo." Is that a good thing to do for all our executables? If so, we will try it. Or would you like instead to receive a report of how attempting that change affect the debuginfo results for the two executable in question? If so, we can try that out and send you the results. * Which linker are you using? jistone can give you better information than I can. It is a decision shared by cargo (the Rust build system) and the Fedora (and RHEL) packaging tools, since cargo allows the linker to be configured. * You'll have to figure out why it is setting the sh_link reference on the .rela.plt entry to point at the .symtab even though it isn't actually referencing any symbols from that table. That is not something I can do on my own with out considerable preparation and possibly guidance as well.
(In reply to mulhern from comment #8) > * Have you tried using %undefine _include_minidebuginfo in your spec file? > > No. I don't know what the consequences will be. It will affect all the > executables, not just these ones. Here is the one relevant quotation I found > in > https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/rawhide/f/ > buildflags.md. > > "By default, a compressed symbol table is preserved in the .gnu_debugdata > section. To disable that, undefine _include_minidebuginfo." > > Is that a good thing to do for all our executables? If so, we will try it. > Or would you like instead to receive a report of how attempting that change > affect the debuginfo results for the two executable in question? If so, we > can try that out and send you the results. Trying to generate minidebuginfo causes a lot of noise in the build logs and it clearly doesn't work for these executables. So I suggest disabling it for testing things more. That way we can see if it has any (other) unintended consequences. > * Which linker are you using? > > jistone can give you better information than I can. It is a decision shared > by cargo (the Rust build system) and the Fedora (and RHEL) packaging tools, > since cargo allows the linker to be configured. > > * You'll have to figure out why it is setting the sh_link reference on the > .rela.plt entry to point at the .symtab even though it isn't actually > referencing any symbols from that table. > > That is not something I can do on my own with out considerable preparation > and possibly guidance as well. OK, lets hope someone (jistone?) knows how the toolchain you are using works so we can track down which linker is generating the these executables.
(In reply to mulhern from comment #8) > * Which linker are you using? > > jistone can give you better information than I can. It is a decision shared > by cargo (the Rust build system) and the Fedora (and RHEL) packaging tools, > since cargo allows the linker to be configured. By default, rustc links with "cc", something like this: "cc" "-m64" [local-objects] "-Wl,--as-needed" "-Wl,-Bstatic" [dependency-rlibs] \ "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" \ "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow" \ "-nodefaultlibs" "-o" [output-file] Fedora's rpm macros add a few link args via RUSTFLAGS, which will be appended: -Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-package-notes It doesn't look like stratisd is changing any of these settings further.
(In reply to Josh Stone from comment #10) > (In reply to mulhern from comment #8) > > * Which linker are you using? > > > > jistone can give you better information than I can. It is a decision shared > > by cargo (the Rust build system) and the Fedora (and RHEL) packaging tools, > > since cargo allows the linker to be configured. > > By default, rustc links with "cc", something like this: > > "cc" "-m64" [local-objects] "-Wl,--as-needed" "-Wl,-Bstatic" > [dependency-rlibs] \ > "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" \ > "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-Wl,--gc-sections" "-pie" > "-Wl,-zrelro,-znow" \ > "-nodefaultlibs" "-o" [output-file] > > Fedora's rpm macros add a few link args via RUSTFLAGS, which will be > appended: > -Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-package-notes > > It doesn't look like stratisd is changing any of these settings further. OK, so cc is gcc I assume. Lets reassign it to gcc then. They can figure out which linker is being used in this configuration and see why that generates the spurious sh_link entry. gcc team, see comment #6 for more background.
If cc is gcc and -fuse-ld= option isn't used, then it would invoke /usr/bin/ld but what linker it is depends on what user set using alternatives. I guess you can always strace to see what exactly is used.
Trying to compile/link int main () {} with gcc -static-pie -fpie (using ld.bfd) results at least on x86-64 in .rela.plt section refering to .dynsym which contains a single 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND symbol.
And on ppc64le fails to link: gcc -fpie -static-pie -o 1 1.c /usr/bin/ld: cannot find rcrt1.o: No such file or directory collect2: error: ld returned 1 exit status
I made a scratch build of the Rawhide version of the stratisd package with the following change: diff --git a/stratisd.spec b/stratisd.spec index 980f7e4..1c595b5 100644 --- a/stratisd.spec +++ b/stratisd.spec @@ -1,11 +1,12 @@ %bcond_without check +%undefine _include_minidebuginfo %global udevdir %(pkg-config --variable=udevdir udev) %global dracutdir %(pkg-config --variable=dracutdir dracut) Name: stratisd Version: 3.5.2 -Release: 2%{?dist} +Release: 3%{?dist} Summary: Daemon that manages block devices to create filesystems # ASL 2.0 @@ -174,6 +175,9 @@ a2x -f manpage docs/stratis-dumpmetadata.txt %{_mandir}/man8/stratis-dumpmetadata.8* %changelog +* Mon Mar 27 2023 Bryan Gurney <bgurney> - 3.5.2-3 +- Undefine _include_minidebuginfo + * Fri Mar 17 2023 Bryan Gurney <bgurney> - 3.5.2-2 - Add BuildRequires for device-mapper-devel ...available at https://koji.fedoraproject.org/koji/taskinfo?taskID=99207945
Well, no nm or objdump or xz errors, just warning: Unsupported auto-load script at offset 0 in section .debug_gdb_scripts of file /builddir/build/BUILDROOT/stratisd-3.5.2-3.fc39.ppc64le/usr/lib/udev/stratis-str-cmp. Use `info auto-load python-scripts [REGEXP]' to list them. for each executable on every architecture. I would consider that an improvement. We can try running the debuginfo test on the statically compiled executables, and see if it turns up anything better than last time.
I still the same failure debuginfo failure. If I have rpminspect running in a directory without the stratisd rpminspect.yaml, or with it, with this change: diff --git a/rpminspect.yaml b/rpminspect.yaml index 27203e6..900e99a 100644 --- a/rpminspect.yaml +++ b/rpminspect.yaml @@ -16,8 +16,8 @@ debuginfo: # rpminspect error: "Contains .symtab" # ignoring because this appears to be a bug in rpmbuild or further down the toolchain # https://bugzilla.redhat.com/show_bug.cgi?id=2166149 - ignore: - - /usr/lib/udev/stratis* + # ignore: + # - /usr/lib/udev/stratis* annocheck: # annocheck error: ...I still see these failures (even with the build with "%undefine _include_minidebuginfo"): debuginfo: ---------- 1) /usr/lib/udev/stratis-base32-decode in stratisd-3.5.2-3.fc39 on aarch64 contains debugging symbols Result: BAD Waiver Authorization: Anyone Details: Contains: .symtab 2) /usr/lib/udev/stratis-str-cmp in stratisd-3.5.2-3.fc39 on aarch64 contains debugging symbols Result: BAD Waiver Authorization: Anyone Details: Contains: .symtab 3) /usr/lib/udev/stratis-base32-decode in stratisd-3.5.2-3.fc39 on ppc64le contains debugging symbols Result: BAD Waiver Authorization: Anyone Details: Contains: .symtab 4) /usr/lib/udev/stratis-str-cmp in stratisd-3.5.2-3.fc39 on ppc64le contains debugging symbols Result: BAD Waiver Authorization: Anyone Details: Contains: .symtab 5) /usr/lib/udev/stratis-base32-decode in stratisd-3.5.2-3.fc39 on s390x contains debugging symbols Result: BAD Waiver Authorization: Anyone Details: Contains: .symtab 6) /usr/lib/udev/stratis-str-cmp in stratisd-3.5.2-3.fc39 on s390x contains debugging symbols Result: BAD Waiver Authorization: Anyone Details: Contains: .symtab
So the minidebuginfo support does generate lots of warnings, but doesn't seem to actually break things. The warning: Unsupported auto-load script must come from the use of -i, which calls gdb-add-index. (which is arguably a gdb bug, it is called with set auto-load no) So the issue really just is aarch64, ppc64le and s390x keeping the .symtab. On x86_64 that doesn't happen, but as jakub points out in that case there is a single symbol .dynsym which the .rela.plt can refer too. What we have to figure out is why binutils ld doesn't do the same on those 3 architectures.
(In reply to Mark Wielaard from comment #18) > What we have to figure out is why binutils ld doesn't do the same on those 3 > architectures. Is there any chance that this problem could be filed as an upstream GNU Binutils bug report ? I suspect that people like Alan Modra or H.J. will be able to provide an explanation and possible fix the behaviour if it really is a bug.
Right - I understand the issue now. But I have not yet tracked down the cause. Investigating...
This message is a reminder that Fedora Linux 37 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 37 on 2023-12-05. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '37'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see it. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 37 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
Fedora Linux 37 entered end-of-life (EOL) status on 2023-12-05. Fedora Linux 37 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora Linux please feel free to reopen this bug against that version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see the version field. If you are unable to reopen this bug, please file a new report against an active release. Thank you for reporting this bug and we are sorry it could not be fixed.