Bug 2323513

Summary: gdb-add-index: Failed to find a useable GDB binary now seems to kill rpm builds
Product: [Fedora] Fedora Reporter: Jan Pazdziora <adelton>
Component: gdbAssignee: Keith Seitz <keiths>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: rawhideCC: ahajkova, fweimer, guinevere, jan, keiths, kevinb, mcermak, mjw, mkolar, ptoscano, suraj.ghimire7
Target Milestone: ---Keywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: gdb-15.2-2.fc42 gdb-15.2-2.fc41 gdb-15.2-2.fc40 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-11-07 00:24:25 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jan Pazdziora 2024-11-03 15:30:01 UTC
Building packages now fails in Fedora rawhide in the

find-debuginfo: starting
Extracting debug info from 1 files
gdb-add-index: Failed to find a useable GDB binary
*** ERROR:: GDB exited with exit status 1 during index generation

step.


Reproducible: Always

Steps to Reproduce:

1. rpm -Uvh https://kojipkgs.fedoraproject.org//packages/patch/2.7.6/25.fc41/src/patch-2.7.6-25.fc41.src.rpm
2. dnf builddep -y ~/rpmbuild/SPECS/patch.spec 
3. dnf install -y rpm-build
4. rpmbuild -ba ~/rpmbuild/SPECS/patch.spec

Actual Results:  

+ /usr/bin/find-debuginfo -j8 --strict-build-id -m -i --build-id-seed 2.7.6-25.fc42 --unique-debug-suffix -2.7.6-25.fc42.x86_64 --unique-debug-src-base patch-2.7.6-25.fc42.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /root/rpmbuild/BUILD/patch-2.7.6-build/patch-2.7.6
find-debuginfo: starting
Extracting debug info from 1 files
gdb-add-index: Failed to find a useable GDB binary
*** ERROR:: GDB exited with exit status 1 during index generation
error: Bad exit status from /var/tmp/rpm-tmp.jt7LdT (%install)

RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.jt7LdT (%install)



Expected Results:  


+ /usr/bin/find-debuginfo -j8 --strict-build-id -m -i --build-id-seed 2.7.6-25.fc41 --unique-debug-suffix -2.7.6-25.fc41.x86_64 --unique-debug-src-base patch-2.7.6-25.fc41.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /root/rpmbuild/BUILD/patch-2.7.6-build/patch-2.7.6
find-debuginfo: starting
Extracting debug info from 1 files
gdb-add-index: Failed to find a useable GDB binary
DWARF-compressing 1 files
sepdebugcrcfix: Updated 1 CRC32s, 0 CRC32s did match.
Creating .debug symlinks for symlinks to ELF files
Copying sources found by 'debugedit -l' to /usr/src/debug/patch-2.7.6-25.fc41.x86_64
2046 blocks
find-debuginfo: done

This is the output on Fedora 41.

Comment 1 Keith Seitz 2024-11-03 16:40:33 UTC
I followed the given steps (more or less) on both f40 and f41 and could
not reproduce this:

$ fedpkg clone -a -b f41 patch && cd patch
$ fedpkg srpm
$ sudo dnf builddep -y ./*.src.rpm
$ rpmbuildlocal -ba patch.spec |& tee make.out

From make.out, I see:

+ /usr/bin/find-debuginfo -j4 --strict-build-id -m -i --build-id-seed 2.7.6-25.fc41 --unique-debug-suffix -2.7.6-25.fc41.x86_64 --unique-debug-src-base patch-2.7.6-25.fc41.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /home/fedora41/patch/patch-2.7.6-build/patch-2.7.6
find-debuginfo: starting
Extracting debug info from 1 files
DWARF-compressing 1 files
sepdebugcrcfix: Updated 1 CRC32s, 0 CRC32s did match.
Creating .debug symlinks for symlinks to ELF files
Copying sources found by 'debugedit -l' to /usr/src/debug/patch-2.7.6-25.fc41.x86_64
2046 blocks
find-debuginfo: done

I did this on a brand new install of f41 workstation VM and dnf update. No other changes.

If find-debuginfo.sh is saying gdb is missing, is gdb installed? What version of debugedit
is installed?

Comment 2 Keith Seitz 2024-11-03 17:59:58 UTC
Rawhide ... my bad. Rechecking on rawhide.

Comment 3 Keith Seitz 2024-11-03 18:14:29 UTC
Unfortunately, even on rawhide VM (updated 10h00 PST Nov 3),
this behaves no differently than f40 or f41.

For some reason, find-debuginfo.sh thinks that GDB is
not found. Is `GDB' set in the environment? Is the
gdb-headless package installed? Does /usr/libexec/gdb
exist?

Comment 4 Jan Pazdziora 2024-11-03 20:05:33 UTC
I'm doing my tests using the registry.fedoraproject.org/fedora:rawhide container (that's what I ultimately run upstream tests in in GitHub Actions).

The rpm-build pulls in gdb-minimal-15.2-1.fc42.x86_64, so that one is installed.

Comment 5 Florian Weimer 2024-11-04 06:17:48 UTC
It looks like a missing dependency on “which”, which was likely installed due to other dependencies before. Alternatively, gdb-add-index could replace the use of ”which” with “command -v”.

Comment 6 Kevin Buettner 2024-11-05 05:40:41 UTC
(In reply to Florian Weimer from comment #5)
> It looks like a missing dependency on “which”, which was likely installed
> due to other dependencies before. Alternatively, gdb-add-index could replace
> the use of ”which” with “command -v”.

I worked on a bug earlier this year which was supposed to address the 'which' problem. See: https://bugzilla.redhat.com/show_bug.cgi?id=2275274

gdb.spec now has a "Requires" for which, but its for the headless package.  Perhaps a similar Requires is also needed for the minimal package?

Comment 7 Keith Seitz 2024-11-05 19:47:00 UTC
Jan, if I give you a scratch build RPM of new gdb, are you
able to test to affirm that Florian's hypothesis is correct?

Comment 8 Jan Pazdziora 2024-11-06 05:38:58 UTC
Yes, sure.

Comment 9 Pino Toscano 2024-11-06 10:28:03 UTC
*** Bug 2324145 has been marked as a duplicate of this bug. ***

Comment 10 Pino Toscano 2024-11-06 10:32:00 UTC
(In reply to Kevin Buettner from comment #6)
> gdb.spec now has a "Requires" for which, but its for the headless package. 
> Perhaps a similar Requires is also needed for the minimal package?

Yes, exactly, "gdb-minimal" needs to depend on "which"; see bug 2324145 comment 1 for minimal reproducer instructions using a Fedora Rawhide container.

Also, it would be nice to fix this in all the supported Fedora versions (i.e. 39+), as the dependency is missing also in older versions.

Comment 11 Mark Wielaard 2024-11-06 11:57:03 UTC
This comes from a fedora local patch
https://src.fedoraproject.org/rpms/gdb/blob/rawhide/f/gdb-add-index.patch
The upstream code already uses command -v (which is posix shell) instead of which.

Comment 12 Mark Wielaard 2024-11-06 12:11:47 UTC
https://src.fedoraproject.org/rpms/gdb/pull-request/153
gdb-add-index.patch: Replace which with command -v

Comment 13 Keith Seitz 2024-11-06 17:41:41 UTC
I've reviewed Mark's PR, did a scratch build and followed the directions
in 2324145 to verify:

$ podman run -it --rm --pull=newer fedora:rawhide
[root@acc9e7112292 /]# dnf --setopt install_weak_deps=False install -y debugedit
[root@a4625b100393 /]# gdb-add-index
gdb-add-index: Failed to find a useable GDB binary

[root@a4625b100393 /]# dnf --setopt install_weak_deps=False install -y https://kojipkgs.fedoraproject.org//work/tasks/8835/125558835/gdb-minimal-15.2-2.fc42.x86_64.rpm
[root@a4625b100393 /]# gdb-add-index
usage: gdb-add-index [-dwarf-5] FILE

So AFAICT this should work, and I will do a build immediately.

Comment 14 Fedora Update System 2024-11-06 19:00:45 UTC
FEDORA-2024-90ecac1fea (gdb-15.2-2.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-90ecac1fea

Comment 15 Fedora Update System 2024-11-07 00:24:25 UTC
FEDORA-2024-90ecac1fea (gdb-15.2-2.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 16 Jan Pazdziora 2024-11-07 05:46:58 UTC
Thanks for the errata. Could you do the same for Fedora 41? It seems something changed there as well recently so the same issue is now present in registry.fedoraproject.org/fedora:41 containers.

Comment 17 Fedora Update System 2024-11-07 20:08:57 UTC
FEDORA-2024-1546079ac0 (gdb-15.2-2.fc41) has been submitted as an update to Fedora 41.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-1546079ac0

Comment 18 Fedora Update System 2024-11-07 21:24:55 UTC
FEDORA-2024-cf449a05a4 (gdb-15.2-2.fc40) has been submitted as an update to Fedora 40.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-cf449a05a4

Comment 19 Fedora Update System 2024-11-08 02:00:29 UTC
FEDORA-2024-cf449a05a4 has been pushed to the Fedora 40 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-cf449a05a4`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-cf449a05a4

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 20 Fedora Update System 2024-11-08 02:12:16 UTC
FEDORA-2024-1546079ac0 has been pushed to the Fedora 41 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-1546079ac0`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-1546079ac0

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 21 Fedora Update System 2024-11-09 01:45:51 UTC
FEDORA-2024-1546079ac0 (gdb-15.2-2.fc41) has been pushed to the Fedora 41 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 22 Fedora Update System 2024-11-11 05:09:30 UTC
FEDORA-2024-cf449a05a4 (gdb-15.2-2.fc40) has been pushed to the Fedora 40 stable repository.
If problem still persists, please make note of it in this bug report.