Bug 733690 - Within kernel-debuginfo, .build-id/XX/YYYY.debug does not exist for vmlinux, and abrt cannot download it
Summary: Within kernel-debuginfo, .build-id/XX/YYYY.debug does not exist for vmlinux, ...
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 19
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
Depends On:
TreeView+ depends on / blocked
Reported: 2011-08-26 14:08 UTC by Chris Meek
Modified: 2013-04-05 19:58 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2013-04-05 19:45:27 UTC
Type: ---

Attachments (Terms of Use)

Description Chris Meek 2011-08-26 14:08:56 UTC
Description of problem:
If the kernel-debuginfo is not installed, the 
/usr/lib/debug/.build-id/XX/YYYY.debug file does not exist,
so abrt-action-install-debuginfo-to-abrt-cache does not
know what to download.

Version-Release number of selected component (if applicable):

How reproducible:
Every Time

Steps to Reproduce:
1. If kernel-debuginfo is installed, remove it.

2. Run abrt with the build id for the kernel-debuginfo, e.g:
       $ echo 853bd35b78f8aa03828ddc3f23584e653b900449 | /usr/bin/abrt-action-install-debuginfo-to-abrt-cache -y --ids=-

3. Abrt will not be able to download the debuginfo.
Actual results:
$ echo 853bd35b78f8aa03828ddc3f23584e653b900449 | /usr/bin/abrt-action-install-debuginfo-to-abrt-cache -y --ids=-
Coredump references 1 debuginfo files, 1 of them are not installed
Looking for needed packages in repositories
Can't find packages for 1 debuginfo files
Missing debuginfo file: /usr/lib/debug/.build-id/85/3bd35b78f8aa03828ddc3f23584e653b900449.debug

Expected results:
Should download the debuginfo.

Additional info:

Comment 1 Frank Ch. Eigler 2011-08-26 14:15:20 UTC
The main problem appears to be that, unlike for normal debuginfo packages,
the kernel image file included in your is not split into vmlinux and
vmlinux.debug, but left in its original unstripped form.  That is fine
for many tools, but ABRT assumes that /usr/lib/debug/.build-id/XX/YYY.debug
is the one with dwarf contents.  This is a good assumption for other 
packages, where .../.build-id/XX/YYY  (no .debug suffix) points to the
stripped binaries.

So it would be nice if the kernel debuginfo construction scripts were
changed to conform to abrt's assumptions.  This could be as simple as
adding one hardlink or symlink from vmlinux.debug->vmlinux, or else
at the .../.build-id/XX/YYYY.debug -> .../.build-id/XX/YYYY level.

Comment 2 Dave Jones 2011-08-26 14:31:15 UTC
why does abrt want to download the debuginfo for the kernel anyway ?

We have the symbols for traces in /proc/kallsyms, so oopses are going to get decoded without it.

Additionally, it's enormous. If abrt is going to download this for every oops, that's going to just annoy people enough that they'll just remove abrt.

Comment 3 Frank Ch. Eigler 2011-08-26 14:39:06 UTC
The specific usage scenario here is downloading debuginfo on demand (based
on build-id), not for automated nor kerneloops purposes, but for symbolic
debugging (systemtap/crash/perf).

To enable this, we'd just need the kernel to fit into the normal
.build-id/XX/YYY* pattern in one small extra way.

Comment 4 Dave Jones 2011-08-26 15:21:47 UTC
ah, ok. I see.

Roland's build-id magic in the kernel spec scares the hell out of me.
Got a diff of what needs to change ?

Comment 5 Chuck Ebbert 2011-11-29 13:40:01 UTC
Like this?

--- a/kernel.spec
+++ b/kernel.spec
@@ -1594,6 +1594,9 @@ BuildKernel() {
     mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/lib/modules/$KernelVer
     cp vmlinux $RPM_BUILD_ROOT%{debuginfodir}/lib/modules/$KernelVer
+    cd $RPM_BUILD_ROOT%{debuginfodir}/lib/modules/$KernelVer
+    ln -s vmlinux vmlinux.debug
+    cd -
     find $RPM_BUILD_ROOT/lib/modules/$KernelVer -name "*.ko" -type f >modnames

We could just add that to the spec - at the worst it should be harmless.

Comment 6 Neal Becker 2011-11-29 14:10:18 UTC
Assigned to me by mistake

Comment 7 Frank Ch. Eigler 2011-11-29 16:33:33 UTC
Chuck, if that works (gets the build-id hex link), that'd be great.

Comment 8 Chuck Ebbert 2011-12-07 10:10:12 UTC
(In reply to comment #7)
> Chuck, if that works (gets the build-id hex link), that'd be great.

Sigh. No, it does not work. The symlink is there but the build-id link does not get created.

Comment 9 Chuck Ebbert 2011-12-07 14:54:42 UTC
I tried this, but it also fails. The only way I have found to test this is to build the kernel and debuginfo packages, install them and then do:

 find /usr/lib/debug/.buildid -lname \*vmlinux\* | xargs ls -l

If it's working there should be two entries; right now there is only one.

--- a/kernel.spec
+++ b/kernel.spec
@@ -1641,6 +1641,7 @@ BuildKernel() {
 %if %{with_debuginfo}
     if test -s vmlinux.id; then
       cp vmlinux.id $RPM_BUILD_ROOT/lib/modules/$KernelVer/build/vmlinux.id
+      cp vmlinux.id $RPM_BUILD_ROOT/lib/modules/$KernelVer/build/vmlinux.debug.id
       echo >&2 "*** ERROR *** no vmlinux build ID! ***"
       exit 1
@@ -1651,6 +1652,9 @@ BuildKernel() {
     mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/lib/modules/$KernelVer
     cp vmlinux $RPM_BUILD_ROOT%{debuginfodir}/lib/modules/$KernelVer
+    cd $RPM_BUILD_ROOT%{debuginfodir}/lib/modules/$KernelVer
+    ln -s vmlinux vmlinux.debug
+    cd -
     find $RPM_BUILD_ROOT/lib/modules/$KernelVer -name "*.ko" -type f >modnames

Comment 10 Chuck Ebbert 2011-12-07 14:56:16 UTC
Staring at the specfile gives no insight how this magic works. I started looking at the build logs but finally just gave up.

Comment 13 Fedora End Of Life 2013-04-03 15:37:16 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:

Comment 14 Frank Ch. Eigler 2013-04-05 19:58:17 UTC
This problem appears to be corrected as of Fedora 18:

1) install kernel-debuginfo
2) readelf -n /usr/lib/debug/lib/modules/`uname -r`/vmlinux | grep Build.ID
    Build ID: 0bffac30a92416743310a666c43bb555cfac6601
3) ls -al /usr/lib/debug/.build-id/0b/ffac*
lrwxrwxrwx. 1 root root 47 Apr  5 07:06 /usr/lib/debug/.build-id/0b/ffac30a92416743310a666c43bb555cfac6601 -> ../../lib/modules/3.8.5-201.fc18.x86_64/vmlinux

Note You need to log in before you can comment on or make changes to this bug.