Bug 1650072 - RFE: support compressed elf objects for build-id generation
Summary: RFE: support compressed elf objects for build-id generation
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1650074
TreeView+ depends on / blocked
 
Reported: 2018-11-15 09:59 UTC by Jiri Olsa
Modified: 2023-09-14 04:42 UTC (History)
7 users (show)

Fixed In Version: rpm-4.14.2.1-10.fc31
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1650074 (view as bug list)
Environment:
Last Closed: 2019-06-04 11:13:06 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Support build-id generation from compressed ELF files on elfutils >= 0.175 (1.83 KB, patch)
2019-02-11 11:47 UTC, Panu Matilainen
no flags Details | Diff

Description Jiri Olsa 2018-11-15 09:59:39 UTC
There are no /usr/lib/.build-id /usr/lib/debug/.build-id
links for kernel modules in kernel rpms like there used to be.

The problem seems to be in rpmbuild which is not able to
read compressed kernel module files and wouldn't add their
.build-id links to the rpm.

Solution was discussed with Mark Wielaard to used elfutils 0.175
which supports compressed elf objects via dwelf_elf_begin:

> /* Creates a read-only Elf handle from the given file handle.  The
>    file may be compressed and/or contain a linux kernel image header,
>    in which case it is eagerly decompressed in full and the Elf handle
>    is created as if created with elf_memory ().  On error NULL is
>    returned.  The Elf handle should be closed with elf_end ().  The
>    file handle will not be closed.  Does not return ELF_K_NONE
> handles.  */
> extern Elf *dwelf_elf_begin (int fd);

elfutils 0.175 should be released shortly.

Comment 1 Panu Matilainen 2019-02-11 10:39:45 UTC
Mark, does this mean that supporting those compressed elf objects from rpm a matter of essentially:

--- a/build/files.c
+++ b/build/files.c
@@ -1804,7 +1804,7 @@ static int generateBuildIDs(FileList fl, ARGV_t *files)
                   kernel modules (ET_REL files with names ending in .ko)
                   should have build-ids. */
                GElf_Ehdr ehdr;
-               Elf *elf = elf_begin (fd, ELF_C_READ, NULL);
+               Elf *elf = dwelf_elf_begin (fd);
                if (elf != NULL && elf_kind(elf) == ELF_K_ELF
                    && gelf_getehdr(elf, &ehdr) != NULL
                    && (ehdr.e_type == ET_EXEC || ehdr.e_type == ET_DYN

...or is it more something considerably more involved?

Comment 2 Mark Wielaard 2019-02-11 10:49:18 UTC
(In reply to Panu Matilainen from comment #1)
> Mark, does this mean that supporting those compressed elf objects from rpm a
> matter of essentially:
> 
> --- a/build/files.c
> +++ b/build/files.c
> @@ -1804,7 +1804,7 @@ static int generateBuildIDs(FileList fl, ARGV_t *files)
>                    kernel modules (ET_REL files with names ending in .ko)
>                    should have build-ids. */
>                 GElf_Ehdr ehdr;
> -               Elf *elf = elf_begin (fd, ELF_C_READ, NULL);
> +               Elf *elf = dwelf_elf_begin (fd);
>                 if (elf != NULL && elf_kind(elf) == ELF_K_ELF
>                     && gelf_getehdr(elf, &ehdr) != NULL
>                     && (ehdr.e_type == ET_EXEC || ehdr.e_type == ET_DYN
> 
> ...or is it more something considerably more involved?

Basically, yes for the build-id handling. It obviously needs some testing. Some configure magic, to make sure you have elfutils 0.175 (libdw with dwelf_elf_begin).

There is the interaction with find-debuginfo.sh/debugedit (which don't handle compressed ELF files at the moment). So that you also can extract debuginfo from compressed ELF files (currently the kernel.spec works around this with a trick to call find-debuginfo.sh before compressing, which is a bit of a hack). Technically a separate issue, bug obviously related.

Comment 3 Panu Matilainen 2019-02-11 11:47:08 UTC
Created attachment 1528946 [details]
Support build-id generation from compressed ELF files on elfutils >= 0.175

So something like this? 

Tested to the point that it compiles with elfutils 0.174 and 0.175 and test-suite passes when dwelf_elf_begin() is used. Not too keen to test an actual kernel build on this poor old laptop :)

Comment 4 Mark Wielaard 2019-02-11 12:37:17 UTC
(In reply to Panu Matilainen from comment #3)
> Created attachment 1528946 [details]
> Support build-id generation from compressed ELF files on elfutils >= 0.175
> 
> So something like this? 

Yes, but note the code has this hacky test:

                /* Only real ELF files, that are ET_EXEC, ET_DYN or
                   kernel modules (ET_REL files with names ending in .ko)
                   should have build-ids. */

[...]
                        || (ehdr.e_type == ET_REL
                            && rpmFileHasSuffix (flp->diskPath, ".ko"))

So we also need to change the detection of whether a file is a kernel module.
(Note that we cannot just pick any ET_REL file, because those are normally
object files that still need linking, and won't have a build-id.)

Since kernel modules can be compressed with various variants (.gz, .bz2, .xz) we either recognize them all as .ko.xx or we do something smarter like looking if there is a .modinfo section and we can parse it (like modinfo does).

Comment 5 Panu Matilainen 2019-04-02 13:26:52 UTC
A more complete version with .modinfo detection submitted upstream now: 
https://github.com/rpm-software-management/rpm/pull/653

Comment 6 Panu Matilainen 2019-04-09 09:29:16 UTC
Mark, I'd appreciate if you could take review the patch(es):
https://github.com/rpm-software-management/rpm/pull/653

Comment 7 Mark Wielaard 2019-04-09 22:05:45 UTC
(In reply to Panu Matilainen from comment #6)
> Mark, I'd appreciate if you could take review the patch(es):
> https://github.com/rpm-software-management/rpm/pull/653

I reviewed the patches on rpm-maint. Look good to me. Thanks.

Comment 8 Panu Matilainen 2019-05-21 09:47:31 UTC
This is supposedly fixed in rpm-4.14.2.1-10.fc31 now. Please verify.

Comment 9 Panu Matilainen 2019-06-04 11:13:06 UTC
[pmatilai🎩︎sopuli Downloads]$ rpm -qplv kernel-debuginfo-5.2.0-0.rc3.git0.1.fc31.x86_64.rpm|grep build-id|grep /modules/|head -5
lrwxrwxrwx    1 root     root                      103 Jun  3 16:59 /usr/lib/debug/.build-id/00/20b13ba1b9a1844cf5be28b77692f21bfbce0b -> ../../../../../lib/modules/5.2.0-0.rc3.git0.1.fc31.x86_64/kernel/net/netfilter/ipset/ip_set_hash_mac.ko
lrwxrwxrwx    1 root     root                      123 Jun  3 16:59 /usr/lib/debug/.build-id/00/20b13ba1b9a1844cf5be28b77692f21bfbce0b.debug -> ../../../../../usr/lib/debug/lib/modules/5.2.0-0.rc3.git0.1.fc31.x86_64/kernel/net/netfilter/ipset/ip_set_hash_mac.ko.debug
lrwxrwxrwx    1 root     root                       84 Jun  3 16:59 /usr/lib/debug/.build-id/00/35fe5e6106599c4298683378f932577e28ccad -> ../../../../../lib/modules/5.2.0-0.rc3.git0.1.fc31.x86_64/extra/net/netrom/netrom.ko
lrwxrwxrwx    1 root     root                      104 Jun  3 16:59 /usr/lib/debug/.build-id/00/35fe5e6106599c4298683378f932577e28ccad.debug -> ../../../../../usr/lib/debug/lib/modules/5.2.0-0.rc3.git0.1.fc31.x86_64/extra/net/netrom/netrom.ko.debug
lrwxrwxrwx    1 root     root                       93 Jun  3 16:59 /usr/lib/debug/.build-id/00/60beca4a039e5d9d11aa1f76aa4e232c1cdcb6 -> ../../../../../lib/modules/5.2.0-0.rc3.git0.1.fc31.x86_64/kernel/drivers/gpio/gpio-amd-fch.ko


Also comparing the number of build-ids against an older build prior to the fix shows a big increase:

[pmatilai🎩︎sopuli Downloads]$ rpm -qplv kernel-debuginfo-5.2.0-0.rc3.git0.1.fc31.x86_64.rpm | grep build-id|wc -l
7158
[pmatilai🎩︎sopuli Downloads]$ rpm -qplv kernel-debuginfo-5.2.0-0.rc1.git0.1.fc31.x86_64.rpm |grep build-id|wc -l
57

AFAICS this is working as expected then.

Comment 10 Red Hat Bugzilla 2023-09-14 04:42:25 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days


Note You need to log in before you can comment on or make changes to this bug.