Bug 2224555

Summary: annocheck reports missing build notes in nbdkit even though it is built corectly
Product: Red Hat Enterprise Linux 9 Reporter: Richard W.M. Jones <rjones>
Component: annobinAssignee: Nick Clifton <nickc>
annobin sub component: system-version QA Contact: qe-baseos-tools-bugs
Status: CLOSED NOTABUG Docs Contact:
Severity: unspecified    
Priority: unspecified CC: chhu, fweimer, hongzliu, juzhou, lersek, mpolacek, mxie, tyan, tzheng, vwu, xiaodwan
Version: 9.3Keywords: Bugfix, Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-07-24 17:08:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Richard W.M. Jones 2023-07-21 12:51:34 UTC 
Description of problem:

(From Florian ...)

I think it can't find the debuginfo for some reason.  Before debuginfo
stripping, the build is clean:

# annocheck /builddir/build/BUILD/nbdkit-1.34.1/plugins/null/.libs/nbdkit-null-plugin.so
annocheck: Version 12.12.
Hardened: nbdkit-null-plugin.so: PASS.

But the stripped build in BUILDROOT is not:

# annocheck /builddir/build/BUILDROOT/nbdkit-1.34.1-1.el9.x86_64/usr/lib64/nbdkit/plugins/nbdkit-null-plugin.so
annocheck: Version 12.12.
Hardened: nbdkit-null-plugin.so: MAYB: test: notes, reason: notes not found and no DWARF info found (could there be a separate debuginfo file ?)
Hardened: nbdkit-null-plugin.so: MAYB: test: optimization, reason: could not determine how the code was created
Hardened: nbdkit-null-plugin.so: MAYB: test: pic, reason: no valid notes found regarding this test
Hardened: nbdkit-null-plugin.so: MAYB: test: stack-clash, reason: could not determine how the code was created
Hardened: nbdkit-null-plugin.so: MAYB: test: stack-prot, reason: could not determine how the code was created
Hardened: Rerun annocheck with --verbose to see more information on the tests.
Hardened: nbdkit-null-plugin.so: Overall: FAIL (due to MAYB results).

This is sort of expected because the debuginfo isn't installed
system-wide.  The error goes away if I supply the correct path:

# annocheck --debug-file=/builddir/build/BUILDROOT/nbdkit-1.34.1-1.el9.x86_64/usr/lib/debug/usr/lib64/nbdkit/plugins/nbdkit-null-plugin.so-1.34.1-1.el9.x86_64.debug /builddir/build/BUILDROOT/nbdkit-1.34.1-1.el9.x86_64/usr/lib64/nbdkit/plugins/nbdkit-null-plugin.so
annocheck: Version 12.12.
Hardened: nbdkit-null-plugin.so: PASS.

But if I install the RPMs:

nbdkit-server-1.34.1-1.el9.x86_64
nbdkit-nbd-plugin-1.34.1-1.el9.x86_64
nbdkit-debuginfo-1.34.1-1.el9.x86_64
nbdkit-nbd-plugin-debuginfo-1.34.1-1.el9.x86_64

it still fails:

# annocheck /usr/lib64/nbdkit/plugins/nbdkit-null-plugin.so
annocheck: Version 12.12.
Hardened: nbdkit-null-plugin.so: MAYB: test: notes, reason: notes not found and no DWARF info found (could there be a separate debuginfo file ?)
Hardened: nbdkit-null-plugin.so: MAYB: test: optimization, reason: could not determine how the code was created
Hardened: nbdkit-null-plugin.so: MAYB: test: pic, reason: no valid notes found regarding this test
Hardened: nbdkit-null-plugin.so: MAYB: test: stack-clash, reason: could not determine how the code was created
Hardened: nbdkit-null-plugin.so: MAYB: test: stack-prot, reason: could not determine how the code was created
Hardened: Rerun annocheck with --verbose to see more information on the tests.
Hardened: nbdkit-null-plugin.so: Overall: FAIL (due to MAYB results).

Looking at the verbose output:

Hardened: nbdkit-null-plugin.so: build_id_len: 20, name: d7d16e8ead09960e637e87e48eeea95d258106.
Hardened: nbdkit-null-plugin.so:  try: /usr/lib/debug/.build-id/05/d7d16e8ead09960e637e87e48eeea95d258106.debug.
Hardened: nbdkit-null-plugin.so: Could not find separate debuginfo file based on build-id.

That looks like a BFD or annocheck bug, given that the file is there:

# file -L  /usr/lib/.build-id/05/d7d16e8ead09960e637e87e48eeea95d258106 
/usr/lib/.build-id/05/d7d16e8ead09960e637e87e48eeea95d258106: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=05d7d16e8ead09960e637e87e48eeea95d258106, stripped


Version-Release number of selected component (if applicable):

annobin-annocheck-12.12-1.el9.x86_64

How reproducible:

100%

Steps to Reproduce:

See above, but another way is:

$ brew download-build nbdkit-1.34.1-1.el9 --arch=x86_64 --arch=debug          
$ annocheck -v --skip-cf-protection --skip-glibcxx-assertions --skip-glibcxx-assertions --skip-stack-realign --section-size=.gnu.build.attributes --ignore-gaps nbdkit-server-1.34.1-1.el9.x86_64.rpm --debug-rpm=nbdkit-debuginfo-1.34.1-1.el9.x86_64.rpm                            

Additional info:

https://lists.corp.redhat.com/archives/rhel-devel/2023-July/005569.html
Comment 1 Marek Polacek 2023-07-24 15:41:24 UTC 
Fixing Assignee.
Comment 2 Nick Clifton 2023-07-24 16:47:32 UTC 
(In reply to Richard W.M. Jones from comment #0)
Hi Richard,

> But if I install the RPMs:
> 
> nbdkit-nbd-plugin-1.34.1-1.el9.x86_64
> nbdkit-nbd-plugin-debuginfo-1.34.1-1.el9.x86_64

> Looking at the verbose output:
> 
> Hardened: nbdkit-null-plugin.so: build_id_len: 20, name:
> d7d16e8ead09960e637e87e48eeea95d258106.
> Hardened: nbdkit-null-plugin.so:  try:
> /usr/lib/debug/.build-id/05/d7d16e8ead09960e637e87e48eeea95d258106.debug.
> Hardened: nbdkit-null-plugin.so: Could not find separate debuginfo file
> based on build-id.

Are you sure that the installed rpms are nbdkit-nbd-plugin-1.34.1-1.el9.x86_64 and nbdkit-nbd-plugin-debuginfo-1.34.1-1.el9.x86_64 ?

I ask because when I download the debuginfo rpm from brew and look at its contents:

  $ rpm -q -l -l nbdkit-nbd-plugin-debuginfo-1.34.1-1.el9.x86_64.rpm 
  /usr/lib/debug
  /usr/lib/debug/.build-id
  /usr/lib/debug/.build-id/58
  /usr/lib/debug/.build-id/58/52a3cd64671beb1d470e6c23f1d8820b83256b
  /usr/lib/debug/.build-id/58/52a3cd64671beb1d470e6c23f1d8820b83256b.debug
  /usr/lib/debug/usr
  /usr/lib/debug/usr/lib64
  /usr/lib/debug/usr/lib64/nbdkit
  /usr/lib/debug/usr/lib64/nbdkit/plugins
  /usr/lib/debug/usr/lib64/nbdkit/plugins/nbdkit-nbd-plugin.so-1.34.1-1.el9.x86_64.debug

Note how the build-id is completely different.
Comment 3 Richard W.M. Jones 2023-07-24 17:08:34 UTC 
Without the -nbd- in the middle.

However I have now realised what the problem is, and it's not annocheck at all.
The problem is we didn't have the right debuginfo packages installed.

nbdkit is a meta-package that pulls in nbdkit-server, nbdkit-basic-plugins
and nbdkit-basic-filters.  Therefore nbdkit-debuginfo is essentially empty:

$ rpm -ql nbdkit-debuginfo 
/usr/lib/debug
/usr/lib/debug/.dwz
/usr/lib/debug/.dwz/nbdkit-1.34.1-1.el9.x86_64

The real package containing symbols is nbdkit-server-debuginfo:

$ rpm -ql nbdkit-server-debuginfo
/usr/lib/debug
/usr/lib/debug/.build-id
/usr/lib/debug/.build-id/1c
/usr/lib/debug/.build-id/1c/2263f9378081bbc4db3ebd289087995584d2eb
/usr/lib/debug/.build-id/1c/2263f9378081bbc4db3ebd289087995584d2eb.debug
/usr/lib/debug/.build-id/5f
/usr/lib/debug/.build-id/5f/10af993a94d9c07796a8845360e76b745a5418
/usr/lib/debug/.build-id/5f/10af993a94d9c07796a8845360e76b745a5418.debug
/usr/lib/debug/usr
/usr/lib/debug/usr/lib64
/usr/lib/debug/usr/lib64/nbdkit
/usr/lib/debug/usr/lib64/nbdkit/plugins
/usr/lib/debug/usr/lib64/nbdkit/plugins/nbdkit-null-plugin.so-1.34.1-1.el9.x86_64.debug
/usr/lib/debug/usr/sbin
/usr/lib/debug/usr/sbin/nbdkit-1.34.1-1.el9.x86_64.debug

And with that installed, annocheck passes fine.

=> NOTABUG!
Comment 4 Nick Clifton 2023-07-25 08:06:57 UTC 
(In reply to Richard W.M. Jones from comment #3)

> nbdkit is a meta-package that pulls in nbdkit-server, nbdkit-basic-plugins
> and nbdkit-basic-filters.  Therefore nbdkit-debuginfo is essentially empty:

Just a thought - would it be possible to do a similar trick with the ndbkit-debuginfo
rpm ?  Ie have it pull in the ndbkit-server-debuginfo, ndbkit-basic-plugins-debuginfo
and ndbkit-basic-filters-debuginfo rpms ?


> And with that installed, annocheck passes fine.
> => NOTABUG!

Now that is my kind of bug report ! :-)
Comment 5 Richard W.M. Jones 2023-07-25 08:14:23 UTC 
(In reply to Nick Clifton from comment #4)
> (In reply to Richard W.M. Jones from comment #3)
> 
> > nbdkit is a meta-package that pulls in nbdkit-server, nbdkit-basic-plugins
> > and nbdkit-basic-filters.  Therefore nbdkit-debuginfo is essentially empty:
> 
> Just a thought - would it be possible to do a similar trick with the
> ndbkit-debuginfo
> rpm ?  Ie have it pull in the ndbkit-server-debuginfo,
> ndbkit-basic-plugins-debuginfo
> and ndbkit-basic-filters-debuginfo rpms ?

debuginfo packages are generated by deeply complex RPM macros so I guess not.

I think what really matters is that dnf debuginfo-install /usr/sbin/nbdkit
pulls in the correct debuginfo RPMs, which I just checked now and it does.
It doesn't do it based off the binary's build ID, but by taking the package
name (nbdkit-server) and adding -debuginfo to the end.
Comment 6 Florian Weimer 2023-07-25 12:34:05 UTC 
Oh, right, I missed that /usr/lib64/nbdkit/plugins/nbdkit-null-plugin.so is not actually /usr/lib64/nbdkit/plugins/nbdkit-nbd-plugin.so (different path, different RPM packages, so different debuginfo packages as well).