Bug 1961686 - annocheck fails on built rpms
Summary: annocheck fails on built rpms
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: libtool
Version: rawhide
Hardware: x86_64
OS: Unspecified
unspecified
medium
Target Milestone: ---
Assignee: Ondrej Dubaj
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1967565
TreeView+ depends on / blocked
 
Reported: 2021-05-18 13:43 UTC by Fabio Massimo Di Nitto
Modified: 2021-07-20 11:25 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-20 11:25:14 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
build log from "make all -j" (279.42 KB, text/plain)
2021-05-18 15:56 UTC, Fabio Massimo Di Nitto
no flags Details
build log from "make rpm" (538.87 KB, text/plain)
2021-05-18 15:56 UTC, Fabio Massimo Di Nitto
no flags Details
libnozzle rpm (28.79 KB, application/x-rpm)
2021-05-18 16:01 UTC, Fabio Massimo Di Nitto
no flags Details
libnozzle debuginfo rpm (49.49 KB, application/x-rpm)
2021-05-18 16:03 UTC, Fabio Massimo Di Nitto
no flags Details
rawhide build log from "make all" (266.88 KB, text/plain)
2021-05-18 16:10 UTC, Fabio Massimo Di Nitto
no flags Details
rawhide build log from "make rpm" (491.29 KB, text/plain)
2021-05-18 16:10 UTC, Fabio Massimo Di Nitto
no flags Details
rawhide libnozzle rpm (28.91 KB, application/x-rpm)
2021-05-18 16:11 UTC, Fabio Massimo Di Nitto
no flags Details
rawhide libnozzle debuginfo rpm (49.45 KB, application/x-rpm)
2021-05-18 16:11 UTC, Fabio Massimo Di Nitto
no flags Details
libnozzle as requested (250.34 KB, application/x-sharedlib)
2021-05-19 15:33 UTC, Fabio Massimo Di Nitto
no flags Details

Description Fabio Massimo Di Nitto 2021-05-18 13:43:35 UTC
Hi guys,

I am not sure this is a bug on annocheck or something I am doing wrong, or something in rpmbuild.

I have been following https://bugzilla.redhat.com/show_bug.cgi?id=1959843 because I am the upstream maintainer for kronosnet, one of the packages mentioned there and I didn´t want to add any clutter in the other bz.

In the process to understand what was going wrong with knet, I ended up implementing upstream everything that is documented here:

https://developers.redhat.com/blog/2019/02/04/annocheck-examining-the-contents-of-binary-files#the_security_hardening_checker

Tho I am getting some weird (but consistent results) on both fedora 33 and fedora rawhide.

The code in question is available here for you to examine as well:

https://github.com/kronosnet/kronosnet/tree/stack-clash-protection

From the upstream tree:
./autogen.sh
./configure
make all -j

annocheck --verbose libnozzle/.libs/libnozzle.so.1.0.0
annocheck: Version 9.72.
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: pie test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: info: set binary producer to GCC version 11.
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: optimization test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: pic test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: stack-prot test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: cf-protection test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: property-note test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: writeable-got test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: dynamic-segment test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: bind-now test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: info: set binary producer to Gas version 2.
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: info: notes produced by assembler plugin version 1
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: info: set binary producer to Gimple version 9.
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: info: notes produced by lto plugin version 9.72
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: warnings test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: lto test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: stack-clash test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: fortify test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: skip: glibcxx-assertions test because source language not C++
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: gnu-stack test because stack segment exists with the correct permissions
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: gnu-relro test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: notes test because no gaps found
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: skip: branch-protection test because not an AArch64 binary
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: skip: dynamic-tags test because AArch64 specific
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: entry test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: skip: go-revision test because no GO compiled code found
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: skip: only-go test because no GO compiled code found
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: production test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: run-path test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: rwx-seg test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: short-enum test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: skip: stack-realign test because not an x86 executable
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: textrel test
Hardened: libnozzle/.libs/libnozzle.so.1.0.0: PASS: threads test

from the same tree:

make rpm

# annocheck --verbose x86_64/libnozzle1-1.90-1.587.ee9e68.fc35.x86_64.rpm --debug-rpm x86_64/libnozzle1-debuginfo-1.90-1.587.ee9e68.fc35.x86_64.rpm
annocheck: Version 9.72.
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: pie test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: WARN: DW_FORM_GNU_strp_alt not yet handled
Hardened: ./usr/lib64/libnozzle.so.1.0.0: WARN: DW_FORM_GNU_strp_alt not yet handled
Hardened: ./usr/lib64/libnozzle.so.1.0.0: WARN: DW_FORM_GNU_strp_alt not yet handled
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: cf-protection test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: property-note test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: writeable-got test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: dynamic-segment test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: bind-now test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: info: set binary producer to Gas version 2.
Hardened: ./usr/lib64/libnozzle.so.1.0.0: info: notes produced by assembler plugin version 1
Hardened: ./usr/lib64/libnozzle.so.1.0.0: info: set binary producer to Gimple version 9.
Hardened: ./usr/lib64/libnozzle.so.1.0.0: info: notes produced by lto plugin version 9.72
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: stack-prot test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: pic test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: skip: fortify test because LTO compilation discards preprocessor options
Hardened: ./usr/lib64/libnozzle.so.1.0.0: skip: glibcxx-assertions test because source language not C++
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: optimization test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: skip: warnings test because LTO compilation discards preprocessor options
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: lto test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: FAIL: stack-clash test because -fstack-clash-protection not enabled (function: destroy_iface)
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: gnu-stack test because stack segment exists with the correct permissions
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: gnu-relro test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: notes test because no gaps found
Hardened: ./usr/lib64/libnozzle.so.1.0.0: skip: branch-protection test because not an AArch64 binary
Hardened: ./usr/lib64/libnozzle.so.1.0.0: skip: dynamic-tags test because AArch64 specific
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: entry test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: skip: go-revision test because no GO compiled code found
Hardened: ./usr/lib64/libnozzle.so.1.0.0: skip: only-go test because no GO compiled code found
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: production test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: run-path test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: rwx-seg test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: short-enum test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: skip: stack-realign test because not an x86 executable
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: textrel test
Hardened: ./usr/lib64/libnozzle.so.1.0.0: PASS: threads test

At this point I am very confused. The upstream spec file doesn´t mingle with the build options or changes anything in the build system.

You can clearly see that rpmbuild does some magic with CFLAGS/LDFDLAGS and what not, tho it´s not clear to me why the "normally" built binary is ok according to annocheck, but not the one built by rpmbuild.

This is latest rawhide as of today.

As I am trying to add annocheck to upstream and upstream CI, I would really appreciate some help to shed light here.

Contrary to https://bugzilla.redhat.com/show_bug.cgi?id=1959843 I don´t think it´s a libtool issues since I can build upstream just fine and checks are passing.

Please let me know if I can provide any help to debug this further down.

Thanks
Fabio

Comment 1 Nick Clifton 2021-05-18 15:17:02 UTC
(In reply to Fabio Massimo Di Nitto from comment #0)
Hi Fabio,

  Do you have the build log available for the rpm build ?


> ./autogen.sh

  Unfortunately this step fails for me.  I get messages about:

    ./configurelibknet/Makefile.am:90: error: Libtool library used but 'LIBTOOL' is undefined


> At this point I am very confused. The upstream spec file doesn´t mingle with
> the build options or changes anything in the build system.

So - to be clear - building with "make all" produces a binary that passes.  Building with "make rpm" locally produces an rpm that fails, but building an rpm with "fedpkg scratch-build" produces an rpm that passes ?  That is weird.

It sounds like you may have a version of annobin installed locally that is different from the version that is installed in koji's buildroot.  If you check the root.log for a scratch build, you should be able to see which version koji is using.

Hmmm, but you are using the latest version of annobin, so, hmm,  Please could you upload the two rpms (the binary rpm and the debuginfo rpm) so that I can take a closer look ?

Cheers
  Nick

Comment 2 Fabio Massimo Di Nitto 2021-05-18 15:56:02 UTC
Created attachment 1784476 [details]
build log from "make all -j"

this is the upstream build log from ./configure && make all -j

Comment 3 Fabio Massimo Di Nitto 2021-05-18 15:56:36 UTC
Created attachment 1784477 [details]
build log from "make rpm"

Comment 4 Fabio Massimo Di Nitto 2021-05-18 16:00:42 UTC
(In reply to Nick Clifton from comment #1)
> (In reply to Fabio Massimo Di Nitto from comment #0)
> Hi Fabio,
> 
>   Do you have the build log available for the rpm build ?

attached to bugzilla now.

> 
> 
> > ./autogen.sh
> 
>   Unfortunately this step fails for me.  I get messages about:
> 
>     ./configurelibknet/Makefile.am:90: error: Libtool library used but
> 'LIBTOOL' is undefined

You need to install the BuildRequires at least ;) no worries, I can provide all the logs you need.

> 
> 
> > At this point I am very confused. The upstream spec file doesn´t mingle with
> > the build options or changes anything in the build system.
> 
> So - to be clear - building with "make all" produces a binary that passes.

correct
 
> Building with "make rpm" locally produces an rpm that fails,

correct

> but building an
> rpm with "fedpkg scratch-build" produces an rpm that passes ?  That is weird.

I haven´t test this one, but I can check of course.

The make rpm does nothing more than rpmbuild on a locally generated srpm from git. The spec file upstream is pretty much the same as fedora (I maintain both).

> 
> It sounds like you may have a version of annobin installed locally that is
> different from the version that is installed in koji's buildroot.

I have tested with fedora 33 and fedora rawhide (both locally).

It shouldn´t matter what version is in koji tho.

>  If you
> check the root.log for a scratch build, you should be able to see which
> version koji is using.
> 
> Hmmm, but you are using the latest version of annobin, so, hmm,  Please
> could you upload the two rpms (the binary rpm and the debuginfo rpm) so that
> I can take a closer look ?

Yes, I will attach them to BZ in a sec...

Thanks!
Fabio

> 
> Cheers
>   Nick

Comment 5 Fabio Massimo Di Nitto 2021-05-18 16:01:29 UTC
Created attachment 1784478 [details]
libnozzle rpm

Comment 6 Fabio Massimo Di Nitto 2021-05-18 16:03:27 UTC
Created attachment 1784480 [details]
libnozzle debuginfo rpm

Comment 7 Fabio Massimo Di Nitto 2021-05-18 16:04:28 UTC
Just uploaded all the logs and rpm from Fedora 33 build. The behavior is consistent with Fedora rawhide.
I will need a few more minutes to collect the info from rawhide as it´s not local machine.

Comment 8 Fabio Massimo Di Nitto 2021-05-18 16:10:22 UTC
Created attachment 1784481 [details]
rawhide build log from "make all"

Comment 9 Fabio Massimo Di Nitto 2021-05-18 16:10:52 UTC
Created attachment 1784482 [details]
rawhide build log from "make rpm"

Comment 10 Fabio Massimo Di Nitto 2021-05-18 16:11:20 UTC
Created attachment 1784483 [details]
rawhide libnozzle rpm

Comment 11 Fabio Massimo Di Nitto 2021-05-18 16:11:48 UTC
Created attachment 1784484 [details]
rawhide libnozzle debuginfo rpm

Comment 12 Fabio Massimo Di Nitto 2021-05-18 16:27:49 UTC
I also tried a scratch build of the local srpm (with annobin support patches)

$ fedpkg build --target rawhide --scratch --srpm kronosnet-1.90-1.585.29db.fc33.src.rpm
[====================================] 100% 00:00:00 711.64 KiB   1.02 MiB/sec
Building kronosnet-1.90-1.585.29db.fc33.src.rpm for rawhide
Created task: 68212257
Task info: https://koji.fedoraproject.org/koji/taskinfo?taskID=68212257
.....

downloaded the libnozzle rpm and debuginfo and I get the exact same behavior.

Comment 13 Fabio Massimo Di Nitto 2021-05-18 16:29:00 UTC
As extra information, I am using libnozzle as example here, I get similar problems also for all other subpackages.

Comment 14 Nick Clifton 2021-05-18 16:47:08 UTC
Hi Fabio,

  I think that this is the libtool issue again.  In the rpm_build_logs.txt file that you uploaded lines 428 and 429 show that -fstack-clash-protection was passed to libtool, but the command line generated by libtool did not include that option....

Cheers
  Nick

Comment 15 Fabio Massimo Di Nitto 2021-05-18 18:19:23 UTC
(In reply to Nick Clifton from comment #14)
> Hi Fabio,
> 
>   I think that this is the libtool issue again.  In the rpm_build_logs.txt
> file that you uploaded lines 428 and 429 show that -fstack-clash-protection
> was passed to libtool, but the command line generated by libtool did not
> include that option....
> 
> Cheers
>   Nick

What I can´t explain is why it works with the normal build tho:

make[1]: Entering directory '/home/fabbione/work/knet/kronosnet/libnozzle'
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..    -O3 -ggdb3 -Werror -Wall -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3  -I/usr/include/libnl3   -MT libnozzle_la-libnozzle.lo -MD -MP -MF .deps/libnozzle_la-libnozzle.Tpo -c -o libnozzle_la-libnozzle.lo `test -f 'libnozzle.c' || echo './'`libnozzle.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -MT libnozzle_la-libnozzle.lo -MD -MP -MF .deps/libnozzle_la-libnozzle.Tpo -c libnozzle.c  -fPIC -DPIC -o .libs/libnozzle_la-libnozzle.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -MT libnozzle_la-libnozzle.lo -MD -MP -MF .deps/libnozzle_la-libnozzle.Tpo -c libnozzle.c -o libnozzle_la-libnozzle.o >/dev/null 2>&1
mv -f .deps/libnozzle_la-libnozzle.Tpo .deps/libnozzle_la-libnozzle.Plo
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..    -O3 -ggdb3 -Werror -Wall -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3  -I/usr/include/libnl3   -MT libnozzle_la-internals.lo -MD -MP -MF .deps/libnozzle_la-internals.Tpo -c -o libnozzle_la-internals.lo `test -f 'internals.c' || echo './'`internals.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -MT libnozzle_la-internals.lo -MD -MP -MF .deps/libnozzle_la-internals.Tpo -c internals.c  -fPIC -DPIC -o .libs/libnozzle_la-internals.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -MT libnozzle_la-internals.lo -MD -MP -MF .deps/libnozzle_la-internals.Tpo -c internals.c -o libnozzle_la-internals.o >/dev/null 2>&1
mv -f .deps/libnozzle_la-internals.Tpo .deps/libnozzle_la-internals.Plo
/bin/sh ../libtool  --tag=CC   --mode=link gcc -O3 -ggdb3 -Werror -Wall -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3  -I/usr/include/libnl3   -Wl,--enable-new-dtags -Wl,--as-needed  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wl,-version-script,./libnozzle_exported_syms -version-info 1:0:0  -o libnozzle.la -rpath /usr/lib64 libnozzle_la-libnozzle.lo libnozzle_la-internals.lo  -lnl-3  -lnl-route-3 -lnl-3
====
libtool: link: gcc -shared  -fPIC -DPIC  .libs/libnozzle_la-libnozzle.o .libs/libnozzle_la-internals.o   -lnl-route-3 -lnl-3  -O3 -ggdb3 -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -pthread -Wl,--enable-new-dtags -Wl,--as-needed -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -Wl,-version-script -Wl,./libnozzle_exported_syms   -pthread -Wl,-soname -Wl,libnozzle.so.1 -o .libs/libnozzle.so.1.0.0
==== ^^^^^^
libtool: link: (cd ".libs" && rm -f "libnozzle.so.1" && ln -s "libnozzle.so.1.0.0" "libnozzle.so.1")
libtool: link: (cd ".libs" && rm -f "libnozzle.so" && ln -s "libnozzle.so.1.0.0" "libnozzle.so")
libtool: link: ar cru .libs/libnozzle.a  libnozzle_la-libnozzle.o libnozzle_la-internals.o
libtool: link: ranlib .libs/libnozzle.a
libtool: link: ( cd ".libs" && rm -f "libnozzle.la" && ln -s "../libnozzle.la" "libnozzle.la" )
make[1]: Leaving directory '/home/fabbione/work/knet/kronosnet/libnozzle'

even here -fstack-clash-protection is not passed during linking, yet it doesn´t fail the test.

This is the part I really can´t understand.

I fear that those rpmbuild automatic changes to the build system are breaking those flags propagation.

I noticed:
[snip]
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ./configure: line 6764:  -e 's/^T .* \(.*\)$/extern int \1();/p' -e 's/^[ABCDGIRSTW][ABCDGIRSTW]* .* \(.*\)$/extern char \1;/p': No such file or directory
ok
checking for sysroot... no
[snip]

could that affect the linking?

Comment 16 Fabio Massimo Di Nitto 2021-05-19 10:03:11 UTC
Hey Nick,

I have been nailing this issue further down (on Fedora 33) and was able to reproduce the failure also upstream.

From the rpmbuild logs:

CFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection'

I decided to start testing option by option in this list and found that:

CFLAGS="-flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"

is what makes the build upstream fail.

The combination of -flto=auto and sourcing the specs, will cause annocheck to claim that there is a failure:

[fabbione@mazikeen libnozzle]$ env |grep CFLAG
CFLAGS=-flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1

(usual configure build dance goes here)

[fabbione@mazikeen libnozzle]$ make clean && make all -j && annocheck --verbose .libs/libnozzle.so.1.0.0 |grep -i fail

make[1]: Entering directory '/home/fabbione/work/knet/kronosnet/libnozzle'
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..    -O3 -ggdb3 -Werror -Wall -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3  -I/usr/include/libnl3  -flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -MT libnozzle_la-libnozzle.lo -MD -MP -MF .deps/libnozzle_la-libnozzle.Tpo -c -o libnozzle_la-libnozzle.lo `test -f 'libnozzle.c' || echo './'`libnozzle.c
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..    -O3 -ggdb3 -Werror -Wall -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3  -I/usr/include/libnl3  -flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -MT libnozzle_la-internals.lo -MD -MP -MF .deps/libnozzle_la-internals.Tpo -c -o libnozzle_la-internals.lo `test -f 'internals.c' || echo './'`internals.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -MT libnozzle_la-libnozzle.lo -MD -MP -MF .deps/libnozzle_la-libnozzle.Tpo -c libnozzle.c  -fPIC -DPIC -o .libs/libnozzle_la-libnozzle.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -MT libnozzle_la-internals.lo -MD -MP -MF .deps/libnozzle_la-internals.Tpo -c internals.c  -fPIC -DPIC -o .libs/libnozzle_la-internals.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -MT libnozzle_la-internals.lo -MD -MP -MF .deps/libnozzle_la-internals.Tpo -c internals.c -o libnozzle_la-internals.o >/dev/null 2>&1
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -MT libnozzle_la-libnozzle.lo -MD -MP -MF .deps/libnozzle_la-libnozzle.Tpo -c libnozzle.c -o libnozzle_la-libnozzle.o >/dev/null 2>&1
mv -f .deps/libnozzle_la-internals.Tpo .deps/libnozzle_la-internals.Plo
mv -f .deps/libnozzle_la-libnozzle.Tpo .deps/libnozzle_la-libnozzle.Plo
/bin/sh ../libtool  --tag=CC   --mode=link gcc -O3 -ggdb3 -Werror -Wall -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3  -I/usr/include/libnl3  -flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--enable-new-dtags -Wl,--as-needed  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wl,-version-script,./libnozzle_exported_syms -version-info 1:0:0  -o libnozzle.la -rpath /usr/lib64 libnozzle_la-libnozzle.lo libnozzle_la-internals.lo  -lnl-3  -lnl-route-3 -lnl-3
libtool: link: gcc -shared  -fPIC -DPIC  .libs/libnozzle_la-libnozzle.o .libs/libnozzle_la-internals.o   -lnl-route-3 -lnl-3  -O3 -ggdb3 -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -pthread -flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--enable-new-dtags -Wl,--as-needed -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -Wl,-version-script -Wl,./libnozzle_exported_syms   -pthread -Wl,-soname -Wl,libnozzle.so.1 -o .libs/libnozzle.so.1.0.0
libtool: link: (cd ".libs" && rm -f "libnozzle.so.1" && ln -s "libnozzle.so.1.0.0" "libnozzle.so.1")
libtool: link: (cd ".libs" && rm -f "libnozzle.so" && ln -s "libnozzle.so.1.0.0" "libnozzle.so")
libtool: link: ar cru .libs/libnozzle.a  libnozzle_la-libnozzle.o libnozzle_la-internals.o
libtool: link: ranlib .libs/libnozzle.a
libtool: link: ( cd ".libs" && rm -f "libnozzle.la" && ln -s "../libnozzle.la" "libnozzle.la" )
make[1]: Leaving directory '/home/fabbione/work/knet/kronosnet/libnozzle'
Making all in tests
make[1]: Entering directory '/home/fabbione/work/knet/kronosnet/libnozzle/tests'
[SNIP]

Hardened: .libs/libnozzle.so.1.0.0: FAIL: (component: get_iface_mac): Compiled without -fstack-clash-protection.
Hardened: .libs/libnozzle.so.1.0.0: FAIL: Parts of the binary were compiled without stack clash protection.

Dropping either of them "solves" the problem. Swapping them around (ordering) doesn´t solve the problem.

I will keep digging for what I can...

Cheers
Fabio

Comment 17 Nick Clifton 2021-05-19 14:01:44 UTC
(In reply to Fabio Massimo Di Nitto from comment #16)
Hi Fabio,

> I decided to start testing option by option in this list and found that:
> CFLAGS="-flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"
> is what makes the build upstream fail.

Right - because LTO mode involves re-compiling the source code, and this time, because of some issue with libtool, the -fstack-clash-protection option is not passed on to gcc, so annobin records the absence and then when annocheck runs it issues a FAIL result.


> /bin/sh ../libtool  --tag=CC   --mode=link gcc -O3 -ggdb3 -Werror -Wall
> -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong
> -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection
> -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread
> -I/usr/include/libnl3  -I/usr/include/libnl3  -flto=auto
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--enable-new-dtags
> -Wl,--as-needed  -fplugin=annobin -D_FORTIFY_SOURCE=2
> -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now
> -fstack-clash-protection -fcf-protection=full -mstackrealign
> -Wl,-version-script,./libnozzle_exported_syms -version-info 1:0:0  -o
> libnozzle.la -rpath /usr/lib64 libnozzle_la-libnozzle.lo
> libnozzle_la-internals.lo  -lnl-3  -lnl-route-3 -lnl-3

So on this command line the -fstack-clash-proection option is present, but ...


> libtool: link: gcc -shared  -fPIC -DPIC  .libs/libnozzle_la-libnozzle.o
> .libs/libnozzle_la-internals.o   -lnl-route-3 -lnl-3  -O3 -ggdb3
> -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -pthread -flto=auto
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--enable-new-dtags
> -Wl,--as-needed -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign
> -Wl,-version-script -Wl,./libnozzle_exported_syms   -pthread -Wl,-soname
> -Wl,libnozzle.so.1 -o .libs/libnozzle.so.1.0.0

... on this one it is absent.

I think that this problem is the same as:

  https://bugzilla.redhat.com/show_bug.cgi?id=1959843

Cheers
  Nick

Comment 18 Fabio Massimo Di Nitto 2021-05-19 14:20:43 UTC
(In reply to Nick Clifton from comment #17)
> (In reply to Fabio Massimo Di Nitto from comment #16)
> Hi Fabio,
> 
> > I decided to start testing option by option in this list and found that:
> > CFLAGS="-flto=auto -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"
> > is what makes the build upstream fail.
> 
> Right - because LTO mode involves re-compiling the source code, and this
> time, because of some issue with libtool, the -fstack-clash-protection
> option is not passed on to gcc, so annobin records the absence and then when
> annocheck runs it issues a FAIL result.

Ok this is what I don´t understand :)

Does -flto rebuild the whole source code at linking time?

If that´s the case, then yes, you are right, it´s a libtool problem (see below).

PS Sorry for re-iterating so many times. I like to understand the beast here ;)

> 
> > /bin/sh ../libtool  --tag=CC   --mode=link gcc -O3 -ggdb3 -Werror -Wall
> > -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong
> > -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection
> > -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread
> > -I/usr/include/libnl3  -I/usr/include/libnl3  -flto=auto
> > -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--enable-new-dtags
> > -Wl,--as-needed  -fplugin=annobin -D_FORTIFY_SOURCE=2
> > -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now
> > -fstack-clash-protection -fcf-protection=full -mstackrealign
> > -Wl,-version-script,./libnozzle_exported_syms -version-info 1:0:0  -o
> > libnozzle.la -rpath /usr/lib64 libnozzle_la-libnozzle.lo
> > libnozzle_la-internals.lo  -lnl-3  -lnl-route-3 -lnl-3
> 
> So on this command line the -fstack-clash-proection option is present, but
> ...
> 
> 
> > libtool: link: gcc -shared  -fPIC -DPIC  .libs/libnozzle_la-libnozzle.o
> > .libs/libnozzle_la-internals.o   -lnl-route-3 -lnl-3  -O3 -ggdb3
> > -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -pthread -flto=auto
> > -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--enable-new-dtags
> > -Wl,--as-needed -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign
> > -Wl,-version-script -Wl,./libnozzle_exported_syms   -pthread -Wl,-soname
> > -Wl,libnozzle.so.1 -o .libs/libnozzle.so.1.0.0
> 
> ... on this one it is absent.

It is absent also when the test is passing.

> 

Tho here it is when doing a manual invocation of the linking above linking command + -fstack-clash-protection:

[fabbione@mazikeen libnozzle]$ gcc -shared  -fPIC -DPIC  .libs/libnozzle_la-libnozzle.o .libs/libnozzle_la-internals.o -fstack-clash-protection  -lnl-route-3 -lnl-3  -O3 -ggdb3 -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -pthread -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -flto=auto -Wl,--enable-new-d
tags -Wl,--as-needed -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -Wl,-version-script -Wl,./libnozzle_exported_syms   -pthread -Wl,-soname -Wl,libnozzle.so.1 -o .libs/libnozzle.so.1.0.0
[fabbione@mazikeen libnozzle]$ annocheck --verbose .libs/libnozzle.so.1.0.0 |grep -i fail
[fabbione@mazikeen libnozzle]$

Comment 19 Fabio Massimo Di Nitto 2021-05-19 14:42:40 UTC
Sorry Nick, but it still doesn´t make sense to me :) I am hard headed :)

Here is a good build:
make[1]: Entering directory '/home/fabbione/work/knet/kronosnet/libnozzle'
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..    -O3 -ggdb3 -Werror -Wall -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3  -I/usr/include/libnl3  -flto=auto -MT libnozzle_la-libnozzle.lo -MD -MP -MF .deps/libnozzle_la-libnozzle.Tpo -c -o libnozzle_la-libnozzle.lo `test -f 'libnozzle.c' || echo './'`libnozzle.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -flto=auto -MT libnozzle_la-libnozzle.lo -MD -MP -MF .deps/libnozzle_la-libnozzle.Tpo -c libnozzle.c  -fPIC -DPIC -o .libs/libnozzle_la-libnozzle.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -flto=auto -MT libnozzle_la-libnozzle.lo -MD -MP -MF .deps/libnozzle_la-libnozzle.Tpo -c libnozzle.c -o libnozzle_la-libnozzle.o >/dev/null 2>&1
mv -f .deps/libnozzle_la-libnozzle.Tpo .deps/libnozzle_la-libnozzle.Plo
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..    -O3 -ggdb3 -Werror -Wall -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3  -I/usr/include/libnl3  -flto=auto -MT libnozzle_la-internals.lo -MD -MP -MF .deps/libnozzle_la-internals.Tpo -c -o libnozzle_la-internals.lo `test -f 'internals.c' || echo './'`internals.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -flto=auto -MT libnozzle_la-internals.lo -MD -MP -MF .deps/libnozzle_la-internals.Tpo -c internals.c  -fPIC -DPIC -o .libs/libnozzle_la-internals.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -O3 -ggdb3 -Werror -Wall -Wextra -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3 -I/usr/include/libnl3 -flto=auto -MT libnozzle_la-internals.lo -MD -MP -MF .deps/libnozzle_la-internals.Tpo -c internals.c -o libnozzle_la-internals.o >/dev/null 2>&1
mv -f .deps/libnozzle_la-internals.Tpo .deps/libnozzle_la-internals.Plo
/bin/sh ../libtool  --tag=CC   --mode=link gcc -O3 -ggdb3 -Werror -Wall -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3  -I/usr/include/libnl3  -flto=auto -Wl,--enable-new-dtags -Wl,--as-needed  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wl,-version-script,./libnozzle_exported_syms -version-info 1:0:0  -o libnozzle.la -rpath /usr/lib64 libnozzle_la-libnozzle.lo libnozzle_la-internals.lo  -lnl-3  -lnl-route-3 -lnl-3
libtool: link: gcc -shared  -fPIC -DPIC  .libs/libnozzle_la-libnozzle.o .libs/libnozzle_la-internals.o   -lnl-route-3 -lnl-3  -O3 -ggdb3 -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -pthread -flto=auto -Wl,--enable-new-dtags -Wl,--as-needed -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -Wl,-version-script -Wl,./libnozzle_exported_syms   -pthread -Wl,-soname -Wl,libnozzle.so.1 -o .libs/libnozzle.so.1.0.0
libtool: link: (cd ".libs" && rm -f "libnozzle.so.1" && ln -s "libnozzle.so.1.0.0" "libnozzle.so.1")
libtool: link: (cd ".libs" && rm -f "libnozzle.so" && ln -s "libnozzle.so.1.0.0" "libnozzle.so")
libtool: link: ar cru .libs/libnozzle.a  libnozzle_la-libnozzle.o libnozzle_la-internals.o
libtool: link: ranlib .libs/libnozzle.a
libtool: link: ( cd ".libs" && rm -f "libnozzle.la" && ln -s "../libnozzle.la" "libnozzle.la" )
make[1]: Leaving directory '/home/fabbione/work/knet/kronosnet/libnozzle'

[fabbione@mazikeen libnozzle]$ annocheck --verbose .libs/libnozzle.so.1.0.0 |grep -i fail
[fabbione@mazikeen libnozzle]$

as you can see -flto is present, both at build time and linking time. -fstack-clash-protection is present at build time, but not at linking time.

What does -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 add to the build vs what´s already there? afaict the -specs only add -fplugin=annobin that is already present.

Comment 20 Nick Clifton 2021-05-19 15:29:22 UTC
(In reply to Fabio Massimo Di Nitto from comment #18)
Hi Fabio,

> > Right - because LTO mode involves re-compiling the source code, and this

> Does -flto rebuild the whole source code at linking time?

Yes.  LTO = Link Time Optmization = re-compiling the program when *all* of the sources are present.  That way even more optimizations can be made as the whole of the program is available to the compiler.  (Technically it is not the source code that gets recompiled but an intermediate representation of that source code that is more compact.  But you do not need to know the details...)


(In reply to Fabio Massimo Di Nitto from comment #19)

> libtool: link: gcc -shared  -fPIC -DPIC  .libs/libnozzle_la-libnozzle.o
> .libs/libnozzle_la-internals.o   -lnl-route-3 -lnl-3  -O3 -ggdb3
> -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -pthread -flto=auto
> -Wl,--enable-new-dtags -Wl,--as-needed -fstack-protector-strong -Wl,-z
> -Wl,now -mstackrealign -Wl,-version-script -Wl,./libnozzle_exported_syms  
> -pthread -Wl,-soname -Wl,libnozzle.so.1 -o .libs/libnozzle.so.1.0.0

> [fabbione@mazikeen libnozzle]$ annocheck --verbose .libs/libnozzle.so.1.0.0
> |grep -i fail

> as you can see -flto is present, both at build time and linking time.
> -fstack-clash-protection is present at build time, but not at linking time.

OK - that is weird.  My only guess is that for some reason that final link did not run the compiler's LTO recompilation, but just linked together the object files as normal.  So no second recompilation happened and the absence of the -fstack-clash-protection flag had no effect.  If you upload the libnozzle.so.1.0.0 file I might be able to tell you more.



> What does -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 add to the build vs
> what´s already there?

It adds the gcc option to find the plugin directory, and an test to make sure that the user has not requested that the plugin be disabed:

  % cat /usr/lib/rpm/redhat/redhat-annobin-cc1 
*cc1_options:
+ %{!-fno-use-annobin:%{!iplugindir*:%:find-plugindir()} -fplugin=annobin}

Also having this file means that, if needed, extra options can be added to support the plugin in the future, and all without having to change any build systems.

Cheers
  Nick

Comment 21 Fabio Massimo Di Nitto 2021-05-19 15:33:30 UTC
Created attachment 1784862 [details]
libnozzle as requested

As requested.

Comment 22 Fabio Massimo Di Nitto 2021-05-19 15:41:30 UTC
(In reply to Nick Clifton from comment #20)
> (In reply to Fabio Massimo Di Nitto from comment #18)
> Hi Fabio,
> 
> > > Right - because LTO mode involves re-compiling the source code, and this
> 
> > Does -flto rebuild the whole source code at linking time?
> 
> Yes.  LTO = Link Time Optmization = re-compiling the program when *all* of
> the sources are present.  That way even more optimizations can be made as
> the whole of the program is available to the compiler.  (Technically it is
> not the source code that gets recompiled but an intermediate representation
> of that source code that is more compact.  But you do not need to know the
> details...)
> 

thanks for the explanation. It starts to make more sense, and yes I understand the concept of intermediate stage representation :)

> 
> (In reply to Fabio Massimo Di Nitto from comment #19)
> 
> > libtool: link: gcc -shared  -fPIC -DPIC  .libs/libnozzle_la-libnozzle.o
> > .libs/libnozzle_la-internals.o   -lnl-route-3 -lnl-3  -O3 -ggdb3
> > -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -pthread -flto=auto
> > -Wl,--enable-new-dtags -Wl,--as-needed -fstack-protector-strong -Wl,-z
> > -Wl,now -mstackrealign -Wl,-version-script -Wl,./libnozzle_exported_syms  
> > -pthread -Wl,-soname -Wl,libnozzle.so.1 -o .libs/libnozzle.so.1.0.0
> 
> > [fabbione@mazikeen libnozzle]$ annocheck --verbose .libs/libnozzle.so.1.0.0
> > |grep -i fail
> 
> > as you can see -flto is present, both at build time and linking time.
> > -fstack-clash-protection is present at build time, but not at linking time.
> 
> OK - that is weird.  My only guess is that for some reason that final link
> did not run the compiler's LTO recompilation, but just linked together the
> object files as normal.  So no second recompilation happened and the absence
> of the -fstack-clash-protection flag had no effect.  If you upload the
> libnozzle.so.1.0.0 file I might be able to tell you more.

Uploaded. Tho, if that´s the case, we should verify why the LTO recompilation did not take place. This
behavior would be rather inconsistent as the log shows that the option is enabled, and we would expect
the binaries to have LTO.

> 
> 
> 
> > What does -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 add to the build vs
> > what´s already there?
> 
> It adds the gcc option to find the plugin directory, and an test to make
> sure that the user has not requested that the plugin be disabed:
> 
>   % cat /usr/lib/rpm/redhat/redhat-annobin-cc1 
> *cc1_options:
> + %{!-fno-use-annobin:%{!iplugindir*:%:find-plugindir()} -fplugin=annobin}
> 
> Also having this file means that, if needed, extra options can be added to
> support the plugin in the future, and all without having to change any build
> systems.
> 

Right, then I think I understood the spec right. Effectively, as you can see, -fplugin is already enabled, readding -fplugin=annobin has no effect.

I wonder if forcing plugindir is the culprit that confuses the build (or make it right.. depending ;)).

[fabbione@mazikeen redhat]$ pwd
/usr/lib/rpm/
[fabbione@mazikeen redhat]$ grep find-plugindir * -r
redhat-annobin-cc1:+ %{!-fno-use-annobin:%{!iplugindir*:%:find-plugindir()} -fplugin=annobin}
[fabbione@mazikeen redhat]$

what does provide the find-plugindir() ?

Comment 23 Nick Clifton 2021-05-19 16:09:55 UTC
(In reply to Fabio Massimo Di Nitto from comment #22)

> thanks for the explanation. It starts to make more sense, and yes I
> understand the concept of intermediate stage representation :)

Sorry - I am never sure how technical I should be in my explanations...

> Uploaded. Tho, if that´s the case, we should verify why the LTO
> recompilation did not take place.

It did.  The difference is that in this version of the test the annobin plugin is not present.
(I missed this when looking at it before).  So you have:

  /bin/sh ../libtool  --tag=CC   --mode=link gcc -O3 -ggdb3 -Werror -Wall -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread -I/usr/include/libnl3  -I/usr/include/libnl3  -flto=auto -Wl,--enable-new-dtags -Wl,--as-needed  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection -fcf-protection=full -mstackrealign -Wl,-version-script,./libnozzle_exported_syms -version-info 1:0:0  -o libnozzle.la -rpath /usr/lib64 libnozzle_la-libnozzle.lo libnozzle_la-internals.lo  -lnl-3  -lnl-route-3 -lnl-3

Where the -fplugin=annobin option is present.  But then when transmogrified by libtool the command line becomes:

  libtool: link: gcc -shared  -fPIC -DPIC  .libs/libnozzle_la-libnozzle.o .libs/libnozzle_la-internals.o   -lnl-route-3 -lnl-3  -O3 -ggdb3 -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -pthread -flto=auto -Wl,--enable-new-dtags -Wl,--as-needed -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -Wl,-version-script -Wl,./libnozzle_exported_syms   -pthread -Wl,-soname -Wl,libnozzle.so.1 -o .libs/libnozzle.so.1.0.0

So, no annobin plugin = no notes.  Annocheck does not complain because it assumes that if an entire file is built without annobin notes, then it cannot be sure which tool(s) produced it.  It might have been created by a different compiler, one that does not support plugins at all.  The source code may not even be C or C++ either.



> I wonder if forcing plugindir is the culprit that confuses the build (or
> make it right.. depending ;)).

Maybe.  I have no real idea how libtool works.  I think that it is a shell script, but I am not at all clear as ot its purpose.


> redhat-annobin-cc1:+ %{!-fno-use-annobin:%{!iplugindir*:%:find-plugindir()}

> what does provide the find-plugindir() ?

GCC itself.  It is part of the real gcc program, the one that runs all of the other parts of the compilation process.

Cheers
  Nick

Comment 24 Fabio Massimo Di Nitto 2021-05-19 16:41:02 UTC
(In reply to Nick Clifton from comment #23)
> (In reply to Fabio Massimo Di Nitto from comment #22)
> 
> > thanks for the explanation. It starts to make more sense, and yes I
> > understand the concept of intermediate stage representation :)
> 
> Sorry - I am never sure how technical I should be in my explanations...

No worries at all, I really appreciate the time you are taking to explain everything and make me understand.

> 
> > Uploaded. Tho, if that´s the case, we should verify why the LTO
> > recompilation did not take place.
> 
> It did.  The difference is that in this version of the test the annobin
> plugin is not present.
> (I missed this when looking at it before).  So you have:
> 
>   /bin/sh ../libtool  --tag=CC   --mode=link gcc -O3 -ggdb3 -Werror -Wall
> -Wextra  -fplugin=annobin -D_FORTIFY_SOURCE=2 -fstack-protector-strong
> -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now -fstack-clash-protection
> -fcf-protection=full -mstackrealign -Wno-unused-parameter -pthread
> -I/usr/include/libnl3  -I/usr/include/libnl3  -flto=auto
> -Wl,--enable-new-dtags -Wl,--as-needed  -fplugin=annobin -D_FORTIFY_SOURCE=2
> -fstack-protector-strong -fexceptions -D_GLIBCXX_ASSERTIONS -Wl,-z,now
> -fstack-clash-protection -fcf-protection=full -mstackrealign
> -Wl,-version-script,./libnozzle_exported_syms -version-info 1:0:0  -o
> libnozzle.la -rpath /usr/lib64 libnozzle_la-libnozzle.lo
> libnozzle_la-internals.lo  -lnl-3  -lnl-route-3 -lnl-3
> 
> Where the -fplugin=annobin option is present.  But then when transmogrified
> by libtool the command line becomes:
> 
>   libtool: link: gcc -shared  -fPIC -DPIC  .libs/libnozzle_la-libnozzle.o
> .libs/libnozzle_la-internals.o   -lnl-route-3 -lnl-3  -O3 -ggdb3
> -fstack-protector-strong -Wl,-z -Wl,now -mstackrealign -pthread -flto=auto
> -Wl,--enable-new-dtags -Wl,--as-needed -fstack-protector-strong -Wl,-z
> -Wl,now -mstackrealign -Wl,-version-script -Wl,./libnozzle_exported_syms  
> -pthread -Wl,-soname -Wl,libnozzle.so.1 -o .libs/libnozzle.so.1.0.0
> 
> So, no annobin plugin = no notes.  Annocheck does not complain because it
> assumes that if an entire file is built without annobin notes, then it
> cannot be sure which tool(s) produced it.  It might have been created by a
> different compiler, one that does not support plugins at all.  The source
> code may not even be C or C++ either.

I see, while when using -spec the fpluing=annobin plugin is there and things go south.
Now it all makes sense.
 
> 
> > I wonder if forcing plugindir is the culprit that confuses the build (or
> > make it right.. depending ;)).
> 
> Maybe.  I have no real idea how libtool works.  I think that it is a shell
> script, but I am not at all clear as ot its purpose.

Probably few people on this planet know what it does, but we use it to build shared libraries :P

> 
> 
> > redhat-annobin-cc1:+ %{!-fno-use-annobin:%{!iplugindir*:%:find-plugindir()}
> 
> > what does provide the find-plugindir() ?
> 
> GCC itself.  It is part of the real gcc program, the one that runs all of
> the other parts of the compilation process.

gotcha.

Thanks a lot
Fabio

Comment 25 Nick Clifton 2021-05-20 12:28:01 UTC
Reassigning to libtool.

Comment 26 Fedora Admin user for bugzilla script actions 2021-07-01 15:31:30 UTC
This package has changed maintainer in Fedora. Reassigning to the new maintainer of this component.

Comment 27 Ondrej Dubaj 2021-07-20 09:53:34 UTC
Added explanation and results of investigation about this issue here [1]. I am also unable to reproduce this issue with kronosnet on rhel-9 (newer tried Fedora, but I suppose it will be the same)

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1959843

Comment 28 Ondrej Dubaj 2021-07-20 09:54:40 UTC
(In reply to Ondrej Dubaj from comment #27)
> Added explanation and results of investigation about this issue here [1]. I
> am also unable to reproduce this issue with kronosnet on rhel-9 (newer tried

..never tried (sorry guys)...
> Fedora, but I suppose it will be the same)
> 
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1959843

Comment 29 Nick Clifton 2021-07-20 10:34:25 UTC
(In reply to Ondrej Dubaj from comment #27)
> Added explanation and results of investigation about this issue here [1]. I
> am also unable to reproduce this issue with kronosnet on rhel-9 (newer tried
> Fedora, but I suppose it will be the same)

Right - this was all my fault. :-(  I had mistakenly thought that when LTO compilation is enabled security options like
 -fstack-clash-protection need to be enabled during the second compilation.  This was wrong.  They only need to be specified when the files are compiled for the first time.  The 9.78 update to annobin fixes this mistake, so RHEL-9, Rawhide and F34 builds should all now generate clean annocheck results.

As such, I would suggest that this BZ can be CLOSED...

Comment 30 Ondrej Dubaj 2021-07-20 11:25:14 UTC
Thank you for clarification! Closing this as NOTABUG.


Note You need to log in before you can comment on or make changes to this bug.