Bug 2065867

Summary: annobin: Build order issue with gcc?
Product: Red Hat Enterprise Linux 8 Reporter: Nir Soffer <nsoffer>
Component: annobinAssignee: Nick Clifton <nickc>
annobin sub component: system-version QA Contact: Václav Kadlčík <vkadlcik>
Status: CLOSED CURRENTRELEASE Docs Contact:
Severity: unspecified    
Priority: unspecified CC: codonell, fberat, fweimer, mcermak, nickc, sipoyare, tschelle
Version: 8.6Keywords: Bugfix, Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-30 09:49:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nir Soffer 2022-03-18 23:50:27 UTC
Description of problem:

Looks like redhat-rpm-config should inject -fcf-protection=full
instead of -fcf-protection.

rpmdiff complains:

Detecting usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/ioutil.cpython-36m-x86_64-linux-gnu.so with not-hardened warnings '
Hardened: ioutil.cpython-36m-x86_64-linux-gnu.so: WARN: The annobin plugin was built to run on a newer version of the compiler
Hardened: ioutil.cpython-36m-x86_64-linux-gnu.so: FAIL: cf-protection test because no protection enabled 
Hardened: Rerun annocheck with --verbose to see more information on the tests.
' on x86_64

But fc-protection was used when building this package:

Brew build log:
http://{host}/brewroot/work/tasks/5592/43785592/build.log

gcc -pthread -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I/usr/include/python3.6m -c ovirt_imageio/_internal/ioutil.c -o build/temp.linux-x86_64-3.6/ovirt_imageio/_internal/ioutil.o

gcc -pthread -shared -Wl,-z,relro -Wl,-z,now -g -Wl,-z,relro -Wl,-z,now -g -Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection build/temp.linux-x86_64-3.6/ovirt_imageio/_internal/ioutil.o -L/usr/lib64 -lpython3.6m -o build/lib.linux-x86_64-3.6/ovirt_imageio/_internal/ioutil.cpython-36m-x86_64-linux-gnu.so

According to
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/developing_c_and_cpp_applications_in_rhel_8/index#hardening-checker-options_hardening-checker-basics

We should have:

    If available, the -fcf-protection=full option was used.

gcc(1) does not document the meaning of -fcf-proection without an argument.

       -fcf-protection=[full|branch|return|none|check]
           Enable code instrumentation of control-flow transfers to increase
           program security by checking that target addresses of control-flow
           transfer instructions (such as indirect function call, function
           return, indirect jump) are valid.  This prevents diverting the flow
           of control to an unexpected target.  This is intended to protect
           against such threats as Return-oriented Programming (ROP), and
           similarly call/jmp-oriented programming (COP/JOP).

           The value "branch" tells the compiler to implement checking of
           validity of control-flow transfer at the point of indirect branch
           instructions, i.e. call/jmp instructions.  The value "return"
           implements checking of validity at the point of returning from a
           function.  The value "full" is an alias for specifying both
           "branch" and "return". The value "none" turns off instrumentation.

           The value "check" is used for the final link with link-time
           optimization (LTO).  An error is issued if LTO object files are
           compiled with different -fcf-protection values.  The value "check"
           is ignored at the compile time.

           The macro "__CET__" is defined when -fcf-protection is used.  The
           first bit of "__CET__" is set to 1 for the value "branch" and the
           second bit of "__CET__" is set to 1 for the "return".

           You can also use the "nocf_check" attribute to identify which
           functions and calls should be skipped from instrumentation.

           Currently the x86 GNU/Linux target provides an implementation based
           on Intel Control-flow Enforcement Technology (CET) which works for
           i686 processor or newer.

I'm not sure about the issue, it may be an issue in rpmdiff,
the wrong version of annobin, or may be using -fcf-protection
instead of -fcf-protection=full. -fcf-proection without an
argument is not documented, so it should not be used, or gcc
should fix the docs.

Version-Release number of selected component (if applicable):
redhat-rpm-config-125-1.el8.noarch (Used in Brew)
redhat-rpm-config-128-1.el8.noarch (tested in RHEL 8.6)

Comment 1 Carlos O'Donell 2022-03-21 15:43:53 UTC
Please note that -fcf-protection is an alias for -fcf-protection=full.

I've asked Nick Clifton, our annobin expert to have a look at this.

Comment 2 Nick Clifton 2022-03-21 15:46:51 UTC
This looks like an annobin problem.  The fact that the annobin plugin was built for a newer version of gcc is worrying.  It implies that annobin was built by a version of gcc that is no longer in the buildroot...

Comment 3 Nick Clifton 2022-03-21 16:32:29 UTC
Hmm, so annocheck reports:

  Hardened: ./usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/ioutil.cpython-36m-x86_64-linux-gnu.so: 
  WARN: The annobin plugin was built to run on a newer version of the compiler
  Hardened: debug: Annobin plugin was built by gcc 8.5.0 but run on gcc version 8.3.1.

Looking in the build log for:

  https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=43785592

Shows:

  DEBUG util.py:636:   gcc                         x86_64 8.3.1-5.1.el8                   build  23 M
  DEBUG util.py:636:   annobin                     x86_64 9.72-1.el8_5.2                  build 110 k

which confirms that gcc 8.3.1 was the build compiler.
Checking with brew confirms this:

  % brew --quiet latest-build rhevm-4.5-rhel-8-build gcc annobin
  gcc-8.3.1-5.1.el8                         rhevm-4.4-rhel-8      mpolacek
  annobin-9.72-1.el8_5.2                    rhel-8.5.0-z          sipoyare

So the issue is that the rhevm-4.5-rhel-8 buildroot either contains an out of date version of gcc or else a too-new version of annobin.  I am not sure which...

Comment 4 Nick Clifton 2022-03-21 16:37:09 UTC
Oh by the way - I should explain that this discrepancy between the version of gcc that was used to build the annobin plugin and the version of gcc on which the plugin is run, explains why there is a bogus message from annocheck about the -fcf-protection option.  GCC does not have a consistent way for plugins to access information about command line options, so instead they have to examine internal gcc structures.  Structures whose layout changes frequently between different versions of gcc.  So the annobin plugin is expecting a structure layout as used by gcc 8.5.0 but it is run on gcc 8.3.1 which has a different layout, and so it gets bad information about the command line options.  This bad information is then recorded in the object files produced, and later on annocheck loads that information and then issues its FAIL messages.

Comment 5 Nick Clifton 2022-03-21 16:48:03 UTC
Just noticed that the "brew latest-build" output shows that annobin was tagged with rhel-8.5.0-z rather than rhevm-4.4-rhel-8.  So this explains the problem.  I am going to try rebuilding annobin for rhevm-4.5.

Comment 13 Nick Clifton 2022-03-30 09:49:08 UTC
The RHEVM 4.5 buildroot has been fixed - for now.  The issue may reoccur in the future, but we are looking into preventing that from happening.