RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2162746 - Annobin suddenly started failing my Container build: Error: symbol `.annobin_main.c' is already defined
Summary: Annobin suddenly started failing my Container build: Error: symbol `.annobin_...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: annobin
Version: 8.7
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Nick Clifton
QA Contact: Václav Kadlčík
URL:
Whiteboard:
Depends On:
Blocks: 2175768 2175772 2175773 2175774
TreeView+ depends on / blocked
 
Reported: 2023-01-20 18:06 UTC by Jiri Danek
Modified: 2023-11-14 16:10 UTC (History)
5 users (show)

Fixed In Version: annobin-11.13-1.el8
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 2175768 (view as bug list)
Environment:
Last Closed: 2023-11-14 15:25:37 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-145927 0 None None None 2023-01-20 18:08:40 UTC
Red Hat Product Errata RHEA-2023:6909 0 None None None 2023-11-14 15:25:42 UTC

Description Jiri Danek 2023-01-20 18:06:35 UTC
My Dockerfile stopped building. I suspect that the only thing that changed is RHEL and RHEL packages. The build failure is

CMake Error at /usr/share/cmake/Modules/CMakeTestCCompiler.cmake:66 (message):
  The C compiler
 
    "/usr/bin/cc"
 
  is not able to compile a simple test program.
 
  It fails with the following output:
 
    Change Dir: /build/proton/app/build_asan/CMakeFiles/CMakeTmp
   
    Run Build Command(s):/usr/bin/gmake -f Makefile cmTC_05c22/fast && /usr/bin/gmake  -f CMakeFiles/cmTC_05c22.dir/build.make CMakeFiles/cmTC_05c22.dir/build
    gmake[1]: Entering directory '/build/proton/app/build_asan/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_05c22.dir/testCCompiler.c.o
    /usr/bin/cc   -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wp,-U_FORTIFY_SOURCE -fplugin=annobin -fplugin-arg-annobin-no-active-checks  -o CMakeFiles/cmTC_05c22.dir/testCCompiler.c.o -c /build/proton/app/build_asan/CMakeFiles/CMakeTmp/testCCompiler.c
    {standard input}: Assembler messages:
    {standard input}:720: Error: symbol `.annobin_testCCompiler.c' is already defined
    {standard input}:863: Error: symbol `.annobin_testCCompiler.c.hot' is already defined
    {standard input}:1006: Error: symbol `.annobin_testCCompiler.c.unlikely' is already defined
    {standard input}:1149: Error: symbol `.annobin_testCCompiler.c.startup' is already defined
    {standard input}:1292: Error: symbol `.annobin_testCCompiler.c.exit' is already defined
    {standard input}:1931: Error: symbol `.annobin_testCCompiler.c_end' is already defined
    gmake[1]: *** [CMakeFiles/cmTC_05c22.dir/build.make:78: CMakeFiles/cmTC_05c22.dir/testCCompiler.c.o] Error 1
    gmake[1]: Leaving directory '/build/proton/app/build_asan/CMakeFiles/CMakeTmp'
    gmake: *** [Makefile:127: cmTC_05c22/fast] Error 2

To reproduce the problem, build the following Dockerfile

```
FROM registry.access.redhat.com/ubi8/ubi-minimal:latest

RUN microdnf -y --setopt=install_weak_deps=0 --setopt=tsflags=nodocs install \
    rpm-build \
    gcc gcc-c++ make cmake \
    cyrus-sasl-devel openssl-devel libuuid-devel \
    python3-devel swig \
    libnghttp2-devel \
    wget tar patch findutils git libasan libubsan libtsan \
 && microdnf clean all -y

RUN echo 'int main(){return 0;}' > hello.c
RUN set +x \
    && eval "$(rpmbuild --eval '%set_build_flags')" \
    && gcc $CFLAGS -Wp,-U_FORTIFY_SOURCE -fplugin=annobin -fplugin-arg-annobin-no-active-checks  -o main -c main.c
RUN echo DONE
```

$ podman pull ubi-minimal:latest
$ podman build --no-cache -f Dockerfile

The ubi-minimal:latest image I have is

registry.access.redhat.com/ubi8/ubi-minimal                   latest                        35585f3ca6c6  2 weeks ago     94.5 MB

The gcc command that ends up executed is (if I expand CFLAGS)

gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wp,-U_FORTIFY_SOURCE -fplugin=annobin -fplugin-arg-annobin-no-active-checks  -o main -c main.c

The packages I have installed

# cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.7 (Ootpa)

# rpm -qa "*annobin*"
gcc-plugin-annobin-8.5.0-16.el8_7.x86_64
annobin-10.67-3.el8.x86_64

Comment 1 Jiri Danek 2023-01-20 18:12:30 UTC
The part of the command that seems to break things is -fplugin=annobin -fplugin-arg-annobin-no-active-checks. I configure this in CMakeLists.txt to override the annobin which is enabled added from $CFLAGS when I also enable -fsanitize=address.

Comment 2 Nick Clifton 2023-01-23 11:14:09 UTC
(In reply to Jiri Danek from comment #1)
Hi Jiri,

> The part of the command that seems to break things is -fplugin=annobin
> -fplugin-arg-annobin-no-active-checks. I configure this in CMakeLists.txt to
> override the annobin which is enabled added from $CFLAGS when I also enable
> -fsanitize=address.

As a guess, off the top of my head, what is happening here is that the annobin plugin is being run twice.  Since the second plugin generates the same symbol names as the first, the assembler then generates those error messages.  

If you try removing the "-fplugin=annobin" from CMakeLists.txt, but leave in the "-fplugin-arg-annobin-no-active-checks" then that might solve the problem.

As an alternative, you might try adding "-fplugin-arg-annobin-rename" to the set of annobin options in CMakeLists.txt.  This option tells the annobin plugin to rename the symbols it creates, in the hopes of avoiding conflicts like the one that you have encountered.  It is not really recommended as you will still have two plugins operating and so now have twice the number of annobin symbols, and annobin notes, in your binaries.  Its real use if for when the annobin plugin is being built and tested by a version of gcc where the annobin plugin is also active.

Please let me know if either of these suggestions work.  If not, then I will investigate further.

Cheers
  Nick

Comment 3 Jiri Danek 2023-01-23 11:29:50 UTC
I think you are right as to the cause. The `-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1` expands to

    $ cat /usr/lib/rpm/redhat/redhat-annobin-cc1
    *cc1_options:
    + %{!-fno-use-annobin:%{!iplugindir*:%:find-plugindir()} -fplugin=gcc-annobin}

That is, my commandline ends up being something like

    gcc [...] -fplugin=gcc-annobin -fplugin=annobin -fplugin-arg-annobin-no-active-checks

and having both gcc-annobin and annobin enabled causes the error with already defined symbols.

Why does annobin and gcc-annobin both exist?

> If you try removing the "-fplugin=annobin" from CMakeLists.txt, but leave in the "-fplugin-arg-annobin-no-active-checks" then that might solve the problem.

This does not work, the "first" annobin is called `gcc-annobin`, so the option for it should be something like `-fplugin-arg-gcc-annobin-no-active-checks`, which does fail also, differently

    $ gcc -c main.c -fplugin=gcc-annobin -fplugin-arg-annobin-no-active-checks
    cc1: error: plugin annobin should be specified before ‘-fplugin-arg-annobin-no-active-checks’ in the command li

    $ gcc -c main.c -fplugin=gcc-annobin -fplugin-arg-gcc-annobin-no-active-checks
    cc1: error: plugin gcc should be specified before ‘-fplugin-arg-gcc-annobin-no-active-checks’ in the command line

How do I talk to gcc-annobin using the "-fplugin-arg-" option (when plugin name contains a dash)?

> you might try adding "-fplugin-arg-annobin-rename" to the set of annobin options

This works, but as you've said, it is not an ideal solution.

Currently, as a workaround, we decided to completely disable annobin, by getting CFLAGS with "--undefine _annotated_build", like this

    BUILD_FLAGS="$(rpmbuild --undefine _annotated_build --eval '%set_build_flags')"
    eval "${BUILD_FLAGS}"

(The original idea of using RPM flags there is to have RPM-like builds.)

(from https://github.com/skupperproject/skupper-router/pull/912)

Comment 4 Nick Clifton 2023-01-23 12:21:34 UTC
(In reply to Jiri Danek from comment #3)
Hi Jiri,
 
> That is, my commandline ends up being something like
> 
>     gcc [...] -fplugin=gcc-annobin -fplugin=annobin
> -fplugin-arg-annobin-no-active-checks

Right.  Oh boy.

Do you really need to the "...-no-active-checks" option for annobin ?

I am guessing that this is because the plugin is complaining about -D_FORTIFY_SOURCE=2 not being used.  Although since it is definitely there on the command line, the plugin should not complain.

If you do not need that option then you could omit both annobin options from your CMakeLists.txt file. 

  
> Why does annobin and gcc-annobin both exist?

Ah - this is a long and sad tale.  The short version is that the annobin plugin for gcc is very sensitive to changes in the gcc sources.  Even minor changes to gcc can cause the plugin to start generating bogus notes, which then get flagged by annocheck as build failures.  In order to work around this problem we now build two versions of the plugin.  One whenever the annobin package is updated and one whenever the gcc package is updated.  Then there is special code in the redhat-rpm-config package which ensures that the correct plugin is chosen whenever gcc runs.

The gcc-built version of the plugin ensures that there is always a plugin that exactly matches the version of gcc installed.  The annobin-built version of the plugin allows new features and bug fixes to be added to the plugin without the need to rebuild all of gcc.



>     $ gcc -c main.c -fplugin=gcc-annobin
> -fplugin-arg-gcc-annobin-no-active-checks
>     cc1: error: plugin gcc should be specified before
> ‘-fplugin-arg-gcc-annobin-no-active-checks’ in the command line

Darn - that is something that we had not anticipated.

Calling the plugin gcc-annobin instead of say gcc_annobin was obviously a mistake.
Hmmm.  Actually having different names for the plugin is a mistake, since there is
no way for someone constructing a command line to know which plugin is going to be
selected.  We really should have called the plugins something else and then created
a symbolic link called "annobin" which would have meant that command lines could be
constructed properly.
 

I think that fixing this issue is going to get messy...


 
> Currently, as a workaround, we decided to completely disable annobin, by
> getting CFLAGS with "--undefine _annotated_build", like this
> 
>     BUILD_FLAGS="$(rpmbuild --undefine _annotated_build --eval
> '%set_build_flags')"
>     eval "${BUILD_FLAGS}"
> 
> (The original idea of using RPM flags there is to have RPM-like builds.)

Ok, that will work, but I agree that it is not ideal.  If you can live without the -fplugin-arg-annobin-no-active-checks then eliminating that and the -fplugin=annobin from the CMakeLists.txt file is a better workaround.


Cheers
  Nick

Comment 5 Jiri Danek 2023-01-23 12:38:53 UTC
Hi Nick, thanks a lot for looking into this!

> -fplugin-arg-annobin-no-active-checks

This is there so that annobin does not freak out when we build with -fsanitize=. (The image contains regular binary, asanised binary, and tsanized binary; for testing.) It seemed the easiest solution at the time.

I am fairly happy with the workaround for now, and regarding this BZ, unless anything else breaks, I'm just going to just wait and see.

Cheers,
Jiri

Comment 9 Nick Clifton 2023-03-03 17:18:06 UTC
Fixed in annobin-11.12-1.el8

Comment 22 errata-xmlrpc 2023-11-14 15:25:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (annobin bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:6909


Note You need to log in before you can comment on or make changes to this bug.