Bug 2144553

Hi Jan,
 
> python3-libs-3.9.10-4.el9_0.x86_64

Please could you point me at this build.  I cannot seem to find it in brew. :-(

Cheers
  Nick

Hi Jan,

  This is another case where annocheck is unable to determine how an executable
  was produced and so it defaults to assuming that it was not compiled from C
  and/or C++.

  It seems that the debuginfo file which should contain the needed information
  is instead empty.  (Empty of DWARF information and annobin data, that is).

  I am not sure if it is relevent, but I did notice these lines in the build.log
  file for python3-libs build:

    gdb-add-index: No index was created for /builddir/build/BUILDROOT/python3.9-3.9.10-4.el9_0.x86_64/usr/lib64/libpython3.so
    gdb-add-index: [Was there no debuginfo? Was there already an index?]

  Maybe this gdb-add-index program/script is having problems and accidentally
  removing the debug info ?

Cheers
  Nick

Thank you Nick for the analysis. That gdb-add-index seems to come from the gdb-headless package, from the gdb component. Do we suspect that to be the problematic part, or does it just point out the problem caused by something else in the build chain?

(In reply to Jan Pazdziora from comment #6)
> Thank you Nick for the analysis. That gdb-add-index seems to come from the
> gdb-headless package, from the gdb component. Do we suspect that to be the
> problematic part, or does it just point out the problem caused by something
> else in the build chain?

Probably that the problem was caused earlier on.  Given the messages generated by gdb-add-index, the implication is that the debug information was already missing by the time that it ran.  But I suppose that there is also the possibility that there is a bug in gdb-add-index which causes it to think that the debug information is missing and then generate an empty(ish) output file.

To be sure we have to take off and nuke the place from orbit.  No, sorry, I meant, to be sure we are going to have to examine the file before it is processed by gdb-add-index.  Is this something that you can do ?  I can try, but I am assuming that you are the python library building expert and would better know how to intercept the build sequence.

I know nothing about how the python package is built, so we'd need Python maintainers' help here.

You should be able to add an exit 1 call to the end of the %install section and retrieve the files from mock. If you need more detail, let us know.

I suspect that this issue is going to be similar to BZ 2144520.  Ie the %install stage is rebuilding the binaries without the security hardening options.  I am currently testing this theory out using scratch builds...

Hmmm - it appears that there is something funny going on with the creation of the debuginfo file for the libpython3.so library.  But only this library, no others.  No idea why at the moment...

Is it possible that this is related? https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/GSSY4PVH2JOSSQJZHKOYFURO4BJT4ES4/

/usr/lib/debug/usr/lib64/libpython3.so-3.11.0-1.fc38.x86_64.debug in python3.11-debuginfo-3.11.0-1.fc38 on x86_64 is missing ELF sections 

(from Fedora)

Checking with

readelf -Ws /usr/lib64/libpython3.so | grep '__stack_chk_fail'

does not find anything, meaning the function to be called when the stack protection detects a corrupted stack is not referenced. So it seems to suggest -fstack-protector-strong was indeed not used.

It appears that libpython3.so is a special case.  Basically is appears to exist so that applications that use the python3 library can have "-lpython3" on their final link command line, and this will work regardless of the version of python3 that is installed.

The libpython3.so library does not contain any real code of its own, (except for the C run time startup and termination files crti.o, crtbeginS.o, crtendS.o and crtn.o).  Instead it contains a dynamic tag indicating that the libpython3.9.so.1.0 shared library is needed.  This contains all of the real C-compiled python support code.

Hence it makes sense that libpython3.so does not contain any references to __stack_chk_fail and that annocheck is unable to determine that it contains compiled code - it does not.  It only contains assembler built code, which neither needs nor uses the stack checking functions.

So, in the short term, it looks like libpython3.so is a candidate for an exception to the annocheck tests.  In the long term it might be nice to find a way to replace libpython3.so.  Maybe with a symbolic link, or a linker script fragment.

So does it mean that in fact annocheck is correct with that

  "skip: stack-prot test because no compiled code found"

message here?

Annocheck emits the same message for iputils (bug 2144509) where it seems the proper compiler flags were missing. Could annocheck be able to distinguish those two cases and emit different messages?

(In reply to Jan Pazdziora from comment #17)
> So does it mean that in fact annocheck is correct with that
> 
>   "skip: stack-prot test because no compiled code found"
> 
> message here?

Yes.  In the case of libpython3.so there is no compiled code,
so annocheck is correct in skipping the test and not flagging
it as a potential problem.

 
> Annocheck emits the same message for iputils (bug 2144509) where it seems
> the proper compiler flags were missing.  Could annocheck be able to
> distinguish those two cases and emit different messages?

It would be nice, but I am not sure how to do it.  The problem is that
there is no real difference between an executable that was built from
code that was not compiled and an executable that was built from compiled
code that has not had any annobin annotation or debug info generation.

Annocheck does use various heuristics, such as looking in the .comment
section (if present) and examining the DW_AT_producer tag (if DWARF debug
information is available).  But if these are missing, and there is no
annobin data either (the annobin data includes details of the compiler
used to create the binary) then annocheck is stuck.

I suppose that we could have a check that says something like "the .text
section in this file is larger than 1K - was it really all produced from
assembler sources ?" but that would probably trigger false positives for
some packages.

Not sure really.  Any suggestions or ideas are most welcome.

> In the long term it might be nice to find a way to replace libpython3.so.  Maybe with a symbolic link, or a linker script fragment.

Is there a (upstream) compelling reason to do so, other than this check?

(In reply to Miro Hrončok from comment #19)
> > In the long term it might be nice to find a way to replace libpython3.so.  Maybe with a symbolic link, or a linker script fragment.
> 
> Is there a (upstream) compelling reason to do so, other than this check?

No. :-}

It would reduce the size of the python3-lib package slightly, but not by much.

In the end the current method works and the only reason for changing it would be for our own convenience.

(In reply to Nick Clifton from comment #18)
> 
> It would be nice, but I am not sure how to do it.  The problem is that
> there is no real difference between an executable that was built from
> code that was not compiled and an executable that was built from compiled
> code that has not had any annobin annotation or debug info generation.

Couldn't annobin help by adding an explicit annotation for the "no compiled code in here" situation?

(In reply to Nick Clifton from comment #20)
> 
> In the end the current method works and the only reason for changing it
> would be for our own convenience.

I wouldn't call it convenience. :-)

The reason I started digging into this is really a case of build process and result compliance with internal guidelines and external requirements, and our ability to prove that compliance.

(In reply to Jan Pazdziora from comment #21)
> (In reply to Nick Clifton from comment #18)
> > 
> > It would be nice, but I am not sure how to do it.  The problem is that
> > there is no real difference between an executable that was built from
> > code that was not compiled and an executable that was built from compiled
> > code that has not had any annobin annotation or debug info generation.
> 
> Couldn't annobin help by adding an explicit annotation for the "no compiled
> code in here" situation?

Oh yes - that is something that I can (and will) do.

FYI,  I am working on an update for annocheck which changes the skip message to:

  skip: stack-prot test because no compiled code found 
  WARNING: Absence of compiler code might indicate a lack of security annotation.
  WARNING: For more details see https://sourceware.org/annobin/annobin.html/Absence-of-compiled-code.html

I decided to put the description of the problem into a web page as it turned out to be more than just a few lines...

Fixed in gcc-toolset-13-annobin-12.09-1.el9

Changing back to ASSIGNED.

We have discovered a problem.  In RHEL-9 the annobin notes are stored in the separate debug info file, rather than the binary itself.  But if the separate debug info file does not contain any actual debug information, eg because the executable was compiled without debugging support, then annocheck thinks that the file is corrupt, and so it does not check it for annobin notes.  This means that annocheck does not see the notes for libython3.so (for RHEL-9) and so it generated a MAYB result.  On RHEL-8, where the notes are held in the actual binary, annocheck sees them and generates a PASS result.

Fixed in gcc-toolset-13-annobin-12.14-1.el9

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (gcc-toolset-13-annobin bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:6475