Bug 1320954 - rpm: Support equivalent of Debian's symbols files for finer-grained ELF dependencies
Summary: rpm: Support equivalent of Debian's symbols files for finer-grained ELF depen...
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1320960
TreeView+ depends on / blocked
 
Reported: 2016-03-24 11:06 UTC by Florian Weimer
Modified: 2021-06-27 18:31 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
: 1320960 (view as bug list)
Environment:
Last Closed: 2021-06-27 18:31:03 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Florian Weimer 2016-03-24 11:06:54 UTC
We have a request to backport additional symbols to glibc.  We currently cannot do this because the auto-generated dependencies would be incorrect (they would refer to GLIBC_2.25, while the glibc package would not provide the full GLIBC_2.25 symbol set).

I propose to implement a facility similar to Debian's symbols files to address this:

  <https://wiki.debian.org/UsingSymbolsFiles>

I'm including some design notes below.  Note that thread_local support is only an example, there are no plans to backport this particular feature (and it is know to be broken and needs redesign upstream anyway).  Perhaps a better example would be backporting the memcpy symbol to glibc in Red Hat Enterprise Linux 6.

# Case study: Adding proper C++ thread_local support to Red Hat Enterprise Linux 7

Let's assume we want to add proper C++ thread_local destructor support to Red Hat Enterprise Linux 7, and this actually requires the __cxa_thread_atexit_impl function from glibc.  (The latter may not be in fact true, but let's assume it is.)

This means that we need to add the function symbol __cxa_thread_atexit_impl to libc.so.6, with version GLIBC_2.18.  But if we do that, our RPM dependency generator will add a dependency on libc.so.6(GLIBC_2.18)(64bit).

We cannot use __cxa_thread_atexit_impl instead because that would make the binaries incompatible with the upstream ABI, and we generally want to avoid that.

We also cannot add libc.so.6(GLIBC_2.18)(64bit) to glibc in Red Hat Enterprise Linux 7 without backporting *all* GLIBC_2.18 symbols.  (Okay, in this case, we can, at least on x86_64, because it seems to be the *only* GLIBC_2.18 symbol, so let's pretend it's not, for the sake of argument.)  If we add libc.so.6(GLIBC_2.18)(64bit) without providing the GLIBC_2.18 ABI, then further symbol additions would not be correctly represented as RPM dependencies.

We have encountered similar situations when we wanted to backport a security fix which introduced a new symbol to a library which uses symbol versioning.

# A potential solution

Debian doesn't use provides/requires for symbol versions.  Instead, versioned library dependencies are used.  By default, the dependency generator picks up the version of the library installed at build time, and adds a dependency which ensures that at run time, the library package is installed in at least that version.

However, this default mechanism can be overridden for individual symbols, and a *lower* version number can be specified.  If all symbols in use have such a lower version, then the dependency generator picks the largest version of these symbols.  The net effect is similar to RPM dependency generator, at least for fully annotated libraries: You can compile and link against a newer library version (provided the library is ABI-stable), but you will only pick up the absolutely required dependency for your application.  Since this approach is not based on symbol versions, it can express partial backports of symbols from newer library versions.

This approach would also work for Fedora.  For Red Hat Enterprise Linux, the complicated branching model may not work well with a simple version comparison.  I think it would be possible to have a symbol list file, just like Debian, but it would not record symbols and library versions, but just the symbol and version.  When the library RPMs are built, the RPM dependency generator provides all the listed symbols in the file, using some sort of syntax we can choose relatively freely (it's just a string to RPM).  The file is also installed as part of the -devel package (similar to Debian).  When building an application, the RPM dependency generator inspects the symbols file, and if a symbol (along with its version) is listed, it generates a dependency according to the syntax used for provides generated for the library package.  If such a dependency is generated, no

In the case of __cxa_thread_atexit_impl, we would list __cxa_thread_atexit_impl in the symbols file, and the RPM capability string used by provides/requires could be:

  libc.so.6(__cxa_thread_atexit_impl)(64bit)

The traditional libc.so.6(GLIBC_2.18)(64bit) capability would neither be required or provided, unless building a 2.18+ glibc version, or linking against a 2.18+ glibc version.  Future glibc versions would have to continue to provide libc.so.6(__cxa_thread_atexit_impl)(64bit), in addition to libc.so.6(GLIBC_2.18)(64bit), for backwards compatibility, so we have to keep the symbols file around indefinitely.

One advantage of this approach (which involves changing both library packages and their -devel packages, the latter need to ship the symbols file) is that during creation of the build root, consistent dependencies are automatically enforced.  If a certain -devel package is put into the build root, it will pull in the required indirect dependencies, and inspection of the package list can reveal if these dependencies are acceptable.

An alternative approach would be to version glibc symbols differently, not by overall library version, but by versioned API set, but I don't think this would be acceptable upstream, and upstream and downstreams might in fact disagree what the scope of a certain interface set actually is.

Comment 1 Neal Gompa 2017-09-17 18:33:08 UTC
So here's a thought... Could we use the new richop 'with' to provide this kind of capability?

For example:

"Provides: libc.so.6(GLIBC_2.18)(64bit) with libc.so.6(__cxa_thread_atexit_impl)(64bit)"

This is similar to what we're doing in Rust for handling features[1]. You can see it in action in the Rust playground COPR[2].

By generating granular Provides and Requires in this manner, you would achieve the same goal.

[1]: https://fedoraproject.org/wiki/PackagingDrafts/Rust
[2]: https://copr.fedorainfracloud.org/coprs/g/rust/playground/

Comment 2 Florian Weimer 2017-09-17 19:43:58 UTC
(In reply to Neal Gompa from comment #1)
> So here's a thought... Could we use the new richop 'with' to provide this
> kind of capability?
> 
> For example:
> 
> "Provides: libc.so.6(GLIBC_2.18)(64bit) with
> libc.so.6(__cxa_thread_atexit_impl)(64bit)"

I don't see what this would accomplish.

There is a benefit from rich dependencies, though.  We could write

Requires: libc.so.6(GLIBC_2.18)(64bit) or libc.so.6(__cxa_thread_atexit_impl)(64bit)

with just:

Provides: libc.so.6(__cxa_thread_atexit_impl)(64bit)

for backports.  This would provide compatibility with already-released glibc packages, and we wouldn't have to add symbol-based Provides: to all future glibc packages.

> This is similar to what we're doing in Rust for handling features[1].

> [1]: https://fedoraproject.org/wiki/PackagingDrafts/Rust

This doesn't make sense to me, either.  It is something that BuildConflicts should be able to handle with regular Provides:.

Comment 3 Neal Gompa 2017-09-18 00:47:34 UTC
I sent a pretty detailed answer to why in rpm-ecosystem ML[1], but the long and short of it is that your strategy only works as long as it's only one level. Once you have multiple levels, BuildConflicts breaks your build.

As for using the richop, the idea is to avoid doing too much weird mangling.

Consider the following:

  (libc.so.6(GLIBC_2.18)(64bit) with libc.so.6(__cxa_thread_atexit_impl)(64bit))

The above clause indicates that it is glibc 2.18 with the function symbol __cxa_thread_atexit_impl. It maintains compatibility with function symbol-less dependencies, and so on.

Your proposed idea of having both current style and your full symbol would work as well, but it contains the redundant GLIBC_2.18 symbol. Heck, we could probably slim it down a bit more by not including "(64bit)" on the function symbol name.

One thing I don't particularly like is the usage of "(64bit)" instead of the actual system architecture (or even just %{?_isa}), but that's how we do things now anyway...

If we had my way, the generate library dependency names would look something like this:

  (libc.so.6(GLIBC_2.18)(x86-64) with libc.so.6(__cxa_thread_atexit_impl))

Or even if we did it the way you propose with: libc.so.6(__cxa_thread_atexit_impl)(x86-64)

[1]: http://lists.rpm.org/pipermail/rpm-ecosystem/2017-February/000471.html

Comment 4 Florian Weimer 2021-06-27 18:31:03 UTC
I think the existing dependency generator framework can be used to implement this on a per-package basis if needed.


Note You need to log in before you can comment on or make changes to this bug.