RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2020989 - glibc: rpmbuild fails with --without benchtests
Summary: glibc: rpmbuild fails with --without benchtests
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: glibc
Version: CentOS Stream
Hardware: All
OS: Linux
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Carlos O'Donell
QA Contact: Martin Coufal
URL:
Whiteboard:
Depends On: 1902514
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-11-08 01:43 UTC by Juro Bystricky
Modified: 2023-07-18 14:30 UTC (History)
12 users (show)

Fixed In Version: glibc-2.28-177.el8
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-10 15:17:47 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-101918 0 None None None 2021-11-08 01:43:45 UTC
Red Hat Product Errata RHBA-2022:2005 0 None None None 2022-05-10 15:18:05 UTC

Description Juro Bystricky 2021-11-08 01:43:18 UTC 
Description of problem:
building of glibc RPMs fails when using the option "--without benchtests"

Version-Release number of selected component (if applicable):
glibc-2.28-167 


How reproducible:
100%

Steps to Reproduce:
$ cd ~
$ mkdir -p rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
$ cd rpmbuild/SOURCES
$ wget https://vault.centos.org/8-stream/BaseOS/Source/SPackages/glibc-2.28-167.el8.src.rpm
$ rpm2cpio glibc-2.28-167.el8.src.rpm | cpio -i
$ cp glibc.spec ../SPECS
$ cd ~
$ rpmbuild --target=x86_64 --without benchtests -ba rpmbuild/SPECS/glibc.spec


Actual results:
Build of all RPMs fails with error:

Processing files: glibc-2.28-167.el8.x86_64
error: Could not open %files file /home/juro/rpmbuild/BUILD/glibc-2.28/glibc.filelist: No such file or directory


Expected results:
All RPMs should build, with the package:
 glibc-benchtests-2.28-167.el8.x86_64.rpm
not being built.


Additional info:
The build with default values succeeds:
$ rpmbuild --target=x86_64 -ba rpmbuild/SPECS/glibc.spec 

The reason for the broken build is a bug in glibc.spec: incorrect conditional sections for "%if %{with benchtests}" and "%endif".

The following patch of glibc.spec fixes the issue:

diff --git a/SPECS/glibc.spec b/SPECS/glibc.spec
index c6b6ac5..c796f84 100644
--- a/SPECS/glibc.spec
+++ b/SPECS/glibc.spec
@@ -1825,6 +1825,7 @@ cp benchtests/scripts/benchout.schema.json %{glibc_sysroot}%{_prefix}/libexec/gl
 cp benchtests/scripts/compare_bench.py %{glibc_sysroot}%{_prefix}/libexec/glibc-benchtests/
 cp benchtests/scripts/import_bench.py %{glibc_sysroot}%{_prefix}/libexec/glibc-benchtests/
 cp benchtests/scripts/validate_benchout.py %{glibc_sysroot}%{_prefix}/libexec/glibc-benchtests/
+%endif
 
 %if 0%{?_enable_debug_packages}
 # The #line directives gperf generates do not give the proper
@@ -2217,7 +2218,7 @@ test $(wc -l < libnsl.filelist) -eq 1
 ###############################################################################
 # glibc-benchtests
 ###############################################################################
-
+%if %{with benchtests}
 # List of benchmarks.
 find build-%{target}/benchtests -type f -executable | while read b; do
        echo "%{_prefix}/libexec/glibc-benchtests/$(basename $b)"
Comment 1 Carlos O'Donell 2021-11-08 17:11:26 UTC 
We fixed this upstream here in Fedora:

commit 5b4007bb565c549efa1b37a61afe390f656398ab
Author: Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen>
Date:   Sun Feb 17 12:02:24 2019 +0100

    Fix bootstrapping glibc
    
    Bootstrapping glibc will disable benchtests, but the if/endif clauses
    excluded essentials from the bootstrapped build. Also, mtrace isn't built
    when bootstrapping.

This fix is included in CentOS 9 Stream.

Can you please talk a little about your CentOS 8 Stream use case for needing this change?
Comment 2 Juro Bystricky 2021-11-08 18:25:52 UTC 
(In reply to Carlos O'Donell from comment #1)
> We fixed this upstream here in Fedora:
> 
> commit 5b4007bb565c549efa1b37a61afe390f656398ab
> Author: Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen>
> Date:   Sun Feb 17 12:02:24 2019 +0100
> 
>     Fix bootstrapping glibc
>     
>     Bootstrapping glibc will disable benchtests, but the if/endif clauses
>     excluded essentials from the bootstrapped build. Also, mtrace isn't built
>     when bootstrapping.
> 
> This fix is included in CentOS 9 Stream.
> 
> Can you please talk a little about your CentOS 8 Stream use case for needing
> this change?

Hmmm, you had a fix for over two years and never applied it.
The need for change is self explanatory: it is a bug and as such it should be fixed.
I ran into this issue when rebuilding glibc with different "configure" options (to 
build and test glibc in CET-permissive mode). 
It takes a fairly long time to build glibc and it is practically at the very end
when the build fails (for no obvious reason) when building without benchtests.
Fixing this bug can save a lot of time and frustration for developers trying to re-build glibc packages
with non-default settings.
Comment 3 Carlos O'Donell 2021-11-08 19:06:37 UTC 
(In reply to Juro Bystricky from comment #2)
> Hmmm, you had a fix for over two years and never applied it.
> The need for change is self explanatory: it is a bug and as such it should
> be fixed.

I appreciate that this can be frustrating. Sorry for that.

We have many customers with many requests for support, just like this one.

We work with our customers to prioritize those requests and ensure we deliver value for those customers.

I'm adding H.J. Lu (Intel) to this issue since we've been working together on Intel CET enablement with HJ since upstream glibc 2.28 (2018-08-01).

> I ran into this issue when rebuilding glibc with different "configure"
> options (to 
> build and test glibc in CET-permissive mode). 

Are you doing specific CET testing for Intel?

Would you be able to expand a little bit more here?

> It takes a fairly long time to build glibc and it is practically at the very
> end
> when the build fails (for no obvious reason) when building without
> benchtests.

I did the initial Intel CET bringup in RHEL8 with HJ's support in order to be ready to deploy the feature on future hardware.

Developers building and testing glibc for Intel CET enablement are generally working in Fedora where this bug is already exist (upstream of RHEL).

You are going to find many more packages need CET support (including libffi which was just rebased in RHEL9 to include CET support) in CentOS Stream 8.

> Fixing this bug can save a lot of time and frustration for developers trying
> to re-build glibc packages
> with non-default settings.

Collaboration on Intel CET enablement is largely happening in Fedora so this is not a normal use case for CentOS Stream 8 today.

Is there a specific reason you're looking at CentOS Stream 8?
Comment 4 Juro Bystricky 2021-11-08 23:00:29 UTC 
The distros we use are based on customer feedbacks.

Our CET work consists mostly of identifying which binaries in RHEL8/CentOS8-Stream are compiled with CET
support. In other words, we try to identify packages that will not run with hardware CET and kernel with
CET support. We also try to measure the performance impact of enabled hardware CET.

It seems most of the binaries in RHEl8.4 are already built with CET support, which is nice. 
However, some fairly important apps cannot run with CET enabled hardware and kernel at all (Firefox, ...).
So we provide an option to install glibc built in CET permissive mode should one insist on running those
apps.

Regarding libffi with CET support: I know libffi-3.4 has CET support, but the ABI has changed. 
Is the RHEL9 libffi with CET support based on libffi-3.1 with backported CET patches, or is it based on libffi-3.4 and the new ABI?
Comment 6 Carlos O'Donell 2021-11-26 18:06:02 UTC 
(In reply to Juro Bystricky from comment #4)
> The distros we use are based on customer feedbacks.

Good to know they are using CentOS 8 Stream, or RHEL8 :-)
 
> Our CET work consists mostly of identifying which binaries in
> RHEL8/CentOS8-Stream are compiled with CET
> support. In other words, we try to identify packages that will not run with
> hardware CET and kernel with
> CET support. We also try to measure the performance impact of enabled
> hardware CET.

Very interesting. I would be happy to hear the results of such an analysis. Please feel free to contact me at carlos

> It seems most of the binaries in RHEl8.4 are already built with CET support,
> which is nice. 

This was part of our RHEL8 GA target, to maximize the coverage.

> However, some fairly important apps cannot run with CET enabled hardware and
> kernel at all (Firefox, ...).

Correct, not everything is built, but the set of such things is smaller than if we had not started enablement.

> So we provide an option to install glibc built in CET permissive mode should
> one insist on running those
> apps.

We should not be doing this, such an installed glibc would invalidate the support provided by a Red Hat Enterprise Linux subscription.

We should be working together to ensure RHEL's glibc supports the use cases of our joint customers.

I'm the technical lead for glibc in RHEL and I can help with those discussions.

CET is per-process enabled.

The RHEL glibc is built with --enable-cet, which *defaults* to CET enabled (non-permissive).

You can change the default.

To run an application in permissive you would use `export GLIBC_TUNABLES=glibc.cpu.x86_ibt=permissive,glibc.cpu.x86_shstk=permissive` (can be on, off, or permissive) to turn on permissive behaviour for the *process*. Does that work for you?

We should be working together, between Intel and Red Hat, to ensure that RHEL8 glibc has any required capabilities needed by our joint customers.

> Regarding libffi with CET support: I know libffi-3.4 has CET support, but
> the ABI has changed. 

Correct.

> Is the RHEL9 libffi with CET support based on libffi-3.1 with backported CET
> patches, or is it based on libffi-3.4 and the new ABI?

In RHEL9 we rebased to libffi-3.4 (SONAME bump).

In RHEL8 we have some ideas for enabling CET in libffi-3.1 without the SONAME bump.
Comment 18 errata-xmlrpc 2022-05-10 15:17:47 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (glibc bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:2005
Note You need to log in before you can comment on or make changes to this bug.