RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1404435 - "yum update" on fresh installation of RHEL 7.0 or 7.1 PPC64 (updating to 7.3) results in various yum errors
Summary: "yum update" on fresh installation of RHEL 7.0 or 7.1 PPC64 (updating to 7.3)...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: glibc
Version: 7.0
Hardware: ppc64
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Florian Weimer
QA Contact: Sergey Kolosov
URL:
Whiteboard:
Depends On:
Blocks: 1390370
TreeView+ depends on / blocked
 
Reported: 2016-12-13 20:50 UTC by jcastran
Modified: 2021-12-10 14:49 UTC (History)
14 users (show)

Fixed In Version: glibc-2.17-165.el7
Doc Type: Bug Fix
Doc Text:
Cause: During updates from earlier versions of Red Hat Enterprise Linux for POWER, the system glibc installationis temporarily in an inconsistent state. Consequence: RPM scriptlets fail. Fix: The power8 AT_PLATFORM directory is removed early during the update. Result: The glibc installation remains during the update, and RPM scriptlets run successfully. Harmless warnings will be printed during the update; see <https://access.redhat.com/articles/2931341>.
Clone Of:
Environment:
Last Closed: 2017-08-01 18:09:25 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
PPC64 7.0 and 7.1 yum update results.txt (62.32 KB, text/plain)
2016-12-13 20:50 UTC, jcastran 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 61633 0 medium CLOSED glibc 2.2.5-27 upgrade from 2.2.4-20 fails 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 156656 0 medium CLOSED FC3->FC4 x86_64 Upgrade, bash segfaults 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHSA-2017:1916 0 normal SHIPPED_LIVE Moderate: glibc security, bug fix, and enhancement update 2017-08-01 18:05:43 UTC

Internal Links: 61633 156656

Description jcastran 2016-12-13 20:50:21 UTC 
Created attachment 1231330 [details]
PPC64 7.0 and 7.1 yum update results.txt

Description of problem:
This is specifically for PPC64 (I have not had the time to test ppc64le). With a fresh installation of RHEL 7.0 or 7.1 PPC64, then registering with RHSM and running "yum update", the system will have various packages fail installation. 

Version-Release number of selected component (if applicable):
RHEL 7.0 PPC64
RHEL 7.1 PPC64

How reproducible:
every time

Steps to Reproduce:
1. Install RHEL 7.0/1 PPC64
2. Register
3. yum update

Actual results:
    System has various packages that error out with post or pre scriptlet warnings. End result is attached showing packages that installed, updated, or failed installation.

Expected results:
   System updates to 7.3 without issue.

Additional info:
Comment 5 Florian Weimer 2017-01-09 21:24:04 UTC 
I can reproduce this.

The old glibc package had this:

/lib64/libc.so.6
/lib64/power8/libc.so.6

The new glibc package has just this:

/lib64/libc.so.6

The dynamic linker will prefer the “power8” platform directory with the old libc.so.6 file, which is incompatible with the new /lib64/ld64.so.1 file (which replaces the old file once the new glibc package installed, so it takes effect much earlier).

This will only happen on POWER8 machines.  On POWER7 machines, the directory is always ignored.

I think this is a fundamental incompatibility between RPM, ldconfig/ld.so and the way platform directories work.
Comment 6 Carlos O'Donell 2017-01-10 03:57:37 UTC 
(In reply to Florian Weimer from comment #5)
> I can reproduce this.
> 
> The old glibc package had this:
> 
> /lib64/libc.so.6
> /lib64/power8/libc.so.6
> 
> The new glibc package has just this:
> 
> /lib64/libc.so.6
> 
> The dynamic linker will prefer the “power8” platform directory with the old
> libc.so.6 file, which is incompatible with the new /lib64/ld64.so.1 file
> (which replaces the old file once the new glibc package installed, so it
> takes effect much earlier).
> 
> This will only happen on POWER8 machines.  On POWER7 machines, the directory
> is always ignored.
> 
> I think this is a fundamental incompatibility between RPM, ldconfig/ld.so
> and the way platform directories work.

Interesting, I had not considered this failure scenario. It effectively means that platform directories can never be removed if you expect in-place upgrades to work robustly.

The easiest solution is to keep /lib64/power8 and just make it a symlink to the /lib64 versions. We can't change the dynamic linker behaviour because the point of the platform directories is to search them first.

Any reason why such a solution wouldn't solve this problem?
Comment 9 Florian Weimer 2017-01-10 12:20:05 UTC 
(In reply to Carlos O'Donell from comment #6)
> Interesting, I had not considered this failure scenario. It effectively
> means that platform directories can never be removed if you expect in-place
> upgrades to work robustly.

Right.

> The easiest solution is to keep /lib64/power8 and just make it a symlink to
> the /lib64 versions. We can't change the dynamic linker behaviour because
> the point of the platform directories is to search them first.
> 
> Any reason why such a solution wouldn't solve this problem?

ldconfig might to something unexpected with the symbolic link.

This problem is *not* new.  There is code in releng/glibc_post_upgrade.c which deletes libraries in platform directories.  We just need to add the “power8” subdirectory there, I think.

This code is very old, and not adjusting it caused quite similar issues in the past:

* Tue May  3 2005 Jakub Jelinek <jakub> 2.3.5-5
…
- clean up /lib64/tls instead of /lib/tls on x86-64, s390x and
  ppc64 in glibc_post_upgrade (#156656)

* Fri Apr  5 2002 Jakub Jelinek <jakub> 2.2.5-31
…
- remove in glibc_post_upgrade on i386 all /lib/i686/libc-*.so,
  /lib/i686/libm-*.so and /lib/i686/libpthread-*.so, not just current
  version (#61633)

It's certainly hackish, but it's apparently quite reliable, so I'd suggest to stick with that instead of coming up with a new solution.
Comment 12 Florian Weimer 2017-01-10 12:53:50 UTC 
Untested workaround:

rpm -ql glibc | grep /power8/ | xargs rm

After this command, the system will use the base versions of the libraries, but will continue working until the glibc update is concluded.
Comment 13 Sergey Kolosov 2017-01-10 15:50:27 UTC 
> Untested workaround:

> rpm -ql glibc | grep /power8/ | xargs rm

> After this command, the system will use the base versions of the libraries, but > will continue working until the glibc update is concluded.

Tested on "POWER8" platforms, workaround works for both versions(7.0 and 7.1)
Comment 15 Florian Weimer 2017-02-10 18:41:36 UTC 
I implemented the usual fix, but error messages remain during an upgrade:

  Updating   : glibc-devel-2.17-163.el7.bz1228114.1.ppc64                         6/11 
  Cleanup    : glibc-devel-2.17-78.el7.ppc64                                      7/11 
  Cleanup    : glibc-2.17-78.el7                                                  8/11 
warning: file /lib/power8/libthread_db.so.1: remove failed: No such file or directory
warning: file /lib/power8/libthread_db-1.0.so: remove failed: No such file or directory
warning: file /lib/power8/librt.so.1: remove failed: No such file or directory
warning: file /lib/power8/librt-2.17.so: remove failed: No such file or directory
warning: file /lib/power8/libpthread.so.0: remove failed: No such file or directory
warning: file /lib/power8/libpthread-2.17.so: remove failed: No such file or directory
warning: file /lib/power8/libm.so.6: remove failed: No such file or directory
warning: file /lib/power8/libm-2.17.so: remove failed: No such file or directory
warning: file /lib/power8/libc.so.6: remove failed: No such file or directory
warning: file /lib/power8/libc-2.17.so: remove failed: No such file or directory
  Cleanup    : glibc-headers-2.17-78.el7.ppc64                                    9/11 
  Cleanup    : glibc-2.17-78.el7                                                 10/11 
warning: file /lib64/power8/libthread_db.so.1: remove failed: No such file or directory
warning: file /lib64/power8/libthread_db-1.0.so: remove failed: No such file or directory
warning: file /lib64/power8/librt.so.1: remove failed: No such file or directory
warning: file /lib64/power8/librt-2.17.so: remove failed: No such file or directory
warning: file /lib64/power8/libpthread.so.0: remove failed: No such file or directory
warning: file /lib64/power8/libpthread-2.17.so: remove failed: No such file or directory
warning: file /lib64/power8/libm.so.6: remove failed: No such file or directory
warning: file /lib64/power8/libm-2.17.so: remove failed: No such file or directory
warning: file /lib64/power8/libc.so.6: remove failed: No such file or directory
warning: file /lib64/power8/libc-2.17.so: remove failed: No such file or directory
  Cleanup    : glibc-common-2.17-78.el7.ppc64                                    11/11 

The messages are harmless (they are just warnings), but I don't like them.  But on the other hand, I do not see any other solution here.
Comment 20 errata-xmlrpc 2017-08-01 18:09:25 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:1916
Note You need to log in before you can comment on or make changes to this bug.