Bug 463921

Seeing this on all arches except s390x. The message doen't appear to be related to any particular package in the transaction, i.e. it happens when upgrading different packages on different arches.

For process shared synchronization primitives, yes, the in-memory representation of the mutexes etc. changed, there is no way around this other than not applying the private futex patch at all.  So, you can't use process shared synchronization between processes linked to the old libpthread (RHEL5.2 and earlier) and new libpthread (RHEL5.3+).  That's because old libpthread was keeping some fields in the structures uninitialized, while new libpthread needs them to differentiate between process shared and process private mutexes/condvars/etc.

OK, we are going toadd a release note recommending installing glibc first, change the openmpi packaging to include an explicit require on glibc, and clone this BZ for rhel6 so the problem doesnt get lost.

Release note added. If any revisions are required, please set the 
"requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

New Contents:
When upgrading from an earlier version of RHEL to RHEL5.3, you may encounter an error such as: 

Updating  : mypackage                 ################### [ 472/1655]rpmdb: unable to lock mutex: Invalid argument

This error is triggered by using the rpm query command from within an rpm scriptlet. The rpm instance performing the upgrade is using the prior  glibc throughout the upgrade, but the rpm instance launched from within the script is using the new glibc.  Packages from Red Hat have been altered to avoid this error, but a third-party package encounter it. 

To avoid this error, upgrade glibc first in a separate run, ie

# yum update glibc
# yum update

Has anybody tested that yum update glibc actually works?  glibc depends on glibc-common of the same NVR and vice versa, so it would surprise me.
yum update glibc\* nscd
yum update
should work IMHO.

Panu et al, I revised the release note - what do you think?

Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -2,9 +2,11 @@
 
 Updating  : mypackage                 ################### [ 472/1655]rpmdb: unable to lock mutex: Invalid argument
 
-This error is triggered by using the rpm query command from within an rpm scriptlet. The rpm instance performing the upgrade is using the prior  glibc throughout the upgrade, but the rpm instance launched from within the script is using the new glibc.  Packages from Red Hat have been altered to avoid this error, but a third-party package encounter it. 
+This error is triggered by using the rpm query command from within an rpm scriptlet. The rpm instance performing the upgrade is using the prior  glibc throughout the upgrade, but the rpm instance launched from within the script is using the new glibc.  
 
 To avoid this error, upgrade glibc first in a separate run, ie
 
 # yum update glibc
-# yum update+# yum update
+
+You will also see this error if you downgrade glibc to an earlier version on an installed 5.3 system.

Hi Denise,

I've made a proposed change to the release notes that you can back out if you don't like.

However, in writing the note, I realized that the core of this problem is programs running the rhel5.2 glibc vs. rhel5.3 glibc.  While rpm showed this error, there is the very real possibility that any system daemon that was running before the upgrade, and which doesn't get restarted after the upgrade, and which needs to lock against programs ran from the command line, will start to fail to work.  Should we be telling people to reboot their machines after this upgrade just to ensure that all long running programs are updated to the new shared futex locking semantics?

Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -2,7 +2,9 @@
 
 Updating  : mypackage                 ################### [ 472/1655]rpmdb: unable to lock mutex: Invalid argument
 
-This error is triggered by using the rpm query command from within an rpm scriptlet. The rpm instance performing the upgrade is using the prior  glibc throughout the upgrade, but the rpm instance launched from within the script is using the new glibc.  
+The cause of the locking issue is the shared futex locking in glibc was enhanced with per-process futexes between rhel5.2 and rhel5.3.  As a result, programs running against the rhel5.2 glibc can not properly perform shared futex locking against programs running with the rhel5.3 glibc.
+
+This particular error message is a side effect of a package calling rpm as part of its install scripts. The rpm instance performing the upgrade is using the prior glibc throughout the upgrade, but the rpm instance launched from within the script is using the new glibc.
 
 To avoid this error, upgrade glibc first in a separate run, ie

Compared to intra-process synchronization using pthread_*, process shared synchronization is relatively rare, though sure, any daemon running before the upgrade which will want to synchronize with programs started after the upgrade using PTHREAD_PROCESS_SHARED pthread_* APIs will misbehave.

Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1,8 +1,8 @@
-When upgrading from an earlier version of RHEL to RHEL5.3, you may encounter an error such as: 
+When upgrading from an earlier version of Red Hat Enterprise Linux to 5.3, you may encounter the following error: 
 
 Updating  : mypackage                 ################### [ 472/1655]rpmdb: unable to lock mutex: Invalid argument
 
-The cause of the locking issue is the shared futex locking in glibc was enhanced with per-process futexes between rhel5.2 and rhel5.3.  As a result, programs running against the rhel5.2 glibc can not properly perform shared futex locking against programs running with the rhel5.3 glibc.
+The cause of the locking issue is that the shared futex locking in glibc was enhanced with per-process futexes between 5.2 and 5.3.  As a result, programs running against the 5.2 glibc can not properly perform shared futex locking against programs running with the 5.3 glibc.
 
 This particular error message is a side effect of a package calling rpm as part of its install scripts. The rpm instance performing the upgrade is using the prior glibc throughout the upgrade, but the rpm instance launched from within the script is using the new glibc.