Bug 88456

I'm having the same issues when I do an rpm -Fvh glibc* It updates glibc-common
just fine but about half way through glibc I get the following:

INIT: id "x" respawning too fast: disabled for 5 minutes
error: %post(glibc-2.3.2.27.9) scriptlet failed, exit status 121

After that the system is useless.

reboot
Segmentation fault...... 

From what version are you updating? If you are updating from 8.0 the new libraries 
use tls e.g. like version change. I had the same problem. What I did was to make 
a compatible boot CD (RedHat 9), boot from that, choose linux rescue. Have all your rpms 
there, including popt, rpm, glibc rpms. Then do rpm -Fvh --root=/mnt/sysimage *.rpm 
(assuming that your systme was mounted under /mnt/sysimage by linux rescue, which 
is the default). If all the way at the end you get an error because in writing something 
rpm is not using the --root=/mnt/sysimage option but wants direct /var/lib/rpm instead. 
Go to your /var/lib/rpm, rm -f db_*, edit /etc/rpm/macros and insert the line: 
%__dbi_cdb create private cdb mpool mp_mmapsize=16Mb mp_size=1Mb 
then do rpm --rebuilddb. Hope this helps....... 
 
Other alternative is to buy RedHat 9 CD's and do a graphical upgrade. After that the 
upgrades should work with no problem. 

This is on a fresh install of rh9. Everything else is updated, then when I do an
rpm -Fvh glibc* on the system is when I see the error posted previously.

I'm confirm it. I have just made rpm -Fhv * on updates directory (i386)  and I
had got "core dumped" during upgradeing. Now, I have to boot from cdrom and
rescue my system. :( System is RH 9.0, last week, i have made upgrade from RH
8.0+updates 
For RH team, please make tests for new RPMS  packages first...

Had the same problem, fresh 9 install on an ASUS laptop, PII 366, 
did the rpm -Fvh on updates directory.

Tried a rescue but could not figure out how to rpm install on the mounted disk; 
a chroot obviously fails because glibc is broken

Tried an update with 9 on the broken system too, could "update" all glibc
components except glibc itself, ended up re-installing

same thing just happened to a freind that used up2date to update a rh8 box. His
system is useless....

I've traced this back to an rpm replacing ld.so.conf with a blank file, and
backing up the original as ld.so.conf.rpmsave - I replaced the new one with the
original and all was fixed.

You're "upgrading" from the i686 glibc to the i386 glibc.  The upgrade
ordering puts the "new" i386 glibc on the system, runs the %post,
which fails because there are some files in /lib/i686 and
/usr/lib/i686 left over from the "old" i686 glibc which are used in
the %post.  Once the %post fails, the "old" i686 glibc never gets
removed, so you're stuck with things using the wrong libraries and
you'll get segfaults all the time.
                                                                                
This is how to correct the problem:
                                                                                
run:
                                                                                
$ export LD_ASSUME_KERNEL=2.2.5
$ su
Password: [enter root password]
# rpm -q glibc
glibc-2.3.2-11.9
glibc-2.3.2-27.9
# rpm -e glibc-2.3.2-11.9
                                                                                
(substitute the glibc for the "old" version, whatever you have
installed)
                                                                                
At this point dynamically linked apps will work again, and I would
STRONGLY suggest that you upgrade to the i686 version of the glibc
packages by using:

# rpm -Uvh --replacepkgs --replacefiles glibc-2.3.2-27.9.i686.rpm

FYI on i686 boxes (ie. [ `rpm -q --qf '%{arch}\n' glibc` == i686 ] before upgrade) you should always use -Fvh line like:
rpm -Fvh *2.3.2-27.9.i686.rpm {glibc-[a-z],nscd}*2.3.2-27.9.i386.rpm
or
rpm -Fvh i686/*2.3.2-27.9.i686.rpm i386/{glibc-[a-z],nscd}*2.3.2-27.9.i386.rpm
Anyway, I'll add code to i386.rpm /usr/sbin/glibc_post_upgrade to effectively rm -rf /lib/i686
(well, at that point rm obviously cannot be used, so it will have to be readdir/unlink).

Ah, I reported this incorrectly. What I really did was:

cd whatever-path/i686
rpm -Fvh glibc-2.3.2-27.9.i686.rpm ../i386/glib-common-2.3.2-27.9.i686.rpm

and the error occurred.

So I'm not sure the solution is the one described.

Hisashi, can you from the rescue do:
cd /mnt/sysimage (or wherever you have your / mounted to)
ls -l lib/ld* lib/i686 lib/tls
usr/bin/strace usr/sbin/chroot /bin/echo | less

I am unsure what you want. I fixed my system (not included in the original bug
report -- I am the original submitter, I think) by booting the rescue disk,
forcing the install of the old glibc defining the root directory, and then
running the update again.

At first I thought the problem was because I tried the update remotely (the
update restarts sshd, which I was running).

Here' some of the info you asked for:

anna:/ % ls -l lib/ld* lib/i686 lib/tls
-rwxr-xr-x    1 root     root       104560 Apr  7 16:25 lib/ld-2.3.2.so
lrwxrwxrwx    1 root     root           11 Apr  9 17:47 lib/ld-linux.so.2 ->
ld-2.3.2.so

lib/i686:
total 1868
-rwxr-xr-x    1 root     root      1548424 Apr  7 16:25 libc-2.3.2.so
lrwxrwxrwx    1 root     root           13 Apr  9 17:47 libc.so.6 -> libc-2.3.2.so
-rwxr-xr-x    1 root     root       212020 Apr  7 16:25 libm-2.3.2.so
lrwxrwxrwx    1 root     root           13 Apr  9 17:47 libm.so.6 -> libm-2.3.2.so
-rwxr-xr-x    1 root     root        93268 Apr  7 16:25 libpthread-0.10.so
lrwxrwxrwx    1 root     root           18 Apr  9 17:47 libpthread.so.0 ->
libpthread-0.10.so
-rwxr-xr-x    1 root     root        37592 Apr  7 16:25 librt-2.3.2.so
lrwxrwxrwx    1 root     root           14 Apr  9 17:47 librt.so.1 -> librt-2.3.2.so

lib/tls:
total 1868
-rwxr-xr-x    1 root     root      1536292 Apr  7 16:25 libc-2.3.2.so
lrwxrwxrwx    1 root     root           13 Apr  9 17:47 libc.so.6 -> libc-2.3.2.so
-rwxr-xr-x    1 root     root       212020 Apr  7 16:25 libm-2.3.2.so
lrwxrwxrwx    1 root     root           13 Apr  9 17:47 libm.so.6 -> libm-2.3.2.so
-rwxr-xr-x    1 root     root        80340 Apr  7 16:25 libpthread-0.34.so
lrwxrwxrwx    1 root     root           18 Apr  9 17:47 libpthread.so.0 ->
libpthread-0.34.so
-rwxr-xr-x    1 root     root        37080 Apr  7 16:25 librt-2.3.2.so
lrwxrwxrwx    1 root     root           14 Apr  9 17:47 librt.so.1 -> librt-2.3.2.so
-rwxr-xr-x    1 root     root        21572 Apr  7 16:25 libthread_db-1.0.so
lrwxrwxrwx    1 root     root           19 Apr  9 17:47 libthread_db.so.1 ->
libthread_db-1.0.so

Well, when you have your system already fixed, that information is not useful.
Anyway, sshd condrestart should restart just the sshd master daemon, not already
running connections. When I was updating my workstation, I was running it through
ssh connection and saw no problems at all.

Encountered this problem upgrading from RH8.0 to latest RH9 without first
upgrading to stock RH9.  There's an incompatibility between glibc-2.3.2-27.9 and
rpm-4.1-1.06.

Fortunately I had SSH and some working RH9 systems, so I forced the replacement
files over from the working system with something like the following (use at
your own risk, this is from memory):

for d in /bin /usr/bin /usr/lib; do rsync -va -e ssh $d/*rpm* broken.host:$d/; done

After that, RPM could be used on the broken system again, including rpm -hUv on
the rpm-4.2-0.69 to fill in various ancillary files not copied via rsync.

Suggest RHBA-2003:136-07 be withdrawn until this is fixed.

(1) post mortem using: linux recover

(a) /mnt/sysimage/etc/ld.so.config was intact

(b) rpm --root=/mnt/sysimage -qa | grep glibc

showed that glibc-common was updated, but both
glibc packages exist in the rpm database.
  OLD: glibc-2.3.2-11.9
  NEW: glibc-2.3.2-27.9

Looks like something failed in the update process.

(2) removed glibc-2.3.2-11.9
    rpm --root=/mnt/sysimage -e glibc-2.3.2-11.9
    reinstalled glibc-2.3.2-27.9 : if you happen to have
      in on the failed system somewhere ; burn it to CD
    rpm --root=/mnt/sysimage -Uvh --replacepkgs --replacefiles glibpkg

The glibc is the i386 package, not the i686 package.

(3) test it out
    chroot /mnt/sysimage

You will know if the system fixed if the chroot does not segfault.

(a) chroot worked  
(b) rebooting: its doing fsck's on the journaled filesystems,
    seems like we are back in operation.

*** Bug 88709 has been marked as a duplicate of this bug. ***

With upgrading from RHL8 directly to RHL9 glibc. The problem is that rpm used
to be statically linked in RHL8 and earlier. There is no binary compatibility
with statically linked applications (which use NSS/iconv/dlopen/locale)
accross versions. So you should have upgraded rpm first (in RHL9 it is not
statically linked) and then glibc, or use rpm outside of your installed root
(e.g. through upgrade process).

It would be a lot safer if the requirement to upgrade rpm before glibc was
stored in the relevant .rpm's along with other dependencies.

The RH8 -> RH9 is a separate bug, isn't it?

And I checked my secure log. The actual command that caused everything to die:

Apr  9 15:51:41 anna sudo:    htodd : TTY=pts/2 ; PWD=/misc/updates/9/i686 ; USER=root ; COMMAND=/bin/rpm -Fvh glibc-2.3.2-27.9.i686.rpm ../i386/glibc-2.3.2-27.9.i386.rpm ../i386/glibc-common-2.3.2-27.9.i386.rpm ../i386/glibc-debug-2.3.2-27.9.i386.rpm ../i386/glibc-devel-2.3.2-27.9.i386.rpm ../i386/glibc-profile-2.3.2-27.9.i386.rpm ../i386/glibc-utils-2.3.2-27.9.i386.rpm

Since that time, I've rpm -e'd debug and profile.

I face the same problem while updating the glibc. But now it works fine after 
following Rob Cermak step-by-step instruction to solve the problem.

*** Bug 88771 has been marked as a duplicate of this bug. ***

I confirmed the same problem when I updated RH9.
Due to some reasones, I have to use the i386 packages (especially glibc)
in i686 machines. This was ok when I use it in Redhat 8.0, but when I
tried the same way in redhat 9 (by using apt-get upgrade, and set the
Architecture "i386" in /etc/apt/apt.conf, it installed the
updates/glibc-2.3.2-27.9.i386.rpm for me), it failed when it boots to
init...
So I use the rescue disk, boot it, then reinstall glibc i686 package,
it' ok again.

What can I do for this if I have to use the glibc i386 package now ?
not upgrade ? 
Any better idea ?
Thanks.

Having the same problem after installing a fresh RH 9.0 and then applying
glibc-2.3.2-27.9.i386.rpm to my system. An interim solution was to take the
_files_ (not the RPMS) of the RH 9 origin glibc and to copy these files to their
places belonging to. The system was then running again, but not 100% correctly
(it seems that the rpm program itself was a little bit damaged).

I had to re-install RH 9 completely.

My experience was as follows:

3 installs all gave the same result:

System unuseable after upgrading the glibc....i386.rpm on the system.

Failure of the "post scriptlet" with error 121

glibc-common installed fine.

The fix was to re-install....

Several suggestions have been made about the cause.

1. Is it required to use the glibc...i686.rpm version?
2. What about Athelon systems?
3. Are we looking at a totally different problem like a
   broken glibc?

Tony

I am seeing almost the exact same problem -- without having used the i686 version.

I have a machine (2.1GHz athlon, 1GB ram, single cpu) on which I installed rh 9
yesterday.  I ftp'ed the updated files and an rpm -Fvh on the new glibc files. 
ALL were the i386's no i686's (and for some reason rh has not provided any
athlon versions this time???).

After the update, I did not check to see whether /etc/ld.so.conf was of length 0.

However, after a reboot, I got the same INIT: id "x" respawning too fast:
disabled for 5 minutes problem described above.

In my case, the routine suggested above by Rob Cermak did not work.  Even after
the 3rd  re-attempt.  I even took the tape (doesn't everyone back up their
system immediately after a new install and before adding *anything* ??? :-),
pulled the ld.so.conf and ld.so.cache off it and copied them back onto this
system -- and the problem *STILL* lived through the reinstall procedure.  After
 3 tries (rpm , force, root mnt/sysimage, etc.) rpm -qa "glibc*" with root
/mnt/sysimage looks OK; however, any attempt to chroot /mnt/sysimage fails with
a segfault.

I am currently in the process of solving the problem by reinstalling EVERYTHING
as ANOTHER fresh install ... agin; however, (a) I think there is a
****SERIOUS**** problem in the updated glibc rpm's and (b) I think they should
be pulled from the website until the bug is found.

I agree that the RPM's should be pulled and fixed ones put in place, but that
would require that redhat admit that they released buggy updates and I doubt
that will happen. This is obviously causing problems for people and most likely
pushing people away from redhat. I suggest that a serious update like this be
tested a little more before being released to break all kinds of systems. I know
someone that used up2date and it still hosed his system. Concetrate guys, and
get it together.

  This bug is a showstopper for us.  We stopped all upgrades to RH9 and
prohibited all updates to systems already upgraded to RH9 until this is fixed.  
  How could such critical functionality with regard to a critical component not
be noticed during testing?  I'd like to see the test results!
  Well, I guess it can happen even to the best of us.  What we'll be looking for
now is how quickly RedHat gets this fixed.

The errata had a bug, I believe we clearly admitted that and made sure
it will be fixed in the future versions (including possible RHL9 glibc errata
when/if that becomes neccessary). But I strongly disagree with calling this
bug a showstopper. It occurs only when an admin does errnously perform an
i686 -> i386 "upgrade", which is something which shouldn't be done, isn't supported
nor has been tested. The only valid reason for i686 -> i386 glibc replacement
is when you're installing a disk on i686+ box for an non-i686+ system and plan to
move the harddisk there later on. This is very rare thing to do.
up2date will choose the right architecture for you and if you're using rpm
by hand, you better do that too, because otherwise you're crippling your system.
As for why there is no .athlon.rpm, there is no measurable gain in it. If somebody writes
Athlon optimized str*/mem* operations, maybe, but anyway, the gain in using
.i686.rpm over .i386.rpm will be always way bigger than any gains of .athlon.rpm
over .i686.rpm might ever be.

Don't you think that it would be a good idea to add something to the rpm's that
would not allow this sort of update? It seems like the right thing to do here
since so many people are having problems, and it may even help Redhat keep some
customers (clue). I really think you have the wrong attitude about this.

I can remember at least one (and I think more than one) cases where RedHat
officially said that they had made an error on rpm's -- this is *NOT* M$, people.

That being said (hey I *like* this release and I finally get the posix threads
I've been whining about for years...), I would have to take exception to the
statment that:

> It occurs only when an admin does errnously perform an
> i686 -> i386 "upgrade", which is something which shouldn't be done, isn't
> supported nor has been tested. 

As I posted above, it happened to me on an ATHLON system on which the only rpm
downloaded was the i386 version, so installing the i686 version was a literal
impossibility.  QED, the bug can hit in other circumstances.

OTOH, I have previously built the athlon glibc (under 8.0 and 7.3) and saw only
minimal improvements (if any) from them -- when I asked, I was only curious.

And that's the problem. The installed of course installed the .i686.rpm glibc
on your box and you were trying to "upgrade" it to .i386.rpm version you've
downloaded. Athlon has i686 ISA, including cmov. Try:
uname -m; cat /proc/cpuinfo | grep ^flags.*:.*cmov

jakub has repeatedly stated this is only when you install glibc for i386 over
glibc or i686. However, at least two people, including me, think it's triggered
when you're just doing a i686 -> i686 upgrade.

I've posted the command I ran in comment #19. Can someone check to see if I made
that mistake?

Ok. But what to do in this rare situation (i686->i386) when i made an upgrade
from RH8 to RH9 on Athlon machine and then put HDD to old Pentium (of cource
reinstalling glibc to i386 package) and reciving "illegal instruction" error
trying to run httpd or wget? It happends on standard glibc from RH9 and from
RH9-updates too. Any ideas, solutions?

The fact is that things that normally worked (for a few years) do not now and it
is a showstopper.  If semantics are now going to be strict (i386, i686 vs
athlon) whether it be kernel or glibc, the package manager (rpm) must perform
this check.   As with comment #33, we have made the mistake of loading a i386
kernel on an athlon box (rpm will let you do it; no warnings), it will still
boot so you can switch back a load the proper arch.  The fix for #33 is to make
sure you boot with a i386 kernel so it will properly operate on the pentium
chip.  At this point, there is NO check for i386, i686 or athlon in the kernel
or glibc package.   A separate bug (enhancement for rpm) should be opened with
this bug listed as a dependency and should block any future development beyond RH9.

I am seeing an instance of the same problem and it has nothing to do with
switching back and forth between i686 and i386 rpms (although I am sure that
does produce a problem as listed in the other comments).  Here is what I did:

1)  Had a nicely running RH 8.0 box.  Performed an upgrade using RH 9.0 CDs
bought from the local computer store.

2)  Once the RH 9.0 upgrade was complete, rebooted the box.  No problems at this
point.  The box came up fine.

3)  Ran up2date --list.  Amongst other things, it listed newer kernel and glibc
packages.

4)  Downloaded kernel-source-2.4.20-9.i386.rpm.  Installed this rpm and followed
standard (RH manuals) procudures to build a new kernel.  Nothing fancy was done
here, just deselected hardware components I do not have and such.  Saw no
options during make menuconfig regarding tls, nptl, or threads in general. 
Built and installed the kernel with no issues.

5)  Downloaded and installed glibc-2.3.2-27.9.i386.rpm and
glibc-common-2.3.2-27.9.i386.rpm.  Rebooted the box with no problems.

6)  Downloaded and installed db4-4.0.14-20.src.rpm.  Tried to do a rpmbuild
--rmsource --clean -bb db4.spec.

7)  The rpmbuild bombed complaining about not finding POSIX mutexes.

This started an adventure that has been going on for three days so far with no
solution.  As far as I can tell, the system does not have support for nptl.  I
have googled and gargled and himmed and hawed to no avail.  I tried all the
suggestions I could possibly apply to the system mentioned in the previous
comments of this bug report to no avail.  At this point, there is not nptl on
this system.  Period.

On another machine, I installed RH 9.0 from scratch.  Everything works fine on
it so far but I have not had a chance to apply the glibc and kernel updated that
I applied to the machine used in the above testing.  I will try to do that
tomorrow and see what happens.

On the first machine (the one with the updated glibc and kernel), I even went so
far as to install the older glibc rpms ( the ones that ship with RH 9.0). 
However, even after doing so (I did not back rev. the kernel though), I still
have the problem with the DB4 rpmbuild complaining about missing POSIX mutexes.

I verified in db4.spec that the message about missing POSIX mutexes is being
generated when the configure is passed --enable-threads.  If I remove that,
rpmbuild succeeds.

The above pretty much proves that RedHat has broken npls in some fashion.  I do
not see how this can be negated.  I will ascertain whether this is broken in the
post RH 9.0 release glibc or kernel updates, or whether it is broken whenever a
RH 8.0 is updated to RH 9.0.

In the meantime I have put on hold a major migration we were about to perform
from existing FBSD boxes to RH.  I will see whether I have any hair left by the
time the nptl issue is resolved to go forward with the RH migration.

--Moby
moby

FWIW (comment #34), I opened a request a few days ago for rpm to consider
%{arch} when upgrading. See Bug 88623. That would prevent (most? all?) of the
problems people experienced here....

While this is not a showstopper (Rob Cermak's recovery recipe worked perfectly
for me), it is a huge pain in the ass. IMHO, RedHat should yank the glibc rpm's
until rpm itself is modified or a much more salient errata is filed to prevent
this from happening to others.

Looks like Redhat does not want to hear about this anymore as jakub has removed
himself from this post, and put the priority to low. That really blows, and in
my opinion it's the same thing I said before about redhat admitting to a buggy
update. Posting a couple comments on bugzilla is much better than pulling the
update and replacing it with something that works properly. This would cause
publicity issues for redhat. I hope you guys learned your lesson here, and
didn't drop too many customers in the process. Concentrate guys, and try to do a
little more testing. Or better yet, make a new release when the release is ready
not when marketing tells you to. A new glibc a week after release. Give me a
break....

My bad I see jakub is still with us here.

ra1, I am testing, in preparation for replacing a bunch of FBSD boxes to Linux :)

Anyway, I think I have found the crux of the problem.

Here are some empirical facts:

1)  rpm -qil -p glibc-2.3.2-27.9.i386.rpm | grep tls 
produces no results at all.

2)  rpm -qil -p glibc-2.3.2-27.9.i686.rpm | grep tls
produces eleven references to directories (/lib/tls) and files.

This shows that RedHat does not have NPTL support in the i386 build of glibc. 
Period, no two ways around it.  I would say this is a bug, a major bug, and a
show stopper by any definition, especially since every ad. for RH 9.0 screams
about NPTL support.

Looking at glibc-9.spec from glibc-2.3.2-27.9.src.rpm, I see a line reading:
%define nptlarches i686 athlon
This shows that nptl is not being compiled for any architectures other than i686
and athlon.

The above is what is causing nptl to have issues on all glibc's other than i686
and athlon architecture builds.

This is a very major issue because packages other than glibc are not setup this
way.  For example, looking at the spec. file of a typical package that is used
by many other packages - DB4 (used by RPM, Cyrus, etc. etc), here is what I see:

%define nptl_arches %{ix86}.

This means that a DB4 build from source rpm will break on any architecture other
than i686.  The binary RPM for db4 from RedHat for the i386 architecture also
appears to be built using nptl support!

This leads to the following questions:

1)  Why is NPTL support in glibc restricted to the i686 and athlon architectures?

2)  Is there any harm in editing the glic-9.spec file and adding nptl support to
all x86 architectures?

3)  Why do the nptl requirements for glibc not match nptl requirements in other
packages from RedHat ?

In light of the above I think this is a serious bug and needs to rectified as
soon as possible.  Otherwise people will continue to see mysterious breakdowns
with various packages.

--Moby

This bug _IS_ a showstopper for us.  Jakub mentions in #28 that the only reason
someone would go from 686->386 is if you plan on moving your hard disk later on.
 This is not true.

We have a server whose filesystem serves 40+ diskless clients via nfs.  For this
application, the server _MUST_ have a i386 glibc so as to be compatible with all
kinds of CPU architectures of clients.

I notice that the Aptus product from Shaolinmicro.com relies on this same
principle.  In short, anyone using diskless or shared nfs root filesystem will
not be able to use RH9 until there is a way to install 386 over top of 686 glibc.

It would be very helpful to us if the 386 post scriptlet could manage to
successfully remove the 686 rpm.  I think this is a much better approach than
trying to disallow rpm from updating an rpm with a different architecture.

To Mobeen Azhar:
NPTL cannot be supported on i386, since it lacks essential instructions
(compare and swap particularly).

OK, here's a workaround based on the steps in #8 that seems to work (for those
like me who need an i386 glibc on their 686 machine):

> export LD_ASSUME_KERNEL=2.2.5
> rpm -Uvh --force glibc-2.3.2-27.9.i386.rpm      #post scriptlet fails
> rm -r /lib/i686 /lib/tls
> ldconfig

Then, just to make sure, I did again:
> rpm -Uvh --force glibc-2.3.2-27.9.i386.rpm       #no scriptlet failure
> ldconfig

<warm_fuzzy>
BTW, some of you guys (ras1) are too hard on Jakub and RH.  Did you _buy_ your
last RH release or did you download it for free?  We have to remember this is
still open source here and we have to work together to make things happen for
the best.
</warm_fuzzy>

Jakub, if NPTL cannot be supported on i386, then RPM packagers at RedHat need to
know so that they do not assume that it is supported on x86.

Can NPTL be supported on i586? My system is acutally i586 but I do not see a
binary i586 RPM for glibc.  The src. RPM only build NPTL support for i686 and
athlon.

What do you suggest for people on non-i686 and non-athlon architectures?  Do we
have to manually build all affected RPMS (such as DB4) after editing the spec
files to remove posix thread support?

Kyle, I am not trying to be too hard on Jakob or RH.  I know the guys are doing
wonderful work.  If nothing else, I am working very hard to get the RH distro.
of Linux into a place that is heavily Solaris and NetWare right now.  Already
got Novell's eDirectory with production replicas on RH boxes.  And yes, I have
bought paid copies of RH since 7.0 _and_ pay for subscription to rhn.

I would just like to add that IMHO this IS a showstopper. I had RH8.0 running to
great satisfaction. Upgraded to RH9, to greater satisfaction still. The rhn
applet thingy told me a week or too later to upgrade certain glibc packages
(specifying precisely which) and according to the rhn pages this was a BUG which
had SECURITY consequences. So I picked up the rpm's which rhn recommended to me
and installed them and blew up my machine. As a newbie user and great redhat fan
I'm not very aware of the more subtle aspects of i386 vs i686 and I certainly
never heard about NPTL. I have two paid subscriptions for two other machines and
in this case I admit I was donwloading iso's and doing things by hand a) for the
learning experience and b) because I'm only using this machine for three months.

All in all it was a very useful learning experience for me but I hope also for
redhat.

*** Bug 90198 has been marked as a duplicate of this bug. ***

Let me get this straight...

The RH9 installer auto-detected my processor and chose the i686 rpm's for me,
right?  Is this a new feature of rh9, or have past versions done this as well?

Are there 686 versions of all rpm's or just a few key ones (like glibc & kernel)?

Now, assuming I do my updates manually, what is the algorithm I should use to
decide what updates to install?  Should I look in the i686 directory first, then
check the i386 directory (omitting anything that was also in the i686 directory)?

Can I force the installer to just use the i386 packages so I can avoid this mess
in the future?

How can I check the architecture of currently-installed packages?

It's not obvious how to check the architecture of installed pacakges.  You have
to use --queryformat

$ rpm -q --queryformat '%{NAME} %{VERSION} %{ARCH}\n'  emacs kernel
emacs 21.3 i386
kernel 2.4.20 i686

Regarding this solution:

***
------- Additional Comment #15 From Rob Cermak on 2003-04-11 16:01 -------

(1) post mortem using: linux recover

*snip
 
(b) rebooting: its doing fsck's on the journaled filesystems,
    seems like we are back in operation.
***

This solution worked perfectly for me.  My situation was a clean install of
Redhat 9 and an immeadiate updating of packages.  It went fubar, I found this
thread and used Rob's solution.  Rob is obviously a fscking genius.

Dear Redhat:  I will never ever pay for your products.  I will be migrating to
Slackware or Debian.  This bug has been found since early April and the update
packages you have at http://updates.redhat.com/9/en/os/i386/ still preserve the
bug for all time's sake apparently.  This is exactly what drove me from
Microsoft.  And getting into bed with IBM ain't gonna save you.

Rob Cermak's solution worked wonderfully for me.
I'd like to put on record here how the bug arose for
me. It was NOT through doing anything particularly 
stupid or non-standard.

The initial situation was glibc-2.3.2-11.9.i686.rpm,
as part of my RH9 (a smooth update from a clean install 
of RH8.0 two months previously, everything kept up2date).
The rhn applet advised me to update glibc-2.3.2-11.9 to 
glibc-2.3.2-27.9.  I found glibc-2.3.2-27-9 on rhn, 
there was no choice of architecture and the recommendation
was glibc-2.3.2-27-9.i386.rpm. In fact I did not even realise
the old version was i686 and anyway I had come to understand that
"i386" means "i386 and above" so even if I had noticed, I would not
have been concerned.

(Copied over from bug 96995, which is a duplicate of this bug)

RedHat has suffered from a dumbass attack, and indeed, this is a
problem with rpm in general.

1.) It is not known what architecture a package is compiled for once it is
installed (the real bug that rpm should detect and complain about). Example:

> rpm -qa | grep glibc
glibc-devel-2.3.2-5
glibc-common-2.3.2-5
glibc-kernheaders-2.4-8.10
glibc-2.3.2-5

No information at all about target architecture. Therefore, no one knows which
was installed.

This should be tracked, checked for, and rpm should protect people from it.

2.) replacing the i686 packages with the i386 packages SHOULD work, since
anything that can run i686 code can run i386 code. Shouldn't both libraries
export the same functions? If the app is statically linked, it doesn't care, and
if it's dynamically linked, it should just get the new functions at runtime.
Sure, the internal bits are different, but that's what data abstraction is for.

3.) I did not realize, and I wonder how many people actually do realize, that
different packages are installed based on architecture. I assumed that with this
release, like with the versions before it, everything was compiled for an i386.
This was obviously incorrect. However, it's news to many people.

*** Bug 90817 has been marked as a duplicate of this bug. ***

Matthew Caron, please do some research before you call someone a dumbass:

1) From comment #48
$ rpm -q --qf "%{name}-%{version}-%{release}-%{arch}\n" glibc

2) It's a bug.  It has been reported as a bug.  The maintainer has acknowledge 
the bug (comments #9 and #28, probably more).  Get over it.

3) According to updates.redhat.com there have be arch specific rpms (for the 
kernel if nothing else) since RH 6.0.

Take it to a mailing list.  Bugzilla isn't the place for this.  I hope Red Hat 
gets and excuse to update glibc pretty soon so they can close this.

I completed the fresh installation of RH9 and then updated all glibc-2.3.2-
11.9 including common,devel,etc also nptl with glib version 2.3.2-
27.9.i686.rpms and it working fine for me.

rpm -Uvh --replacefiles --replcapkgs --nodeps glibc-2.3.2-27.9.i686.rpms. 
 

Since RH still "stay calm" so far... If I want to have my own glibc update, how
can I do for the glibc-9.spec in
redhat/9/en/i386/updates/glibc-2.3.2-27.9.src.rpm when I rebuild that srpm ?
Which part shouild I modify so that it will not happen like this ?
Thanks.

Hi everyone,

After reading all the previous comments, 3 questions come to my mind:

1) Can we upgrade safely to the I386 library from the I686 library?
   (Hi want to support a park of I486 with NFS root)

2) Does the I386 contains the NPTL ?
   If not what are the application impacted by this ?

3) Could we get an option in the RedHat installer to force to
   installation of I386 library/package only?

Thanks for your help!!!

PS. 
I do not think that this bug should get a low Priority and a low Severity.
The glibc library is the fundation of most applications. 
Priority and Severity shoud be set to high for this bug.
Also, wether we use the I386 or the I686 library, application should run
identically.

From glibc bugfix errata: https://rhn.redhat.com/errata/RHBA-2003-136.html 
------------------
Solution

Before applying this update, make sure all previously released errata
relevant to your system have been applied.

To update all RPMs for your particular architecture, run:

rpm -Fvh [filenames]

where [filenames] is a list of the RPMs you wish to upgrade. Only those
RPMs which are currently installed will be updated. Those RPMs which are
not installed but included in the list will not be updated. Note that you
can also use wildcards (*.rpm) if your current directory *only* contains the
desired RPMs.

--------------

I would like to point out the fact that this DOES NOT MENTION i386 versus i686.
Sorry for yelling, but this problem seems to be a frequent problem (I just had
to fix it and was a little peeved that it had not been fixed even though I had
the same i686 -> i386 problem in redhat 7.1).  Any package that will make the
system completely inoperable if upgraded incorrectly should include a warning
somewhere about this problem.  Since rpm does not warn about this problem then
the errata for glibc should.  

In my humble opinion, one or both of two things should happen:

1. Every redhat glibc errata should have a statement along the lines of this:
*Warning:* updating from i686 -> i386 versions will cause the system to be
broken.  Use rpm -q --qf "%{name}-%{version}-%{release}-%{arch}\n" glibc
to determine the architecture of the installed glibc and use the same
architecture for the new version.

2. rpm should check if there is an architecture downgrade and should not do it
unless the user specifies something like --architecturedowngrade

Lastly, a bug that will keep bash and init from running until I physically go to
the machine and boot off of a cdrom is high severity so far as I am concerned.

To Jakub Jelinek:
When will you try to fix this bug ? It's been 4 months since
glibc-2.3.2-27.9.i386.rpm is released.
Please fix this asap, we do appreciate that.
Thanks.

Before installing glibc fixes from Errata, check the existing glibc versions and
the arch used:

rpm -qa --queryformat "%{arch}\t%{name}-%{version}-%{release}\n"

The following URL has pretty good recovery procedure, if you happened to install
grong glibc version and seg.faulted the whole system:

http://www.linuxquestions.org/questions/archive/5/2003/05/4/57607

It's all very well that if everything goes wrong and you can still get onto internet you can find 
out how to recover; and it's also all very well that the problem could have been avoided 
altogether by using a whole load of sophisticated options to rpm, but I think it's clear that  
this problem is still causing a lot of trouble to some unlucky new redhat users who were 
basically just doing things by the book ... and this means that something should be done  
about it! I recently perversely installed redhat 9 on a new machine and upgraded all  
the packages which were recommended by up2date and the same thing happended again... 
this time I knew what to do. I did use the redhat 9 distribution as it first came out, maybe 
the current iso's have some things changed... 

Hi!
The bug fix is shown in 2.3.2-28,
where Jakub Jelinek <jakub> wrote:
-------------------------------------------------------------
...
- remove /lib/i686 directory before running ldconfig in glibc post
  during i686 -> i386 glibc "upgrades" (#88456)
-------------------------------------------------------------
It's fixed, but the rpm is not in the RH9 updates.
Well, in that case, I am trying to use glibc in rawhide
(glibc-2.3.2-78.src.rpm) or severn (glibc-2.3.2-57.src.rpm) to rebuild
it, but failed. Both I got the same error:
-----------------------------------
...
++ sed 's/^linuxthreads-\([0-9.]*\) .*$/\1/' ../linuxthreads/Banner
sed: relocation error: ./libc.so.6: symbol __libc_stack_end, version
GLIBC_PRIVATE not defined in file ld-linux.so.2 with link time reference
+ gcc -static -L. -Os ../redhat/glibc_post_upgrade.c -o
glibc_post_upgrade -DARCH_386 '-DVERSION="2.3.2"' '-DPVERSION=""'
'-DGCONV_MODULES_CACHE="/usr/lib/gconv/gconv-modules.cache"'
gcc: relocation error: ./libc.so.6: symbol __libc_stack_end, version
GLIBC_PRIVATE not defined in file ld-linux.so.2 with link time reference
error: Bad exit status from /home/steven/rpmbuild/tmp/rpm-tmp.78959 (%build)


RPM build errors:
    Bad exit status from /home/steven/rpmbuild/tmp/rpm-tmp.78959 (%build)
-------------------------------------------------------------

Is that possible I can rebuild this fixed version in RH9 ?

Thanks.

This worked for me all fixed !!

Used knoppix, boot to runlevel 2, mkdir /newroot mounted /dev/hda3 /newroot
deleted all __db* on /newroot

$ export LD_ASSUME_KERNEL=2.2.5
$ su
Password: [enter root password]
# rpm -q glibc
glibc-2.3.2-11.9
glibc-2.3.2-27.9
# rpm -e glibc-2.3.2-11.9
                                                                    

# rpm -Uvh --replacepkgs --replacefiles glibc-*

This worked for me all fixed !!

Used knoppix, boot to runlevel 2, mkdir /newroot mounted /dev/hda3 /newroot
deleted all __db* on /newroot

$ export LD_ASSUME_KERNEL=2.2.5
$ su
Password: [enter root password]
# chroot /newroot rpm -q glibc
glibc-2.3.2-11.9
glibc-2.3.2-27.9
# chroot /newroot rpm -e glibc-2.3.2-11.9
                                                                    

# chroot /newroot rpm -Uvh --replacepkgs --replacefiles glibc-*

This is silly. Please fix the RPM so no more hosed systems occur.

I just spend two entire days working on a server, trying to figure out why a
"Segmentation fault" occured after a standard "rpm -Fvh *.rpm" command. How is a
user to know that first they go to the i686 directory for updates, then to the
i386 directoy (but not for the glibc packages lest you nuke your brand new
system), but only if your processor has been detected as i686 (which isn't
exactly obvious when issuing the "rpm -Fvh" command. Well actually, rpm doesn't
say anything about nuking your system if you're using i686 but trying to install
i386 (but how to know this anyways? Digital telepathy?))

Yes, this is a rant, rare for me. But two days of time lost to an error in a
critical RPM that was identified months ago is extremely frustrating.

*** Bug 107819 has been marked as a duplicate of this bug. ***

*** Bug 102569 has been marked as a duplicate of this bug. ***

Does Anyone know what other RPM Update that still have the similiar error like 
this. so i do not crashing my system the second time. Tankyu!

Jakub are asking for the testing glibc-2.3.2-27.9.3 (RHL9 errata candidate),
check this:
https://www.redhat.com/archives/shrike-list/2003-October/msg01417.html

The files are in
ftp://people.redhat.com/jakub/glibc/errata/2.3.2-27.9.3/

The new version for this glibc is at
ftp://people.redhat.com/jakub/glibc/errata/2.3.2-27.9.4/
I'd like to confirm I can "upgrade" i686 glibc to i386 version in RH9.
Check this:
--------------------------------------------------------------
[steven@rh9 steven]$ rpm -q --qf '%{arch}\n' glibc
i686
[steven@rh9 glibc]$ sudo rpm -Fvh
{glibc{,-profile,-common,-utils,-debug,-devel}
,nscd}-2.3.2-27.9.4.i386.rpm
Preparing...               
########################################### [100%]
   1:glibc-common          
########################################### [ 25%]
   2:glibc                 
########################################### [ 50%]
Stopping sshd:[  OK  ]
Starting sshd:[  OK  ]
   3:glibc-devel           
########################################### [ 75%]
   4:nscd                  
########################################### [100%]
[steven@rh9 glibc]$ rpm -q --qf '%{arch}\n' glibc                    
          
i386
--------------------------------------------------------------
After this, I reboot my machine, it seems everything is fine.
Nice going! Thank you, Jakub. Finally this "bug" is fixed, although it
exists over 6 months. Thanks again.

*** Bug 108756 has been marked as a duplicate of this bug. ***

Anybody still having problem with this with the errata version?  If
not, I'll close the bug soon.

Haven't heard about any problems, so I'm closing the bug now.

Same thing still happens with Fedora Core 1. Yesterday I made the 
mistake to update glibc from up2date and now my system is useless. 

Haven't yet tried #15, I hope it helps. Fedora Core is a clean 
install. I compiled custom 2.6.0 kernel though.

Also experienced the same problem with Fedora Core 1 after running 
up2date.

Jaska and Jason, even though the symptoms may be the same, it could be
a completely different bug. (Without more information it's hard to
know, however.)

I haven't seen the glibc upgrade via up2date on Fedora Core cause
anything like this on the computers I maintain at work...