Red Hat Bugzilla – Bug 163407
mkinitrd needs update to fix race condition in sh
Last modified: 2007-11-30 17:11:10 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.9) Gecko/20050711 Firefox/1.0.5
Description of problem:
did upgrade via yum, all went well. Rebooted to load new kernel and got panic. Was able to catch grub on next boot and booted old version no problem.
This happened on both smp servers, but worked fine on the single cpu boxes.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1.make kernel-smp-2.6.12-1.1372_FC3 the default boot kernel and reboot
Actual Results: kernel panic not syncing
Expected Results: system should have booted.
we'll need more of the messages to be able to diagnose this.
1. make sure you don't have 'quiet' in your boot command line in grub
2. adding vga=791 will put your console into a higher resolution so you get more
lines of text
Even if you have to resort to attaching a digital camera picture, it'd help.
As it stands, there's nothing in this report to work with. Sorry.
Created attachment 116826 [details]
Picture of boot panic screen
From YUM :
Install: kernel.i686 0:2.6.12-1.1372_FC3 - updates-released
Install: kernel-smp.i686 0:2.6.12-1.1372_FC3 - updates-released
Total download size: 28 M
(1/2): kernel-2.6.12-1.13 100% |=========================| 14 MB 00:22
(2/2): kernel-smp-2.6.12- 100% |=========================| 14 MB 00:22
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Installing: kernel-smp 100 % done 1/2
Installing: kernel 100 % done 2/2
Installed: kernel.i686 0:2.6.12-1.1372_FC3 kernel-smp.i686 0:2.6.12-1.1372_FC3
As noted previously, this only happens on the SMP boxes, the single cpu boxes
are fine. All are supermicro motherboards with on board ati video, scsi,
Let me know what else you need. The attached picture shjows most of the boot.
this looks like part of your initrd is missing. (In particular the jbd part of ext3)
what does your /boot/initrd-2.6.12-1.1372_FC3 look like (ls -l)
Can you run (as root)
mkinitrd /boot/initrd-2.6.12-1.1372_FC3 2.6.12-1.1372_FC3
and try to boot again ?
> ls -l /boot/initrd-2.6.12-1.1372_FC3
> ls: /boot/initrd-2.6.12-1.1372_FC3: No such file or directory
> mkinitrd /boot/initrd-2.6.12-1.1372_FC3 2.6.12-1.1372_FC3
>ls -l /boot/initrd-2.6.12-1.1372_FC3
-rw-r--r-- 1 root root 978581 Jul 15 20:32 /boot/initrd-2.6.12-1.1372_FC3
rebooted and got the same exact same panic.
rebooted again same panic
rebooted 3rd time caught grub and loaded older version, worked fine.
argh, my bad. that command line should be..
mkinitrd /boot/initrd-2.6.12-1.1372_FC3.img 2.6.12-1.1372_FC3
(I missed off the .img)
You can delete the /boot/initrd-2.6.12-1.1372_FC3 that got created after the
just to be sure we are working with the same environemnt, I removed then
re-installed kernel. Verified all old .1372 files were removed before
re-installing. Ran the following command which produced and error :
> mkinitrd /boot/initrd-2.6.12-1.1372_FC3.img 2.6.12-1.1372_FC3
/boot/initrd-2.6.12-1.1372_FC3.img already exists.
you can safely rm or mv it out of the way, the mkinitrd command will recreate it.
Since this is an smp box, I ran you command as well as modifying it slightly to
regen the smp file also :
mkinitrd /boot/initrd-2.6.12-1.1372_FC3smp.img 2.6.12-1.1372_FC3smp
Tried a reboot and got the exact same panic again
interesting the new build is smaller than the old one.
drwxr-xr-x 4 root root 4096 Jul 16 07:39 .
drwxr-xr-x 23 root root 4096 Jul 16 07:43 ..
-rw-r--r-- 1 root root 55504 Jun 13 01:10 config-2.6.11-1.35_FC3
-rw-r--r-- 1 root root 55276 Jun 13 01:36 config-2.6.11-1.35_FC3smp
-rw-r--r-- 1 root root 56834 Jul 15 01:18 config-2.6.12-1.1372_FC3
-rw-r--r-- 1 root root 56701 Jul 15 01:46 config-2.6.12-1.1372_FC3smp
drwxr-xr-x 2 root root 1024 Jul 15 23:03 grub
-rw-r--r-- 1 root root 1162633 Jun 25 10:38 initrd-2.6.11-1.35_FC3.img
-rw-r--r-- 1 root root 1143297 Jun 25 10:38 initrd-2.6.11-1.35_FC3smp.img
-rw-r--r-- 1 root root 978577 Jul 16 07:29 initrd-2.6.12-1.1372_FC3.img
-rw-r--r-- 1 root root 968390 Jul 16 07:39 initrd-2.6.12-1.1372_FC3smp.img
drwx------ 2 root root 12288 Nov 20 2004 lost+found
-rw-r--r-- 1 root root 747046 Jun 13 01:10 System.map-2.6.11-1.35_FC3
-rw-r--r-- 1 root root 774796 Jun 13 01:36 System.map-2.6.11-1.35_FC3smp
-rw-r--r-- 1 root root 760080 Jul 15 01:18 System.map-2.6.12-1.1372_FC3
-rw-r--r-- 1 root root 790080 Jul 15 01:46 System.map-2.6.12-1.1372_FC3smp
-rw-r--r-- 1 root root 1635072 Jun 13 01:10 vmlinuz-2.6.11-1.35_FC3
-rw-r--r-- 1 root root 1553144 Jun 13 01:36 vmlinuz-2.6.11-1.35_FC3smp
-rw-r--r-- 1 root root 1652295 Jul 15 01:18 vmlinuz-2.6.12-1.1372_FC3
-rw-r--r-- 1 root root 1575017 Jul 15 01:46 vmlinuz-2.6.12-1.1372_FC3smp
OK, here's a question: does everyone with this problem have ATI chipsets? I
happen to have a whole load of ASUS Pundit-R machines lying around my office, so
that's what I've been doing all of my testing on. They have ATI integrated
everything. Unfortunately they're all at work and all of them are dead due to
this update, so I can't provide lspci or lsmod output right now.
Is anyone seeing this problem on a machine that does not have an ATI chipset?
I can provide full access to one of these boxes if it would help. Heck, I'll
even overnight one to RedHat if necessary.
I have many, many boxes with ATI video chipsets, it's only the dual cpu boxes
that have the problem, the single cpu boxes are just peachy with this update.
These are all supermicro mb's with the scsi 160/320 [adaptec] and intel 10/100
or 1000 and the ATI stuff built in.
This is a me too - smp server fails to boot, ati graphics. Have rebuilt initrd
(as per comment 5) for smp kernel and will try again.
Hi, seeing this with SMP kernel only on Dell PE2650, dual cpu, PERC SCSI raid,
ATI rageXL. The non SMP boots (but selinux policy seems to be blocking various
apps like: yum, mozilla + others). Boot is not helped by removing rhgb + quiet
and adding acpi=off noacpi (can't remember which is the correct param :)
Also, seeing it on a Dell PE750, dual cpu, 2x SATA drives (software raid), ATI
rageXL. SMP fails at (about) the same spot as posted error image. This does
start SMP if I remove rhgb + quiet, and add acpi=off noacpi.
Created attachment 116866 [details]
kernel-smp-2.6.12-1.1372_FC3 panic during boot
Okay, using the newly created initrd made no difference. Using the
uni-processor kernel worked fine (as reported by others).
However, I'm not sure this is the same problem. I've attached the last part of
the panic output I got when trying to boot and it doesn't report anything about
unknown symbols, in fact jbd.ko loads fine. It seems to have trouble locating
the root device. It all works fine with the kernel-smp-2.6.11-1.35_FC3 kernel.
If you want me to submit this as a different bug let me know.
Same here, all of 1.137 panic during boot on 5 different boxes with HT
processors. Booting the up kernel works; booting the smp kernel fails to mount
the root fs after failing to load ata_piix.ko, ext3.ko and dm-mirror.ko. All
boxes have Intel D865 mobos.
I'm pretty sure this is the same as bug 160652
It's hard to be certain without seeing some more context from the kernel panic
in comment #13... There were some other initrd bugs with scsi modules loading
but I think those have been fixed. So I'm pretty sure this is a dup of bug 160652
BTW. The solution is to downgrade to the last working kernel until a new
mkinitrd package is released for FC3. It's a mkinitrd bug really, not a kernel bug.
A lot of people have been hit by this so it won't take long.
This is definitely not a bug in mkinitrd. I was able to boot the SMP kernel
using the apparently-broken initrd by simply disabling HT in the BIOS
configuration. It sounds more like a race condition in the kernel.
FWIW, the latest FC4 kernel (1.1398) boots correctly on these boxes, after an
upgrade to FC4 (or rather full install in separate partitions, to preserve FC3
just in case).
1) Re comment 15 (re comment 13 output) - sorry but all I had was pen/paper to
write down the last part of the output. Being a 2x3GHz 2GB memory system you can
imagine that the rest of the output flies off the top of the (14" monitor)
screen extremely quickly! I'll see what the 'vga=791' boot option does.
2) Re comment 17 - I disabled HT in the bios but it still didn't work. Oddly
enough the first reboot (with HT disabled) *did* show a load of unresolved
symbol errors in the ext3 module and then paniced. Upon rebooting again there
were no such errors and jdb.ko and ext3.ko loaded fine, but it still had
problems finding/mounting root (as per comment 13). I'm back running on the
I have a dual opteron CPU machine running FC3 and
With the recent upgrade to new FC3 2.6.12 kernels, i encounter the
Only the smp version crashes (vmlinuz-2.6.12-1.1372_FC3smp). The single
processor kernel (vmlinuz-2.6.12-1.1372_FC3) boots in properly.
AMD Opteron(tm) Processor 246
Creating root device
umount /sys failed: 16
Mounting root filesystem
mount: error 19 mounting xfs
mount: error 2 mounting none
switch to new root
switchroot: mount failed: 22
umount /initrd /dev/ failed: 2
kernel panic - not syncing: Attempted to kill init!
Similar situation here.
$ uname -a
Linux machine 2.6.12-1.1372_FC3smp #1 SMP Fri Jul 15 01:08:54 EDT 2005 x86_64
x86_64 x86_64 GNU/Linux
but from other discussions I knew to precede the smp-kernel
install with an install of the latest available mkinitrd
(built from source)
$ mkinitrd --version
mkinitrd: version 4.2.18
I was skeptical, but it worked. No joy on dual-IA32 machines, yet.
Peter, for more info on this, see Prarit's mail archived in this thread here..
It's quite likely that 163437 is also a dupe of this bug.
*** Bug 164358 has been marked as a duplicate of this bug. ***
*** Bug 163917 has been marked as a duplicate of this bug. ***
*** Bug 163308 has been marked as a duplicate of this bug. ***
*** Bug 163448 has been marked as a duplicate of this bug. ***
*** Bug 162859 has been marked as a duplicate of this bug. ***
An update to mkinitrd just got pushed out to fc3-updates-testing.
After installing this update, remove the latest kernel update, and reinstall it.
The initrd should then be recreated correctly. Hopefully closing a slew of these
The update has fixed the problem for me.
Fixed my problem on a small 5 node dual-P4 Xeon test cluster.
For those of us who saw the warning *before* attempting an upgrade (and
therefore have not yet installed the 2.6.12 kernel yet), what is the proper
upgrade path? Can we just run up2date and have everything work now? (Possibly
after waiting for the new mkinitrd to escape testing -- when will that happen
and what's its version number so we will recognize it?) Or do we need to update
mkinitrd first in one up2date session, and the kernel in a second one?
Please be explicit about what to do to cause this to happen or point to a URL
that describes the process. I'm sorry if I am the only one that does not know
all the specific commands and implications of these steps.
After installing this update, // with up2date?
remove the latest kernel update, // with what command? rm initrd-*? rm vmlinuz-*?
and reinstall it. // with up2date?
This is better discussed on the mailing list. Still:
If you're not up to pulling from the testing repository then perhaps it's best
to wait until the update is actually released. But if you can't wait, then
(assuming you are on i386 and not x86_64) and also grab the latest kernel from
Then install the new mkinitrd:
rpm -Fvh mkinitrd-22.214.171.124-1.i386.rpm
Then uninstall the bad kernel:
rpm -e kernel-smp-2.6.12-1.1372_FC3
and install it again:
rpm -ivh kernel-smp-2.6.12-1.1372_FC3.i686.rpm
then reboot and enjoy.
This fixed it for me. Supermicro dual intel xeon mb with ati / adaptec scsi /
intel gig ethernet ...
Thank you !
Confirmed. This fixed it for me too using mkinitrd-126.96.36.199-1 rpm from testing.
System is an HP DL360 (dual xeon cpus).
this now got moved to -final updates.
Thanks for testing folks, and apologies for the time this took to get fixed.
The new mkinitrd will be installed with up2date. However, the kernel must still
be erased and installed in order for the change to be complete.
rpm -e kernel-smp-2.6.12-1.1372_FC3
and install it again:
rpm -ivh kernel-smp-2.6.12-1.1372_FC3.i686.rpm
*** Bug 164709 has been marked as a duplicate of this bug. ***
*** Bug 164604 has been marked as a duplicate of this bug. ***
*** Bug 151176 has been marked as a duplicate of this bug. ***
*** Bug 162800 has been marked as a duplicate of this bug. ***
I had same trouble in updating from 2.6.9 to 2.6.12-1.1456_FC4.src.rpm.
Insmod error : ata-piix.ko -1 unknown symbol in module
mkinitrd version is 4.1.18-2.i386.
M/C configuration is
MOtherboard Supermicro P8Sci
CPU P4 3Ghz, DDR2 1GB, SATA 160GB
Anyone can help me?