Bug 521959

Created attachment 360132 [details]
dmesg

Created attachment 360133 [details]
boot.log

Created attachment 360134 [details]
lspci

(22:53:04) warren: nim-nim: add rdbreak=pre-pivot
(22:53:06) warren: nim-nim: boot it
(22:53:21) warren: nim-nim: look in the filesystem for the script that runs mdmon
(22:53:29) warren: nim-nim: run mdmon in the same manner, see if it segfaults
(22:54:16) warren: nim-nim: if so, ulimit -c unlimited; make it crash; #plug usb stick in; mkdir /tmp/whatever; mount /dev/dev/sdwhatever1 /tmp/whatever; copy it
22:55
(22:55:10) warren: nim-nim: with that core file, hopefully we can get a backtrace
(22:56:18) warren: nim-nim: hopefully ulimit is a dash built-in

For the advanced user, here is a scratch version to test:

# rpm -e '*dracut*' --nodeps
# rpm -ivh 'http://koji.fedoraproject.org/koji/getfile?taskID=1680533&name=dracut-001-10.git4d924752.fc12.noarch.rpm'

Created attachment 361116 [details]
dmesg with dracut-001-10.git4d924752.fc12.noarch.rpm

Well, this one does not work any better. It fails exactly the same way at the same place

If I look at the dmesg, it seems you did not recreate the initramfs image, with which you are testing...

dmesg says:
dracut: dracut-001-9.git6f0e469d.fc12

please recreate the initramfs with dracut like you would do with mkinitrd.

# dracut /boot/initramfs-<kernel version>.img <kernel version>

I did (with a -f and a rm of the old file even) but it seems I appached the wrong log (though the new one is hardly different)

That, or your rpm has the wrong code inside. I'll check this evening when I have acces to this system

Anyway the new log has the same errors, didn't notice anything new when looking at it. But I didn't try diff-ing

ok, please provide the output of:

# lsinitrd /boot/<image which fails> | grep dracut
# lsinitrd /boot/<image which fails> | grep mdadm.conf
# grep mdadm /etc/dracut.conf

and I really would like to see the new dmesg output

Well I made the mistake of updating rawhide before re-doing a new test, and now the system does not reboot at all anymore (the system drops into the "please unbork me if you're an admin" at fs check time

So before taking any more risk with my data I've rebooted from an usb key into a rescue session, to re-sync the raid (which has not been synced since the start of august when dracut started not assembling it correctly at boot time)

This way even if I crap one disk the other will still have a fresh data copy as it was always supposed to. I've been living too dangerously this past month (granted I was away a few weeks with the system off)

It should be almost done now, been running for about 90 min

Well it's done, and it craps itself just the same way it did 2 hours ago before I decided to create a rescue flash

The main change since the posted dmesg is udev logging '/sbin/mdmadm --detail --export /dev/dm127' unexpected exit with status 0x000b

Could be the new mdadm version in koji today. Or the new selinux that does not block anymore some mdadm files. If ails the same way with old kernels, so it's not something new in the initramfs itself

If I remove the /boot /dev/md0 mount in fstab the system boots

But I can assemble md0 post-boot, mdadm is wedged in a strange state and does not accept any md manipulation (md127 is live with / over lvm on it, as dracut assembled it, though)

So I can't change the initramfs from rawhide anymore :(

Next try: reboot in rescue mode and chroot / from there. The F11 rescue disk does not crap over md like dracut

well, actually I had the bright idea to downgrade mdadm first (since it does not need /boot access to install) that restored the system to yesterday's level of breakage (boots, but only half the array under / is assembled)

seems mdadm-3.0-3.fc12.x86_64 is bad mojo

(In reply to comment #8)
> please recreate the initramfs with dracut like you would do with mkinitrd.
> 
> # dracut /boot/initramfs-<kernel version>.img <kernel version>  

Oh, now I see why it didn't work, I reused and old dracut command, and it created and initrd-generic- file instead of an initramfs- file

An this time it seems to work! Yahoo!

Except, I managed to re-break the md in the meanwhile, so I can't check if the full stop => boot cycle works 100%. I'll need to wait before the disks are re-synced to test a full cycle in "clean" conditions

Anyway, just to be complete

grep mdadm /etc/dracut.conf  
# install local /etc/mdadm.conf
mdadmconf="yes"


New working initramfs:

lsinitrd /boot/initramfs-2.6.31-17.fc12.x86_64.img |grep dracut
-rw-r--r--   1 root     root           31 Sep 17 00:07 dracut-001-10.git4d924752.fc12
-rw-r--r--   1 root     root         2675 Sep 15 15:54 lib/dracut-lib.sh
init
. /lib/dracut-lib.sh
    echo "file a bug against dracut."

lsinitrd /boot/initramfs-2.6.31-17.fc12.x86_64.img |grep mdadm.conf
-rw-r--r--   1 root     root          164 Jul  4 16:38 etc/mdadm.conf
init

Old failing initramfs

lsinitrd /boot/initramfs-2.6.31-14.fc12.x86_64.img |grep dracut
-rw-r--r--   1 root     root           30 Sep 15 18:33 dracut-001-9.git6f0e469d.fc12
-rw-r--r--   1 root     root         2540 Sep  9 19:50 lib/dracut-lib.sh
. /lib/dracut-lib.sh
init

lsinitrd /boot/initramfs-2.6.31-14.fc12.x86_64.img |grep mdadm.conf
init

Created attachment 361388 [details]
dmesg with dracut-001-10.git4d924752.fc12.noarch

new working dmesg

(In reply to comment #18)
> An this time it seems to work! Yahoo!
> 
> Except, I managed to re-break the md in the meanwhile, so I can't check if the
> full stop => boot cycle works 100%. I'll need to wait before the disks are
> re-synced to test a full cycle in "clean" conditions  

And cycling the system works too. The raid is still in a sane unbroken state

if it works please close the bug

Please test dracut-001-12.git0f7e10ce.fc12.
Either wait for it to appear in rawhide or do:
# yum install koji
# cd $(mktemp -d)
# koji download-build 132403
# rpm -Fvh *.rpm

and recreate the image with

# dracut /boot/<image> <kernel version>

Note: in recent installs the <image> is named initramfs-<kernel version>.img

Created attachment 361532 [details]
dmesg with dracut-002-1.fc12

I assume you're more interested in dracut-002-1.fc12 and yes it works