Bug 523378

System actually complains about a missing library /usr/lib64/libfreebl3.so.

Root volume /dev/VolGroup00/LogVol00 has file system type ext4.

After a fresh install from the "rawhide" tree using the 2009-08-18 boot.iso due to anaconda crashes of the most recent versions, no initrd entry is present in /etc/grub.conf, and subsequently, the kernel panics upon the next reboot.
After booting into rescue mode, chrooting and installing the latest device-mapper, dracut, lvm2 and kernel packages from Koji, the new kernel entry in /etc/grub.conf looks correct, and the system seems to boot as expected.
However, shortly after starting udev, the system reports "Unexpected inconsistencies of file system", and then "Automatic reboot in progress".
Even running fsck.ext4 on /dev/VolGroup00/LogVol00 has no effect: after rebooting the system, the same error messages about inconsistencies reappear.
There seems to be no way to rescue the system other than intalling it from scratch, possibly using ext3 instead of ext4 .. :o(

Created attachment 363426 [details]
boot.log after forced reboot due to file system inconsistencies

Because of temporary problems with the power supply, the system happens to power off fairly frequently. Here is the typical boot log after the forced reboot triggered by the kernel after file inconsistencies have been detected which happens every time after a power cut. They always appear on the root [ext4] volume, never on /usr [ext4] or /home [ext3] volumes.

Comment on attachment 363426 [details]
boot.log after forced reboot due to file system inconsistencies

Welcome to Fedora 
Press 'I' to enter interactive startup.
Starting udev: [ OK ]
Setting hostname banach:  [ OK ]
Setting up Logical Volume Management:   4 logical volume(s) in volume group "VolGroup00" now active  [ OK ]
Checking filesystems
/dev/mapper/VolGroup00-LogVol00 contains a file system with errors, check forced.
/dev/mapper/VolGroup00-LogVol00: |=                               |  2.2%
/dev/mapper/VolGroup00-LogVol00: |=                               /  4.4%
/dev/mapper/VolGroup00-LogVol00: |======                          - 19.7%
/dev/mapper/VolGroup00-LogVol00: |========================        \ 74.9%
/dev/mapper/VolGroup00-LogVol00: |===========================     | 85.7%
/dev/mapper/VolGroup00-LogVol00: |================================| 100.0%
/dev/mapper/VolGroup00-LogVol00: 25890/262144 files (0.4% non-contiguous), 443960/1048576 blocks
/dev/sda2: recovering journal
/dev/sda2: clean, 36/26208 files, 26549/104420 blocks
/dev/mapper/VolGroup00-LogVol02: recovering journal
/dev/mapper/VolGroup00-LogVol02: clean, 79288/6635520 files, 13493266/26533888 blocks
/dev/mapper/VolGroup00-LogVol01: recovering journal
/dev/mapper/VolGroup00-LogVol01: clean, 266965/524288 files, 1648210/2097152 blocks  [ OK ]
Remounting root filesystem in read-write mode:  [ OK ]
Mounting local filesystems:  [ OK ]
Enabling local filesystem quotas:  [ OK ]
Enabling /etc/fstab swaps:  [ OK ]
Entering non-interactive startup
Starting monitoring for VG VolGroup00:   4 logical volume(s) in volume group "VolGroup00" monitored  [ OK ]

After a shutdown triggered by pressing the soft power button while investigating bug 526433, the system does not recover from the file system inconsistencies anymore. It always proceeds to the automatic reboot step and is hence caught in an endless loop of reboots.
In the past I had already tried to run fsck.ext4 in rescue mode but after rebooting the system from the system disk, the same inconsistencies were reported. The hardware profile of the affected system can be found at

http://www.smolts.org/client/show/pub_ce3d646e-2918-44c9-a6c0-10fb3d9a0180

I should add that the system lives on a Hitachi Deskstar T7K250 160GB PATA disk which is connected to the onboard nVidia Corporation CK804 IDE interface.

(In reply to comment #1)
> System actually complains about a missing library /usr/lib64/libfreebl3.so.  

From the nss-softokn-freebl package.

The issue has been confirmed for two other systems, thus no faulty hardware. It might be worthwile checking the ext4 default settings. It seems that recent kernels are a bit too aggressive regarding maximizing disk I/O performance.

Apart from the  libfreebl3.so issue which I haven't seen this reads to me like bug 522969. Are you east of UTC ?

(In reply to comment #10)
Correct, local time zone is CEST (UTC+2h). I have to repeat that this issue is fairly recent, maybe 1-2 months old. It does not occur for current F11 even when the root volume has file type ext4.

Btw, before the automatic reboot is triggered with the announcement,

  "Automatic reboot in progress."

there -is- a message

  "Superblock last write time is in the future"

And yes, the hardware clock is set to the local time zone CEST.

Regarding the superblock write time in the future, it should be fixed as of bug 522969 's resolution in rawhide & f12, so you might update that to get that annoyance out of the way.

Other than that, there have been other reports of corruption problems in ext4 upstream, if you have a repeatable testcase we may have a patch you could try if you're handy with such things; it's on my list of things to look into today.

Thanks,
-Eric

*** This bug has been marked as a duplicate of bug 522969 ***

Joachim, so were no other errors found during fsck?

No, situation has evolved toward the normal procedure after an unclean reboot:

Setting up Logical Volume Management:   4 logical volume(s) in volume group "VolGroup00" now active
                                                           [  OK  ]
Checking filesystems
/dev/mapper/VolGroup00-LogVol00: clean, 26468/262144 files, 293417/1048576 blocks
/dev/sda2: clean, 36/51200 files, 29848/204800 blocks
/dev/mapper/VolGroup00-LogVol02: clean, 81179/6635520 files, 11247838/26524672 blocks
/dev/mapper/VolGroup00-LogVol01: clean, 279431/524288 files, 1713916/2097152 blocks
                                                           [  OK  ]
Remounting root filesystem in read-write mode:             [  OK  ]
Mounting local filesystems:                                [  OK  ]
Enabling local filesystem quotas:                          [  OK  ]
Enabling /etc/fstab swaps:                                 [  OK  ]
Entering non-interactive startup
Starting monitoring for VG VolGroup00:   4 logical volume(s) in volume group "VolGroup00" monitored

Just adding to this report as I've seen these exact symptoms after upgrading my laptop from F11 to the Beta tonight, including the complaint about a missing libfreebl3.so.

I will try installing the updated e2fsprogs via the rescue image.

It's now also complaining that it's unable to write some udev-related files, before the "Welcome to Fedora" text appears.