Bug 1707822

Summary: fsck finds error on clean ext4 filesystems and prevent fedora 30 to boot
Product: [Fedora] Fedora Reporter: Pierre Juhen <pierre.juhen>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 30CC: airlied, bskeggs, esandeen, hdegoede, ichavero, itamar, jarodwilson, jeremy, jglisse, john.j5live, jonathan, josef, kasal, kernel-maint, kzak, lczerner, linville, mchehab, mjg59, oliver, steved
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-05-12 04:03:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Install log of Fedora 30
none
Fsck (script command) of home filesystem
none
Fsck (script command) of new_root filesystem none

Description Pierre Juhen 2019-05-08 13:22:01 UTC
Description of problem:

I have failed to install fedora 30 on 2 different PC that use bcache (SSD + HDD).
After installation, fsck in initramfs declares that the root file system is corrupted (clean flag is set in superblock, but journal contains data).

Therefore the system cannot boot.

Used manually, fsck find errors in ext4 filesystem (or create ones).

the only way I found to get back was a clean reinstall of Fedora 29.

The problem for me is clearly fsck, that finds unexisting erros, and eventually creates new ones.

This bug is critical !!!


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Eric Sandeen 2019-05-08 16:04:38 UTC
Please provide actual e2fsck output or other logs, so we know what is actually happening here.

Comment 2 Eric Sandeen 2019-05-08 16:10:16 UTC
We'll also need to know how you set up your storage, as it seems quite likely that the root cause may be storage related. If Fedora 30 failed to install on a standard storage setup, we'd have many more bug reports.

Comment 3 Pierre Juhen 2019-05-08 18:02:28 UTC
On both PCs, I use a bcache frontend in a SDD.

The bache backend is in one case case a hDD partition, in the other case a raid array (mdadm raid1)

Bcache devices are used as a physical volumes for an LVM volume group.

Then logical volumes are created in the volume group. There are all ext4 except a swap volume.

I agree with you, the problem comes probably from this complex approach. But it has been working perfectly for several years (at least since fc23), and no problem during upgrades.

It still works after going back to fc29 (home partition is OK, after apperaring corrupted with fsck/fc30).

Il will reinstall fc30 and send you logs.

Comment 4 Pierre Juhen 2019-05-09 04:54:53 UTC
I have big trouble in doing a fresh install of fedora 30 on my configuration. I have defined a simple configuration : a new logical volume (new_root), and I reuse /boot, /boot/efi.

Anaconda fails during storage definition (formating the new volume, new-root).

When I restart under fc29, the boot process stops in emergency mode, with the same issue, on /root, which shouldn't have been impacted :

fsck says the clean bit is set in the superblock, but the journal contains data. Running fsck (fc29) finds orphaned inodes, and inconsistencies in the bitmap (pass 5).

fsck (fc29) is able to correct everything, and restart, but there is a big big problem in my storage configuration, with fsck/mkfs (ext4) on fc30.

Comment 5 Eric Sandeen 2019-05-09 13:44:16 UTC
We're not going to be able to do much without actual logs.  I can't translate a verbal narrative of fsck behavior into the actual e2fsck code.  As far as I know, e2fsck never emits the message "clean bit is set in the superblock, but the journal contains data"

If anaconda is failing, please attach all of the anaconda logs.

Comment 6 Pierre Juhen 2019-05-09 19:51:09 UTC
Created attachment 1566387 [details]
Install log of Fedora 30

Comment 7 Pierre Juhen 2019-05-09 19:52:04 UTC
Created attachment 1566388 [details]
Fsck (script command) of home filesystem

Comment 8 Pierre Juhen 2019-05-09 19:53:10 UTC
Created attachment 1566389 [details]
Fsck (script command) of new_root filesystem

Sorry, it's in French

Comment 9 Pierre Juhen 2019-05-09 20:05:00 UTC
That's what I did :

1/ Try to install Fedora 30 from live USB 

2/ Failed after creating the ext4 fs on new_root, in the process of creation of swap -> See Install log

3/ mounted my home partition to save Install log and cleanly unmounted it

4/ rebooted under Fedora 29

5/ Boot process exited in emergency mode because home fs was not clean

6/ Fsck home filesystem -> See log

7/ Ended boot

8/ Fsck the new_root filesystem, created under Fedora 30, under Fedora 29 ; many errors, see log


Clearly there is a problem in my configuration : as soon a a filesystem is created / written in Fedora 30, it appears as corrupted.

Ps : that's what I saw many times on my first PC and yesterday :

/dev/....: recovering journal
Superblock needs_recovery flag is clear, but journal has data.
Run journal anyway? yes

See : https://ubuntuforums.org/showthread.php?t=2282368

Comment 10 Eric Sandeen 2019-05-09 20:17:17 UTC
The new root filesystem doesn't even have a proper magic value:

> ext2fs_open2: Numéro magique invalide dans le super-bloc

this really points to a storage problem, not an ext4 problem.

I would suggest installing without bcache; if that works, you know the problem lies with storage, and you'll need to find someone who understands bcache to help you with your bug.

Comment 11 Pierre Juhen 2019-05-12 04:03:55 UTC

*** This bug has been marked as a duplicate of bug 1708315 ***