Bug 1912598

Summary: btrfs, unable to mount following suspected write failure, leaf parent key incorrect, bad block
Product: [Fedora] Fedora Reporter: Chris Murphy <bugzilla>
Component: kernelAssignee: fedora-kernel-btrfs
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 33CC: acaringi, adscvr, airlied, bskeggs, hdegoede, itamar, jarodwilson, jeremy, jglisse, jonathan, josef, kernel-maint, lgoncalv, linville, masami256, mchehab, ptalbert, steved
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-01-14 08:36:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Chris Murphy 2021-01-04 21:37:50 UTC 
https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/CKYVNI7OP4ECK6JC242S3TYEP2NA3HEY/

kernel 5.9.x
HGST_HTS541010A9E680
btrfs on dm-crypt, bfq

User reports deleting some files from ~/ and soon after the fs goes ro, and now will not mount. Under 5.9.16-200.fc33.x86_64 the mount attempt:


[29867.234062] BTRFS info (device dm-4): disk space caching is enabled
[29867.234067] BTRFS info (device dm-4): has skinny extents
[29867.317955] BTRFS error (device dm-4): parent transid verify failed on 55640064 wanted 44146 found 44438
[29867.326701] BTRFS error (device dm-4): parent transid verify failed on 55640064 wanted 44146 found 44438
[29867.326727] BTRFS warning (device dm-4): failed to read root (objectid=9): -5
[29867.333668] BTRFS error (device dm-4): open_ctree failed

btrfs check --readonly
Opening filesystem to check...
parent transid verify failed on 55640064 wanted 44146 found 44438
parent transid verify failed on 55640064 wanted 44146 found 44438
parent transid verify failed on 55640064 wanted 44146 found 44438
Ignoring transid failure
Checking filesystem on /dev/mapper/dm_crypt
UUID: 7d9dbe1b-dea6-4141-807b-026325123ad8
[1/7] checking root items
parent transid verify failed on 55902208 wanted 44170 found 44438
parent transid verify failed on 55902208 wanted 44170 found 44438
parent transid verify failed on 55902208 wanted 44170 found 44438
Ignoring transid failure
parent transid verify failed on 56410112 wanted 44170 found 44439
parent transid verify failed on 56410112 wanted 44170 found 44439
parent transid verify failed on 56410112 wanted 44170 found 44439
Ignoring transid failure
parent transid verify failed on 58621952 wanted 44170 found 44439
parent transid verify failed on 58621952 wanted 44170 found 44439
parent transid verify failed on 58621952 wanted 44170 found 44439
Ignoring transid failure
ERROR: child eb corrupted: parent bytenr=178081497088 item=246 parent level=1 child level=2
ERROR: failed to repair root items: Input/output error
[2/7] checking extents
parent transid verify failed on 55902208 wanted 44170 found 44438
Ignoring transid failure
leaf parent key incorrect 55902208
bad block 55902208
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space cache
parent transid verify failed on 55902208 wanted 44170 found 44438
Ignoring transid failure
there is no free space entry for 81025253376-1775296512
there is no free space entry for 81025253376-81634787328
cache appears valid but isn't 80561045504
parent transid verify failed on 58621952 wanted 44170 found 44439
Ignoring transid failure
ERROR: child eb corrupted: parent bytenr=178081497088 item=246 parent level=1 child level=2
cache appears valid but isn't 81634787328
parent transid verify failed on 59359232 wanted 44170 found 44439
parent transid verify failed on 59359232 wanted 44170 found 44439
parent transid verify failed on 59359232 wanted 44170 found 44439
Ignoring transid failure
there is no free space entry for 173176602624-1736577024
there is no free space entry for 173176602624-173976584192
cache appears valid but isn't 172902842368
parent transid verify failed on 59850752 wanted 44170 found 44440
parent transid verify failed on 59850752 wanted 44170 found 44440
parent transid verify failed on 59850752 wanted 44170 found 44440
Ignoring transid failure
wanted bytes 15204352, found 49152 for off 177252646912
wanted bytes 1018904576, found 49152 for off 177252646912
cache appears valid but isn't 177197809664
[4/7] checking fs roots
parent transid verify failed on 59850752 wanted 44170 found 44440
Ignoring transid failure
leaf parent key incorrect 59850752
parent transid verify failed on 59850752 wanted 44170 found 44440
Ignoring transid failure
...snip many repeats...
leaf parent key incorrect 59850752
root 444 inode 2178 errors 2000, link count wrong
	unresolved ref dir 262 index 8 namelen 7 name .bashrc filetype 1 errors 0
	unresolved ref dir 189649 index 66 namelen 7 name .bashrc filetype 0 errors 3, no dir item, no dir index
root 444 inode 119301 errors 2001, no inode item, link count wrong
	unresolved ref dir 42133 index 324 namelen 5 name km_KH filetype 2 errors 4, no inode ref
root 444 inode 119302 errors 2001, no inode item, link count wrong
	unresolved ref dir 42133 index 325 namelen 3 name kmb filetype 2 errors 4, no inode ref
root 444 inode 119303 errors 2001, no inode item, link count wrong
	unresolved ref dir 42133 index 326 namelen 2 name kn filetype 2 errors 4, no inode ref
root 444 inode 119304 errors 2001, no inode item, link count wrong
	unresolved ref dir 42133 index 327 namelen 2 name ko filetype 2 errors 4, no inode ref
root 444 inode 119305 errors 2001, no inode item, link count wrong
...snip many repeats of the same "errors 4" for different files...
ERROR: errors found in fs roots
found 61374464 bytes used, error(s) found
total csum bytes: 0
total tree bytes: 229376
total fs tree bytes: 0
total extent tree bytes: 131072
btree space waste bytes: 109122
file data blocks allocated: 29622272
 referenced 29622272
Comment 1 Chris Murphy 2021-01-05 14:39:10 UTC 
 5.9.14-200.fc33.x86_64 at the time of the problem.
Comment 2 Josef Bacik 2021-01-07 14:53:25 UTC 
I've got patches for btrfsck to possibly fix this, but I want to make sure you've managed to scrape everything off the disk before you use it, just in case.  Have you done a btrfs restore -s /dev/whatever /dir?  Chris mentioned you had trouble with it, could you please paste the error you got?  Once you've scraped everything I'll point you at the branch that has the fsck changes I've made from your btrfs image and hopefully we can get you fixed up.
Comment 3 Chris Murphy 2021-01-14 08:36:26 UTC 
User previously reports ~2 months ago device-mapper checksum errors related to LVM thin metadata on the same drive. And also smartctl -x reports a bunch of write and read errors, including

  10 -- 41 05 40 00 00 60 d9 6e 08 00 00  Error: IDNF at LBA = 0x60d96e08 = 1624862216

which has various dire descriptions. But anyway internal read, and write errors especially, indicate serious drive problems.


Josef's btrfsck patches did fix the file system enough to mount, but the user has reported that the system won't boot (unclear what it gets tripped up on) and snapshots are missing.