Description of problem: I have a corrupted partition (luks-Music from bug #510823) which I wanted to repair. The errors reported by xfs_check were: extent count for ino 7723 data fork too low (6) for file format agi unlinked bucket 25 is 18967193 in ag 3 (inode=421620377) link count mismatch for inode 151967493 (name ?), nlink 2, counted 3 link count mismatch for inode 421620377 (name ?), nlink 0, counted 1 I run xfs_repair on this partition and then xfs_check to double check. Unfortunately xfs_check reported some errors. Version-Release number of selected component (if applicable): xfsprogs-3.0.3-2.fc12.x86_64.rpm How reproducible: One time Steps to Reproduce: 1. Get a corrupted partition. 2. Run xfs_repair 3. Run xfs_check Actual results: link count mismatch for inode 7723 (name ?), nlink 2, counted 3 Expected results: No errors. Additional info: The metadump of this partiton can be found here https://bugzilla.redhat.com/attachment.cgi?id=380368&action=edit Also, a second xfs_repair will fix 100% the partition. xfs_check won't report any errors after this.
This message is a reminder that Fedora 12 is nearing its end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 12. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '12'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 12's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 12 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug to the applicable version. If you are unable to change the version, please add a comment here and someone will do it for you. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Fedora 12 changed to end-of-life (EOL) status on 2010-12-02. Fedora 12 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. Thank you for reporting this bug and we are sorry it could not be fixed.
Still a problem with xfsprogs 3.1.1: [root@inode tmp]# xfs_repair xfs.img Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... agi unlinked bucket 25 is 18967193 in ag 3 (inode=421620377) - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 extent count for ino 7723 data fork too low (6) for file format bad data fork in inode 7723 cleared inode 7723 - agno = 1 - agno = 2 7fb5a3fff710: Badness in key lookup (length) bp=(bno 182497856, len 16384 bytes) key=(bno 182497856, len 8192 bytes) - agno = 3 - agno = 4 - agno = 5 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 entry "n>\Cm YmI;{aYZ3}K|n9\fRv0" at block 0 offset 48 in directory inode 7722 references free inode 7723 clearing inode number in entry at offset 48... - agno = 2 - agno = 3 - agno = 4 - agno = 5 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... bad hash table for directory inode 7722 (no data entry): rebuilding rebuilding directory inode 7722 - traversal finished ... - moving disconnected inodes to lost+found ... disconnected dir inode 421620377, moving to lost+found Phase 7 - verify and correct link counts... resetting inode 421620377 nlinks from 0 to 2 done [root@inode tmp]# xfs_repair xfs.img Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... resetting inode 7723 nlinks from 2 to 3 done [root@inode tmp]# xfs_repair xfs.img Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... done
Current upstream still fails as well.
Funky; inode 7723 starts out as some file on the filesystem with problems, and it is cleared: extent count for ino 7723 data fork too low (6) for file format bad data fork in inode 7723 cleared inode 7723 But after the first repair, the newly-created lost+found is inode 7723, with the wrong link count after a file is added to it... strange. I think what's happened is that the inode was in the avl trees and got cleared, but not removed from the avl trees. lost+found gets recreated with that same inode number, and it's found in the avl tree already, with the old link count of 1, rather than what it should have had ...
That bug is also eating me on Fedora whatever version (Yes, I should fill another bug report, when time permit). Cheers.
No need to file another bug if this one suffices. Sorry we haven't gotten to this one, it's a bit of a matter of triage. But I may have a willing victim to take a look ...
The main pb is that xfs_repair left the FS in an inconsistent state with data losses (I've encountered it with git and have lost remote commits). Once done, on Fedora 15, XFS module trigger Oopses in a very reliable way. Kernel 2.6.40.6 I do not run XFS on sensible date since (and because I do not have time to debug it) Oct 27 08:56:22 nemesis kernel: [25675.950140] kernel BUG at fs/buffer.c:3234! Oct 27 08:56:22 nemesis kernel: [25675.950142] invalid opcode: 0000 [#1] SMP Oct 27 08:56:22 nemesis kernel: [25675.950144] CPU 2 Oct 27 08:56:22 nemesis kernel: [25675.950146] Modules linked in: tcp_lp fuse ecryptfs ip6table_filter ip6_tables rmd160 ebtable_nat ebtables crypto_null camellia lzo cast6 cast5 defl ate zlib_deflate cts gcm ccm serpent blowfish twofish_generic twofish_x86_64 twofish_common xcbc sha256_generic sha512_generic des_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunn el tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key ipt_MASQU ERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_CHECKSUM iptable_mangle bridge stp llc tpm_infineon ppdev parport_pc lp parport vboxpci vboxnetadp v boxnetflt vboxdrv cpufreq_ondemand acpi_cpufreq mperf capi kernelcapi nfs lockd fscache auth_rpcgss nfs_acl sunrpc bnep bluetooth rfkill coretemp snd_hda_codec_idt nvidia(P) snd_hda_i ntel snd_ice1712 snd_ice17xx_ak4xxx snd_hda_codec snd_ak4xxx_adda snd_cs8427 snd_ac97_codec ac97_bus snd_i2c snd_ Oct 27 08:56:22 nemesis kernel: mpu401_uart snd_hwdep snd_seq snd_pcm snd_timer xfs snd_page_alloc iTCO_wdt iTCO_vendor_support snd_rawmidi snd_seq_device snd soundcore e1000e i2c_i801 i2c_core joydev serio_raw microcode virtio_net kvm_intel kvm ipv6 usb_storage uas firewire_ohci firewire_core crc_itu_t video [last unloaded: scsi_wait_scan] Oct 27 08:56:22 nemesis kernel: [25675.950213] Oct 27 08:56:22 nemesis kernel: [25675.950215] Pid: 35, comm: kswapd0 Tainted: P 2.6.40.6-0.fc15.x86_64 #1 /DG45ID Oct 27 08:56:22 nemesis kernel: [25675.950219] RIP: 0010:[<ffffffff8114a82c>] [<ffffffff8114a82c>] free_buffer_head+0x16/0x33 Oct 27 08:56:22 nemesis kernel: [25675.950226] RSP: 0018:ffff88012145ba20 EFLAGS: 00010287 Oct 27 08:56:22 nemesis kernel: [25675.950228] RAX: ffff880082318048 RBX: ffff880082318000 RCX: ffff880082318000 Oct 27 08:56:22 nemesis kernel: [25675.950230] RDX: 0000000000000000 RSI: ffff880082318000 RDI: ffff880082318000 Oct 27 08:56:22 nemesis kernel: [25675.950232] RBP: ffff88012145ba20 R08: ffffea0000636de8 R09: ffff880082334000 Oct 27 08:56:22 nemesis kernel: [25675.950233] R10: ffffffff8114a83a R11: 0000000000015788 R12: ffff8800bb6972e0 Oct 27 08:56:22 nemesis kernel: [25675.950235] R13: ffff8800bb697190 R14: ffff8800bb6972e0 R15: ffff88012145bc08 Oct 27 08:56:22 nemesis kernel: [25675.950237] FS: 0000000000000000(0000) GS:ffff88012bd00000(0000) knlGS:0000000000000000 Oct 27 08:56:22 nemesis kernel: [25675.950239] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b Oct 27 08:56:22 nemesis kernel: [25675.950241] CR2: 00007f9c044ddef0 CR3: 00000000acab0000 CR4: 00000000000406e0 Oct 27 08:56:22 nemesis kernel: [25675.950243] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Oct 27 08:56:22 nemesis kernel: [25675.950245] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Oct 27 08:56:22 nemesis kernel: [25675.950247] Process kswapd0 (pid: 35, threadinfo ffff88012145a000, task ffff880124471730) Oct 27 08:56:22 nemesis kernel: [25675.950248] Stack: Oct 27 08:56:22 nemesis kernel: [25675.950250] ffff88012145ba60 ffffffff8114a8de ffff88012145bb78 ffffffff00000001 Oct 27 08:56:22 nemesis kernel: [25675.950253] 00000000ffffffdc ffff880082318000 ffffea0000636de8 ffffea0000636dc0 Oct 27 08:56:22 nemesis kernel: [25675.950257] ffff88012145baa0 ffffffffa01daf92 ffff88012145ba80 0000000000000000 Oct 27 08:56:22 nemesis kernel: [25675.950260] Call Trace: Oct 27 08:56:22 nemesis kernel: [25675.950263] [<ffffffff8114a8de>] try_to_free_buffers+0x95/0xa7 Oct 27 08:56:22 nemesis kernel: [25675.950296] [<ffffffffa01daf92>] xfs_vm_releasepage+0x79/0x92 [xfs] Oct 27 08:56:22 nemesis kernel: [25675.950300] [<ffffffff810dc007>] try_to_release_page+0x34/0x3d Oct 27 08:56:22 nemesis kernel: [25675.950304] [<ffffffff810e9fd1>] shrink_page_list+0x547/0x6f4 Oct 27 08:56:22 nemesis kernel: [25675.950308] [<ffffffff81120cf6>] ? mem_cgroup_del_lru+0x1d/0x21 Oct 27 08:56:22 nemesis kernel: [25675.950311] [<ffffffff810e8db1>] ? update_isolated_counts+0x139/0x157 Oct 27 08:56:22 nemesis kernel: [25675.950313] [<ffffffff810ea596>] shrink_inactive_list+0x230/0x399 Oct 27 08:56:22 nemesis kernel: [25675.950316] [<ffffffff810eadd5>] shrink_zone+0x426/0x569 Oct 27 08:56:22 nemesis kernel: [25675.950319] [<ffffffff810ebb4d>] balance_pgdat+0x2c1/0x574 Oct 27 08:56:22 nemesis kernel: [25675.950333] [<ffffffff8106ff5b>] kthread+0x84/0x8c Oct 27 08:56:22 nemesis kernel: [25675.950337] [<ffffffff8148fe24>] kernel_thread_helper+0x4/0x10 Oct 27 08:56:22 nemesis kernel: [25675.950339] [<ffffffff8106fed7>] ? kthread_worker_fn+0x148/0x148 Oct 27 08:56:22 nemesis kernel: [25675.950342] [<ffffffff8148fe20>] ? gs_change+0x13/0x13 Oct 27 08:56:22 nemesis kernel: [25675.950343] Code: 50 48 48 89 50 50 48 89 45 f8 e8 36 ff ff ff 48 8b 45 f8 c9 c3 55 48 89 e5 66 66 66 66 90 48 8d 47 48 48 39 47 48 48 89 fe 74 02 <0f> 0b 48 8b 3d 7b 72 c2 00 e8 8c c5 fc ff 65 ff 0c 25 d0 f8 00 Oct 27 08:56:22 nemesis kernel: [25675.950367] RIP [<ffffffff8114a82c>] free_buffer_head+0x16/0x33 Oct 27 08:56:22 nemesis kernel: [25675.950369] RSP <ffff88012145ba20> Oct 27 08:56:22 nemesis kernel: [25675.950372] ---[ end trace a2881dc09c7eb870 ]--- The trace is tainted but the trace is the same without nvidia module and reliably reproducible on a loopback image or on external disk; the key is to wait during the XFS FS is tortured. Cheers.
A kernel oops is a different problem... could you please file a new bug for this? Ideally, if you think it is due to an unfixable/corrupted filesystem problem, attaching an xfs_metadump image (which obfuscates most filenames by default) would be helpful. Thanks, -Eric
I found the BUG but I'm not sure if my fix is really reasonable, I'm discussing this with Eric and some others XFS engineers to understand some points and write a better patch. For while, this is the current result of my patch, I'm not posting it here because I'm not sure how correct it is [cmaiolino@hades 510823]$ xfs_check xfs-image.img extent count for ino 7723 data fork too low (6) for file format agi unlinked bucket 25 is 18967193 in ag 3 (inode=421620377) link count mismatch for inode 151967493 (name ?), nlink 2, counted 3 link count mismatch for inode 421620377 (name ?), nlink 0, counted 1 [cmaiolino@hades 510823]$ xfs_repair xfs-image.img Phase 1 - find and verify superblock... Phase 2 - using internal log... Phase 3 - for each AG... Phase 4 - check for duplicate blocks... Phase 5 - rebuild AG headers and trees... Phase 6 - check inode connectivity... Phase 7 - verify and correct link counts... resetting inode 421620377 nlinks from 0 to 2 done [cmaiolino@hades 510823]$ xfs_check xfs-image.img [cmaiolino@hades 510823]$ ^ No more corrupted nlink count after the first xfs_repair. As soon as I'm sure about my patch I post it here.
The bug is caused because the function mk_orphanage, used to create the lost+found directory does not set the inode assigned to it as used in the in-memory AVL tree. The inode is properly set as used on disk, but not in the in-memory AVL tree. The phase 7 of xfs_repair compares the inode on-disk link counts with the AVL tree link counts, but it bypasses the inode link count check if the inode is set as free in the AVL tree. Since the on-disk data is correct the next execution of xfs_repair will fix it.
Cristian, I think we can just mark this as upstream, and we'll pick it up on the next release. Sound ok?
Sure, no problem.
Patch sent upstream: http://oss.sgi.com/archives/xfs/2011-11/msg00205.html Will close it as upstream as soon as I get an ACK on the maillist
Patches pushed upstream: commit 198b747f255346bca64408875763b6ca0ed3d57d Closing as upstream
Created attachment 532887 [details] Patch accepted upstream