Bug 1257625
| Summary: | fsck.gfs2 pass1c time scalability issue | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Nate Straz <nstraz> | ||||||
| Component: | gfs2-utils | Assignee: | Robert Peterson <rpeterso> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | cluster-qe <cluster-qe> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | medium | ||||||||
| Version: | 7.2 | CC: | cluster-maint, gfs2-maint, rpeterso, swhiteho | ||||||
| Target Milestone: | rc | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | gfs2-utils-3.1.9-1.el7 | Doc Type: | Bug Fix | ||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2016-11-04 06:30:11 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | 1271674 | ||||||||
| Bug Blocks: | 1111393, 1497636 | ||||||||
| Attachments: |
|
||||||||
|
Description
Nate Straz
2015-08-27 13:31:08 UTC
Pass1c is responsible for analyzing extended attributes, and it works off a linked list of items built in pass1. I see no reason why we can't eliminate the linked list and have pass1 do the analysis while it's checking each dinode. It seems logical to assume that it was moved to follow pass1b (duplicate reference processing) for a reason, so we need to determine if that's still necessary, given today's improved pass1 logic. I'm going to investigate whether we can eliminate pass1c altogether. Between pass1 and pass1b, I think we do almost everything necessary from pass1c with regard to extended attributes. I plan to do a detailed examination of pass1c to see if it does anything important, and if so, whether that can be safely moved to pass1. Reassigning to myself. I've done a thorough examination of pass1c. In my opinion, there's no reason not to perform its checks inside of pass1. That saves us from building the linked list, chewing up all that memory and time to make another pass through the file system. I've coded a patch that completely eliminates pass1c from fsck and moves its checks to the appropriate places in pass1c.c. I'm testing it now against my latest collection of metadata. Created attachment 1071846 [details]
Early prototype #1
This prototype eliminates pass1c in favor of simple checks
added to pass1.
I'm making good progress, but this turns out to be a bigger project than I anticipated. I'm finding some nasty bugs in fsck.gfs2 related to extended attributes and how we deal with them. I'm testing a new prototype now. I'll attach it when it gets further along in testing. Created attachment 1077949 [details]
Set of 20 RHEL7 patches - posted and tested
This tarball contains a set of 20 patches that I posted today:
0000-cover-letter.patch
0001-libgfs2-Check-block-range-when-inserting-into-rgrp-t.patch
0002-libgfs2-Check-rgd-bits-before-referencing-it.patch
0003-fsck.gfs2-Add-check-for-gfs1-invalid-inode-refs-in-d.patch
0004-fsck.gfs2-Make-debug-messages-more-succinct-wrt-exte.patch
0005-fsck.gfs2-Break-up-funtion-handle_dup_blk.patch
0006-fsck.gfs2-Only-preserve-the-_first_-acceptable-inode.patch
0007-fsck.gfs2-Don-t-just-assume-the-remaining-EA-referen.patch
0008-fsck.gfs2-Don-t-delete-inode-for-duplicate-reference.patch
0009-fsck.gfs2-Don-t-traverse-EAs-that-belong-to-another-.patch
0010-fsck.gfs2-Refactor-function-check_indirect_eattr.patch
0011-fsck.gfs2-Once-an-indirect-ea-error-is-found-flag-al.patch
0012-fsck.gfs2-Always-restore-saved-value-for-di_eattr.patch
0013-fsck.gfs2-Remove-redundancy-in-add_duplicate_ref.patch
0014-fsck.gfs2-Don-t-remove-duplicate-eattr-blocks.patch
0015-fsck.gfs2-Refactor-check_eattr_entries-and-add-error.patch
0016-fsck.gfs2-remove-bad-EAs-at-the-end-not-as-you-go.patch
0017-fsck.gfs2-Combine-remove_inode_eattr-with-its-only-c.patch
0018-fsck.gfs2-Print-debug-message-to-dilineate-metadata-.patch
0019-fsck.gfs2-Remove-pass1c-in-favor-of-processing-in-pa.patch
0020-fsck.gfs2-Clone-duplicate-data-block-pointers.patch
Bumping to 7.3. Changing status to POST. These patches were tested against my entire metadata collection. They were posted to cluster-devel, and then pushed to the gfs2-utils master branch. Our current plan is to rebase RHEL7.3 from master, as per a discussion I had with Andy Price this morning. Therefore I'm setting this to POST and we'll make it dependent on the rebase bz. This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions pass1c was removed and testing on gfs2-utils-3.1.9-3.el7.x86_64 shows that pass1 is scaling nearly linear from 256GB to 8TB. 256G fsck80: elapsed = 0:29:24.42, pass1 = 1760, pass1b = 0, pass2 = 2, pass3 = 0, pass4 = 0, maxresident = 0.02GB 512G fsck80: elapsed = 0:56:01.83, pass1 = 3353, pass1b = 0, pass2 = 4, pass3 = 0, pass4 = 0, maxresident = 0.04GB 1T fsck80: elapsed = 2:31:13.00, pass1 = 9055, pass1b = 0, pass2 = 11, pass3 = 0, pass4 = 0, maxresident = 0.09GB 2T fsck80: elapsed = 6:25:06.00, pass1 = 23068, pass1b = 0, pass2 = 22, pass3 = 0, pass4 = 1, maxresident = 0.18GB 4T fsck80: elapsed = 14:52:42.00, pass1 = 53481, pass1b = 0, pass2 = 53, pass3 = 0, pass4 = 2, maxresident = 0.35GB 8T fsck80: elapsed = 29:20:49.00, pass1 = 105445, pass1b = 0, pass2 = 169, pass3 = 0, pass4 = 5, maxresident = 0.68GB Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-2438.html |