Bug 474707 - GFS2: gfs2_convert not freeing blocks when removing file with height >=2
GFS2: gfs2_convert not freeing blocks when removing file with height >=2
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: gfs2-utils (Show other bugs)
5.3
All Linux
low Severity medium
: rc
: ---
Assigned To: Robert Peterson
Cluster QE
:
Depends On:
Blocks: RHEL5u3_relnotes
  Show dependency treegraph
 
Reported: 2008-12-04 17:03 EST by Nate Straz
Modified: 2010-01-11 22:41 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-09-02 07:02:39 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Patch to fix the problem (3.40 KB, patch)
2008-12-05 19:27 EST, Robert Peterson
no flags Details | Diff

  None (edit)
Description Nate Straz 2008-12-04 17:03:43 EST
Description of problem:

While trying to convert a 3T file system with a 1k block size, gfs2_convert did not properly remove the rindex file.  A subsequent gfs2_fsck of the converted file system found the following problems.

Starting pass5
Ondisk and fsck bitmaps differ at block 137 (0x89) 
Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free)
Metadata type is 0 (free)
Bitmap at block 137 (0x89) left inconsistent
Ondisk and fsck bitmaps differ at block 138 (0x8a) 
Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free)
Metadata type is 0 (free)
Bitmap at block 138 (0x8a) left inconsistent
[ 4000+ lines of output ]


Version-Release number of selected component (if applicable):
gfs2-utils-0.1.50-1.el5 + gfs2_convert test binary from Bob.

How reproducible:
100%

Steps to Reproduce:
1. mkfs -t gfs -j 1 -p lock_nolock -b 1024 /dev/3T_volume
2. gfs2_convert /dev/3T_volume
3. fsck -t gfs2 -n /dev/3T_volume
  
Actual results:
See above

Expected results:
fsck after a convert should reveal no errors in the file system.

Additional info:
Comment 1 Steve Whitehouse 2008-12-05 05:36:55 EST
I think there is still time (just) to fix this. Can we mark this one a blocker?
Comment 2 Robert Peterson 2008-12-05 09:53:43 EST
As I noted in bug #471618, the problem was caused because the rindex
file was so big it went to height 2, but libgfs2 function gfs2_freedi
cannot currently free blocks above height 1.  It shouldn't take much
time (a couple of hours) to cook up a non-recursive algorithm to
properly free the unused blocks when the height is greater than 1.

Today this problem only affects gfs2_convert, although it could also
potentially affect other tools that might need to delete inodes from
userspace.  I need to investigate how gfs2_fsck does this if/when it
encounters a file that it deems necessary to delete.  I'm not sure if
it's using this function or a better one that we might use in place
of gfs2_freedi.  I'm assuming a journal delete function (which
currently doesn't exist) would use the metafs to do its work.
Comment 3 Nate Straz 2008-12-05 11:55:41 EST
gfs2_fsck is able to fix the rindex file and a second run of gfs2_fsck shows no errors.  This is easy to work around and it does not cause data corruption on the file system.  It just leaves a bunch of blocks allocated which gfs2_fsck can fix.

Moving back to 5.4, this should get a release note for 5.3.
Comment 4 Robert Peterson 2008-12-05 12:19:10 EST
Release note added. If any revisions are required, please set the 
"requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

New Contents:
In release 5.3, the gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2.  These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system.  These unused blocks will be flagged by gfs2_fsck with messages such as these:

Ondisk and fsck bitmaps differ at block 137 (0x89) 
Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free)
Metadata type is 0 (free)

These messages do not indicate corruption in the GFS2 file system.  They merely indicate blocks that should have been freed, but weren't.  If not using the -y option, we recommend you answer Y to allow gfs2_fsck to free the unused metadata blocks.
Comment 5 Robert Peterson 2008-12-05 12:34:51 EST
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1,7 +1,9 @@
-In release 5.3, the gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2.  These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system.  These unused blocks will be flagged by gfs2_fsck with messages such as these:
+In release 5.3, the gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2.  These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system.  We recommend you run gfs2_fsck after converting your filesystem to free the unused blocks.  These unused blocks will be flagged by gfs2_fsck with messages such as these:
 
 Ondisk and fsck bitmaps differ at block 137 (0x89) 
 Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free)
 Metadata type is 0 (free)
+
+The number of blocks needing to be freed will vary depending on the size of your file system and its block size.  Many file systems won't have this issue at all.  Large file systems may have a small number of blocks (typically less than 100).
 
 These messages do not indicate corruption in the GFS2 file system.  They merely indicate blocks that should have been freed, but weren't.  If not using the -y option, we recommend you answer Y to allow gfs2_fsck to free the unused metadata blocks.
Comment 6 Robert Peterson 2008-12-05 19:27:40 EST
Created attachment 325951 [details]
Patch to fix the problem

This patch fixes the problem; I tested it with a rindex of height 2.
Comment 8 Ryan Lerch 2008-12-07 19:10:01 EST
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1,9 +1,7 @@
-In release 5.3, the gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2.  These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system.  We recommend you run gfs2_fsck after converting your filesystem to free the unused blocks.  These unused blocks will be flagged by gfs2_fsck with messages such as these:
+The gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2.  These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system.  It is recommended that gfs2_fsck be run after the filesystem has been converted to free the unused blocks.  These unused blocks will be flagged by gfs2_fsck with messages such as:
 
 Ondisk and fsck bitmaps differ at block 137 (0x89) 
 Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free)
 Metadata type is 0 (free)
 
-The number of blocks needing to be freed will vary depending on the size of your file system and its block size.  Many file systems won't have this issue at all.  Large file systems may have a small number of blocks (typically less than 100).
+These messages do not indicate corruption in the GFS2 file system,  they indicate blocks that should have been freed, but were not. The number of blocks needing to be freed will vary depending on the size of the file system and block size.  Many file systems will not encounter this issue at all. Large file systems may have a small number of blocks (typically less than 100).-
-These messages do not indicate corruption in the GFS2 file system.  They merely indicate blocks that should have been freed, but weren't.  If not using the -y option, we recommend you answer Y to allow gfs2_fsck to free the unused metadata blocks.
Comment 9 Robert Peterson 2008-12-16 12:13:55 EST
I pushed the patch to the RHEL5, master and STABLE2 branches of the
cluster git tree.  Changing status to Modified.
Comment 10 Fedora Update System 2009-01-23 21:36:33 EST
gfs2-utils-2.03.11-1.fc9, cman-2.03.11-1.fc9, rgmanager-2.03.11-1.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 13 Steve Whitehouse 2009-06-02 11:54:23 EDT
Release note is not required for RHEL 5.4
Comment 14 Steve Whitehouse 2009-06-02 11:54:23 EDT
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1,7 +1 @@
-The gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2.  These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system.  It is recommended that gfs2_fsck be run after the filesystem has been converted to free the unused blocks.  These unused blocks will be flagged by gfs2_fsck with messages such as:
+T-
-Ondisk and fsck bitmaps differ at block 137 (0x89) 
-Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free)
-Metadata type is 0 (free)
-
-These messages do not indicate corruption in the GFS2 file system,  they indicate blocks that should have been freed, but were not. The number of blocks needing to be freed will vary depending on the size of the file system and block size.  Many file systems will not encounter this issue at all. Large file systems may have a small number of blocks (typically less than 100).
Comment 15 Steve Whitehouse 2009-06-25 11:28:13 EDT
Deleted Release Notes Contents.

Old Contents:
T
Comment 17 errata-xmlrpc 2009-09-02 07:02:39 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2009-1337.html

Note You need to log in before you can comment on or make changes to this bug.