Description of problem: While trying to convert a 3T file system with a 1k block size, gfs2_convert did not properly remove the rindex file. A subsequent gfs2_fsck of the converted file system found the following problems. Starting pass5 Ondisk and fsck bitmaps differ at block 137 (0x89) Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free) Metadata type is 0 (free) Bitmap at block 137 (0x89) left inconsistent Ondisk and fsck bitmaps differ at block 138 (0x8a) Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free) Metadata type is 0 (free) Bitmap at block 138 (0x8a) left inconsistent [ 4000+ lines of output ] Version-Release number of selected component (if applicable): gfs2-utils-0.1.50-1.el5 + gfs2_convert test binary from Bob. How reproducible: 100% Steps to Reproduce: 1. mkfs -t gfs -j 1 -p lock_nolock -b 1024 /dev/3T_volume 2. gfs2_convert /dev/3T_volume 3. fsck -t gfs2 -n /dev/3T_volume Actual results: See above Expected results: fsck after a convert should reveal no errors in the file system. Additional info:
I think there is still time (just) to fix this. Can we mark this one a blocker?
As I noted in bug #471618, the problem was caused because the rindex file was so big it went to height 2, but libgfs2 function gfs2_freedi cannot currently free blocks above height 1. It shouldn't take much time (a couple of hours) to cook up a non-recursive algorithm to properly free the unused blocks when the height is greater than 1. Today this problem only affects gfs2_convert, although it could also potentially affect other tools that might need to delete inodes from userspace. I need to investigate how gfs2_fsck does this if/when it encounters a file that it deems necessary to delete. I'm not sure if it's using this function or a better one that we might use in place of gfs2_freedi. I'm assuming a journal delete function (which currently doesn't exist) would use the metafs to do its work.
gfs2_fsck is able to fix the rindex file and a second run of gfs2_fsck shows no errors. This is easy to work around and it does not cause data corruption on the file system. It just leaves a bunch of blocks allocated which gfs2_fsck can fix. Moving back to 5.4, this should get a release note for 5.3.
Release note added. If any revisions are required, please set the "requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: In release 5.3, the gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2. These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system. These unused blocks will be flagged by gfs2_fsck with messages such as these: Ondisk and fsck bitmaps differ at block 137 (0x89) Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free) Metadata type is 0 (free) These messages do not indicate corruption in the GFS2 file system. They merely indicate blocks that should have been freed, but weren't. If not using the -y option, we recommend you answer Y to allow gfs2_fsck to free the unused metadata blocks.
Release note updated. If any revisions are required, please set the "requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -1,7 +1,9 @@ -In release 5.3, the gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2. These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system. These unused blocks will be flagged by gfs2_fsck with messages such as these: +In release 5.3, the gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2. These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system. We recommend you run gfs2_fsck after converting your filesystem to free the unused blocks. These unused blocks will be flagged by gfs2_fsck with messages such as these: Ondisk and fsck bitmaps differ at block 137 (0x89) Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free) Metadata type is 0 (free) + +The number of blocks needing to be freed will vary depending on the size of your file system and its block size. Many file systems won't have this issue at all. Large file systems may have a small number of blocks (typically less than 100). These messages do not indicate corruption in the GFS2 file system. They merely indicate blocks that should have been freed, but weren't. If not using the -y option, we recommend you answer Y to allow gfs2_fsck to free the unused metadata blocks.
Created attachment 325951 [details] Patch to fix the problem This patch fixes the problem; I tested it with a rindex of height 2.
Release note updated. If any revisions are required, please set the "requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -1,9 +1,7 @@ -In release 5.3, the gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2. These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system. We recommend you run gfs2_fsck after converting your filesystem to free the unused blocks. These unused blocks will be flagged by gfs2_fsck with messages such as these: +The gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2. These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system. It is recommended that gfs2_fsck be run after the filesystem has been converted to free the unused blocks. These unused blocks will be flagged by gfs2_fsck with messages such as: Ondisk and fsck bitmaps differ at block 137 (0x89) Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free) Metadata type is 0 (free) -The number of blocks needing to be freed will vary depending on the size of your file system and its block size. Many file systems won't have this issue at all. Large file systems may have a small number of blocks (typically less than 100). +These messages do not indicate corruption in the GFS2 file system, they indicate blocks that should have been freed, but were not. The number of blocks needing to be freed will vary depending on the size of the file system and block size. Many file systems will not encounter this issue at all. Large file systems may have a small number of blocks (typically less than 100).- -These messages do not indicate corruption in the GFS2 file system. They merely indicate blocks that should have been freed, but weren't. If not using the -y option, we recommend you answer Y to allow gfs2_fsck to free the unused metadata blocks.
I pushed the patch to the RHEL5, master and STABLE2 branches of the cluster git tree. Changing status to Modified.
gfs2-utils-2.03.11-1.fc9, cman-2.03.11-1.fc9, rgmanager-2.03.11-1.fc9 has been pushed to the Fedora 9 stable repository. If problems still persist, please make note of it in this bug report.
Release note is not required for RHEL 5.4
Release note updated. If any revisions are required, please set the "requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -1,7 +1 @@ -The gfs2_convert program may not free up all blocks from the GFS metadata that are no longer used under GFS2. These unused metadata blocks will be discovered and freed the next time gfs2_fsck is run on the file system. It is recommended that gfs2_fsck be run after the filesystem has been converted to free the unused blocks. These unused blocks will be flagged by gfs2_fsck with messages such as: +T- -Ondisk and fsck bitmaps differ at block 137 (0x89) -Ondisk status is 1 (Data) but FSCK thinks it should be 0 (Free) -Metadata type is 0 (free) - -These messages do not indicate corruption in the GFS2 file system, they indicate blocks that should have been freed, but were not. The number of blocks needing to be freed will vary depending on the size of the file system and block size. Many file systems will not encounter this issue at all. Large file systems may have a small number of blocks (typically less than 100).
Deleted Release Notes Contents. Old Contents: T
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2009-1337.html