Bug 637970

Summary: GFS2: Not enough space reserved in gfs2_write_begin and possibly elsewhere.
Product: Red Hat Enterprise Linux 5 Reporter: Ben Marzinski <bmarzins>
Component: kernelAssignee: Ben Marzinski <bmarzins>
Status: CLOSED ERRATA QA Contact: Cluster QE <mspqa-list>
Severity: high Docs Contact:
Priority: urgent    
Version: 5.7CC: adas, anton, cww, dougsland, gansalmon, itamar, jonathan, kernel-maint, madhu.chinakonda, rpeterso, sforsber, swhiteho
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Some of the functions in the GFS2 file system were not reserving enough space for the resource group header in a transaction and for resource groups bit blocks that get added when a memory allocation is performed. That resulted in failed write and allocation operations. With this update, GFS2 makes sure to reserve space in the described scenario, using the new gfs2_rg_blocks() inline function.
 Story Points: ---
Clone Of: 626686 Environment:
Last Closed: 2011-07-21 10:24:05 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 626686, 637972    
Bug Blocks: 729092    
Attachments:
Description Flags
Patch to fix this issue. none

Description Ben Marzinski 2010-09-27 21:30:26 UTC 
+++ This bug was initially created as a clone of Bug #626686 +++

For a non journaled write, gfs2 reserves enough space to for all the indirect blocks it may possibly need to allocate or modify, plus a block for the dinode, a block for the local statfs change file, a block for quota changes, and an block if there are no journalled datablocks.

By picking a specific set of writes, I can get GFS2 to modify: all the indirect blocks that it reserves, the dinode, the local statfs change file, a resource group header block, and a resource group bit block.

This happens to come out the same number.  However, if it is possible that gfs2 might need to write to the quota file during a transaction, then it will fail. Also, if gfs2 can't allocate all the blocks from one resource group bit block, it will fail.

The solution seems to be to make gfs2 additionally reserve either as many blocks as it might allocate (both direct and indirect) or as many blocks as there are bit blocks in the resource group, whichever is fewer.  This guarantees that gfs2 will be able to allocate all of the blocks for the write, even if that requires modifying the maximum number of resource group bit blocks possible.

To make gfs2 use up at least all of the reserved blocks, I made a 1024 byte filesystem, mounted it, and then ran

# dd if=/dev/zero of=/mnt/test/foo bs=512 count=1
# dd if=/dev/zero of=/mnt/test/foo bs=4096 seek=238418579101562 count=1

This will force gfs2 to allocate all of the indirect blocks that it reserved.

--- Additional comment from bmarzins on 2010-09-27 17:19:43 EDT ---

Created attachment 450040 [details]
latest patch to fix issue

This is the latest patch I submitted to cluster-devel to fix this issue.  It adds an inline function, gfs2_rg_blocks() that returns either the number of allocated blocks plus one for the rg header, or the total number of blocks in the rg, whichever is less.  This is used by functions that need to allocate blocks in transactions to makes sure and reserve enough blocks to deal with the resource groups.
Comment 1 Steve Whitehouse 2010-11-18 12:51:05 UTC 
Is there anything preventing us posting this patch now?
Comment 2 Ben Marzinski 2010-11-18 15:40:59 UTC 
Nope. In fact I did, on Oct 1st, and you and bob ACK'ed it.  I just forgot to update the bugzilla.

Sorry.
Comment 3 Ben Marzinski 2010-11-18 15:44:12 UTC 
Created attachment 461326 [details]
Patch to fix this issue.
Comment 4 RHEL Program Management 2011-02-01 16:53:59 UTC 
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 6 Jarod Wilson 2011-02-09 14:56:25 UTC 
in kernel-2.6.18-243.el5
You can download this test kernel (or newer) from http://people.redhat.com/jwilson/el5

Detailed testing feedback is always welcomed.
Comment 8 errata-xmlrpc 2011-07-21 10:24:05 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-1065.html
Comment 11 Tomas Capek 2011-09-13 13:04:00 UTC 
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Some of the functions in the GFS2 file system were not reserving enough space for the resource group header in a transaction and for resource groups bit blocks that get added when a memory allocation is performed. That resulted in failed write and allocation operations. With this update, GFS2 makes sure to reserve space in the described scenario, using the new gfs2_rg_blocks() inline function.