Red Hat Bugzilla – Bug 652013
If EXT4_EXTENTS_FL flag is not set, the max file size of write() is different than seek().
Last modified: 2016-04-18 01:55:40 EDT
Created attachment 459526 [details] reproducer Description of problem: Environment: Red Hat Enterprise Linux 6 RC4 (kernel 2.6.32-71.el6) Problem: This is ext4 issue. Max limit of write(2) and seek(2) are different against files with EXT4_EXTENTS_FL flag. These should be the same. How to reproduce: Reproducer program is attached. Get "3.repro.tar" in File Attachment and run it and you can see the result easily. Patch: A patch is posted to ext4 community. http://marc.info/?l=linux-ext4&m=128470263711784&w=2 http://marc.info/?l=linux-ext4&m=128665669808374&w=2 ===================================================== Patch was backported and modified to work with RHEL6. The patch along with the reproducer is attached. ======================================================== Description of Problem: I have tested the following cases in order to confirm the maximum file size. For the tests, I selected two parameters: (These parameters relate to the max file size.) 1) Filesystem Feature "extent" 2) File(Inode) Flag "EXT4_EXTENTS_FL" (This parameter corresponds to the case where people shifts from ext3 into ext4. (Files which are created with ext3 have no "EXT4_EXTENTS_FL" flag.)) Table. the max file size which we can write or seek at each filesystem feature tuning and file flag setting +============+=========================+=========================+ | \ File flag| | | | \ | !EXT4_EXTENTS_FL | EXT4_EXTETNS_FL | |Fs Features\| | | +------------+-------------------------+-------------------------+ | !extent | write: 2194719883264 | write: -------------- | | | seek: 2199023251456 | seek: -------------- | +------------+-------------------------+-------------------------+ | extent | write: 4402345721856 | write: 17592186044415 | | | seek: 17592186044415 | seek: 17592186044415 | +------------+-------------------------+-------------------------+ ( The symbols (!extent, extent) mean: !extent: The filesystem feature "extent" is not set. ex. mkfs.ext3 <dev>; mount -t ext4 <dev> extent: The filesystem feature "extent" is set. ex. mkfs.ext3 <dev>; tune2fs -Oextent,huge_file <dev>; mount -t ext4 <dev> The symbols ("!EXT4_EXTENTS_FL","EXT4_EXTENTS_FL") mean: !EXT4_EXTENS_FL: The file flag, "EXT4_EXTENTS_FL" is not set. EXT4_EXTENS_FL: The file flag, "EXT4_EXTENTS_FL" is set. ) According to the table, if EXT4_EXTETNS_FL flag is not set, the max file size of write() is different from the one of seek(). These differences might cause some wrong filesystem actions which a user doesn't expect. Please fix it. P.S. This fix was posted to EXT4 community: http://marc.info/?l=linux-ext4&m=128665669808374&w=2 (And EXT4 maintainer applied it.) Version-Release number of selected component: Red Hat Enterprise Linux Version Number: RHEL6 Release Number: 6.0rc Architecture: x86_64 Kernel Version: 2.6.32-71.el6 Related Package Version: kernel Related Middleware / Application: None Drivers or hardware or architecture dependency: None. How reproducible: Always. Step to Reproduce: Run attached reproducer. (run.sh) Actual Results: The max file size of each case is different. Expected Results: The max file size of each case is all the same. Summary of actions taken to resolve issue: None. Location of diagnostic data: None. Hardware configuration: - Model: PRIMERGY TX150S5 - CPU Info: Xeon 1.86GHz - Memory Info: 6GB Business Impact: This problem affects especially the customers who mount their ext3 filesystems as ext4. They can encounter this problem when they call lseek systemcall for the file which is more than 2TB in their filesystem. Those differences may cause a critical problem on my customer's systems.
Created attachment 459527 [details] proposed patch
Just as a general note, using the ext4 driver for ext3 filesystems in RHEL6 is not recommended. We can fix this, but it's not the preferred mode of operation.
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux maintenance release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Update release for currently deployed products. This request is not yet committed for inclusion in an Update release.
Upstream commit: commit e0d10bfa91b0a089a9e2c378b5c42f4e96171d95 Author: Toshiyuki Okajima <toshi.okajima@jp.fujitsu.com> Date: Wed Oct 27 21:30:06 2010 -0400 ext4: improve llseek error handling for overly large seek offsets The llseek system call should return EINVAL if passed a seek offset which results in a write error. What this maximum offset should be depends on whether or not the huge_file file system feature is set, and whether or not the file is extent based or not. If the file has no "EXT4_EXTENTS_FL" flag, the maximum size which can be written (write systemcall) is different from the maximum size which can be sought (lseek systemcall). For example, the following 2 cases demonstrates the differences between the maximum size which can be written, versus the seek offset allowed by the llseek system call: #1: mkfs.ext3 <dev>; mount -t ext4 <dev> #2: mkfs.ext3 <dev>; tune2fs -Oextent,huge_file <dev>; mount -t ext4 <dev> Table. the max file size which we can write or seek at each filesystem feature tuning and file flag setting +============+===============================+===============================+ | \ File flag| | | | \ | !EXT4_EXTENTS_FL | EXT4_EXTETNS_FL | |case \| | | +------------+-------------------------------+-------------------------------+ | #1 | write: 2194719883264 | write: -------------- | | | seek: 2199023251456 | seek: -------------- | +------------+-------------------------------+-------------------------------+ | #2 | write: 4402345721856 | write: 17592186044415 | | | seek: 17592186044415 | seek: 17592186044415 | +------------+-------------------------------+-------------------------------+ The differences exist because ext4 has 2 maxbytes which are sb->s_maxbytes (= extent-mapped maxbytes) and EXT4_SB(sb)->s_bitmap_maxbytes (= block-mapped maxbytes). Although generic_file_llseek uses only extent-mapped maxbytes. (llseek of ext4_file_operations is generic_file_llseek which uses sb->s_maxbytes.) Therefore we create ext4 llseek function which uses 2 maxbytes. The new own function originates from generic_file_llseek(). If the file flag, "EXT4_EXTENTS_FL" is not set, the function alters inode->i_sb->s_maxbytes into EXT4_SB(inode->i_sb)->s_bitmap_maxbytes. Signed-off-by: Toshiyuki Okajima <toshi.okajima@jp.fujitsu.com>
I think that there is an error in the attached backported patch: + if (!(inode->i_flags & EXT4_EXTENTS_FL)) + maxbytes = EXT4_SB(inode->i_sb)->s_bitmap_maxbytes; + else + maxbytes = inode->i_sb->s_maxbytes; upstream this is: + if (!(ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS))) + maxbytes = EXT4_SB(inode->i_sb)->s_bitmap_maxbytes; + else + maxbytes = inode->i_sb->s_maxbytes; but the ext4_test_inode_flag function is not yet available in rhel6. However, it should be changed to: + if (!(EXT4_I(inode)->i_flags & EXT4_EXTENTS_FL)) not + if (!(inode->i_flags & EXT4_EXTENTS_FL)) because the EXT4_EXTENTS_FL flag is only valid on the extended ext4 inode not the vfs inode.
Patch(es) available on kernel-2.6.32-91.el6
Reproduced in 2.6.32-90.el6 and verified in 2.6.32-91.el.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2011-0542.html