Bug 144703 (IT_69399) - ia32 apps that are not large file aware can access files >= 4GB
Summary: ia32 apps that are not large file aware can access files >= 4GB
Alias: IT_69399
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: kernel   
(Show other bugs)
Version: 4.0
Hardware: ia64
OS: Linux
Target Milestone: ---
: ---
Assignee: Peter Staubach
QA Contact: Brian Brock
Depends On:
Blocks: 168430
TreeView+ depends on / blocked
Reported: 2005-01-10 20:03 UTC by Sreevatsa Nagarajarao
Modified: 2007-11-30 22:07 UTC (History)
10 users (show)

Fixed In Version: RHSA-2006-0132
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2006-03-07 18:39:14 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Proposed patch (1.30 KB, patch)
2005-08-26 14:11 UTC, Peter Staubach
no flags Details | Diff
Proposed patch (1.65 KB, patch)
2005-09-07 12:54 UTC, Peter Staubach
no flags Details | Diff

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2005:808 normal SHIPPED_LIVE Important: kernel security update 2005-10-27 04:00:00 UTC
Red Hat Product Errata RHSA-2006:0132 qe-ready SHIPPED_LIVE Moderate: Updated kernel packages available for Red Hat Enterprise Linux 4 Update 3 2006-03-09 16:31:00 UTC

Description Sreevatsa Nagarajarao 2005-01-10 20:03:31 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (compatible; Konqueror/3.2; Linux) (KHTML, like Gecko)

Description of problem:
When an ia32 application is running under ia32-el, the kernel sees only the ia64 application (ia32-el) ... so whenever files are opened the O_LARGEFILE mode is automatically turned on.

This leads to two test failures in our QA suite:
1) An ia32 application that is not large file aware succeeds in opening a file that has a size of 4GB.
2) An ia32 application that is not large file aware succeeds in seeking to an offset above 4G.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
Take an ia32 application that is not compiled to enable large file access, and use it under ia32-el on an ia64 system to open files >= 4GB, or to open a smaller file and then grow it above 4G (either by relative seeks, or by writing to the file at offset 4G-1).

We believe that the fix for this may require a kernel change (to allow fcntl() to turn off the O_LARGEFILE behaviour) in addition to changes to ia32-el.

Additional info:

Comment 1 Yoav Zach 2005-01-10 21:35:35 UTC
this solution isn't complete. For the case where an application tries 
to open a file with size >= 4GB, the open should fail.  A user-space 
fix to this would require ia32-el to stat the file, and then open it 
if it isn't large ... 
but there is an obvious race condition there where the file might 
grow (or shrink) between the stat and the open, so the wrong action 
would be taken.

another solution, which we believe is preferred, is to turn off the 
automatic O_LARGEFILE on ia64 in case the personality of the process 
is LINUX32.


Comment 2 Tony Luck 2005-01-27 01:03:13 UTC
Modifying fcntl so that it can clear O_LARGEFILE may introduce new problems, 
programs that use F_SETFL option of fcntl might accidently clear O_LARGEFILE.

I'd really like to see a way for a user application to specify to the kernel 
that it doesn't want O_LARGEFILE (as suggested above by Yoav).  Checking 
personality would be convenient, but may run into resistance from other 
architectures that don't use personality.

Comment 4 Tony Luck 2005-05-26 16:11:16 UTC
Andrew Morton took the kernel patch for this into 2.6.12-rc4-mm2


So it may finally get into the upstream kernel soon.

Comment 5 Tony Luck 2005-06-30 16:10:41 UTC
Patch went into Linus' tree on June 23rd.


It's very small, 2-4 lines in each of three files, so a backport for an EL4
update release should be trivial.

Comment 6 Peter Staubach 2005-07-13 14:31:08 UTC
What file system is being used on the ia64 system when the 32 bit application
is incorrectly allowed to open the large file?

Comment 7 Tony Luck 2005-07-13 16:48:51 UTC
Bug is filesystem independent.  Generic Linux kernel code unconditionally sets 
the O_LARGEFILE flag on 64-bit systems.  See fs/open.c

#if BITS_PER_LONG != 32
          flags |= O_LARGEFILE;

So this bug should be observable on every filesystem type.

[N.B. today bugzilla won't let me unset the "Leave as NEEDINFO" radio button, 
could someone please do that]

Comment 8 Peter Staubach 2005-07-13 17:33:43 UTC
I realize that the solution is probably file system independent.  I also
realize that there is a much larger problem than what is described here.

However, what'd I like to know is what file system is being used on the
ia64 system which showed the problem?  Or is this just a problem during
code review?

I ran a test on an x86_64 system using the ext3 file system and it
worked as expected, without showing this problem.  I do agree that there
is a problem, but I need a little more information in the short term.

Comment 9 Tony Luck 2005-07-13 18:09:34 UTC
The problem was found by Veritas while running their validation suites for the 
port of their proprietary VxFS filesystem (scroll up and see the original bug 
report by Sreevatsa).

To demonstrate this problem you need to make sure that you have compiled your 
application without support for large files.

I don't think that you will be able to reproduce on x86_64 though.  They have a 
sys32_open() call in arch/x86_64/ia32/sys_ia32.c that replicates the 
functionality of fs/open.c:sys_open() ... but without the unconditional 
addition of the O_LARGEFILE bit to flags.

As far as I know this bug only shows up on ia64 systems using ia32-el to 
emulate an ia32 processor.  In that case the open system call is executed by 
the ia64 process that is providing the emulation, so it goes to the "native" 
sys_open() routine in fs/open.c


Comment 11 Peter Staubach 2005-07-29 12:55:15 UTC
Backed out.  Still working the possible kabi issue.

Comment 12 Tony Luck 2005-07-29 16:07:16 UTC
kabi issue?  Can you elaborate?

Comment 13 Peter Staubach 2005-07-29 17:16:51 UTC
The Module.symvers file show changes when the kernel is built with the
changes to address this problem.  The affected symbols are:


The changed file shows up on all platforms.

Frankly, I don't see how these changes could be qualified as a KABI change,
so I am mystified.  However, I/we need an explanation before we can proceed.
I am still pondering and have asked for help internally.  Got any ideas?

Comment 14 Peter Staubach 2005-08-26 14:11:50 UTC
Created attachment 118153 [details]
Proposed patch

Comment 15 Tony Luck 2005-08-26 15:47:36 UTC
I was worried at first by the change from checking for the PER_LINUX32 bit to 
checking for "!= PER_LINUX32".  But it appears that this is equivalent since 
the "personality()" macro limits the test to the low-byte.


Comment 16 Yoav Zach 2005-08-28 09:42:38 UTC
force_o_largefile should be defined as
( ! (current->personality & PER_LINUX32) )
and not -
(personality(current->personality) != PER_LINUX32)

this is important, as the personality field might have more bits set. also, 
this is the code that was accepted upstream.


Comment 17 Tony Luck 2005-08-28 20:39:16 UTC
The definition of the enum in linux/personality.h is somewhat confusing.  Enough
so that I didn't notice this problem when looking at this fix when Acking the
patch to go upstream.  RedHat have the correct fix here, the low-order byte is
NOT a bitmask (see comment in linux/personality.h about this).  Also consider
that we would not want to have this behaviour for processes that set PER_IRIX32,
PER_IRIXN32, ... PER_OSF4.  The upstream kernel currently fails to set the
O_LARGEFILE mode for processes using any of these personalities since they all
have the 0x8 bit set ... but this is wrong.  I'll fix it tomorrow by changing
the test to do "personality(current->personality) != PER_LINUX32", the same as
this patch (though this won't go in for 2.6.13).

I do have a suggestion for the kABI problem (but it isn't the prettiest code in
the world).  The problem arises because linux/personality.h has the definition
of the exec_domain structure, the presence of this causes the Modules.symvers
calculation of function signatures to change for several functions in fs/open.c

The solution is to wrap the definition inside a #ifndef of a symbol defined only
in fs/open.c.  E.g. pseudo-patch:

--- fs/open.c
  #include <linux/fs.h>
+ #define NO_EXEC_DOMAIN
+ #include <linux/personality.h>
  #include <linux/pagemap.h>

--- include/linux/personality.h

+ #ifndef NO_EXEC_DOMAIN
  struct exec_domain {

+ #endif

Comment 18 Peter Staubach 2005-09-07 12:54:39 UTC
Created attachment 118556 [details]
Proposed patch

Comment 19 Tony Luck 2005-09-07 15:31:25 UTC
That looks more elegant than wrapping the definition of struct exec_domain 
inside a #ifndef.  I assume that adding "#include <linux/fcntl.h> to process.c 
didn't change the signature of any exported functions in that file (now I know 
this can happen I'm paranoid about it!).

Comment 20 Peter Staubach 2005-09-07 16:05:27 UTC
Yes, me too.  I am definitely more careful about it now.  I still don't
fully understand why things changed and still suspect a bug in the
modpost program.  Whatever though...

I did full builds on all RHEL-4 supported platforms and did not see any
changes this time around.

Comment 26 Jay Turner 2006-01-03 20:15:43 UTC
Please test and confirm resolution with 2.6.9-27.EL or later.  Thanks.

Comment 27 Martin Jenner 2006-02-21 16:02:20 UTC
We are trying to verify this issue for the RHEL4-U3 kernel.  Would someone
provide confirmation that customer is satisifed with the fix ? Thank you.

Comment 29 Sreevatsa Nagarajarao 2006-02-22 17:28:48 UTC
We will verify the fix in the next few days and get back to you. Thanks.

Comment 31 Sreevatsa Nagarajarao 2006-03-02 22:09:30 UTC
Fyi, we are having some difficulty running x86 binaries on our ia64 test system
because the correct version of some rpms (glibc-commons, glibc, etc) are
missing. We will verify the bug fix soon after the installation issues are resolved.

Comment 32 Red Hat Bugzilla 2006-03-07 18:39:14 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Comment 33 Sreevatsa Nagarajarao 2006-03-29 22:22:02 UTC
We verfied that the bug is fixed in RHEL4-U3. Thanks.

Note You need to log in before you can comment on or make changes to this bug.