Bug 166067 - Can not write a buffer beyond 2 giga bytes in a C program (if kernel version >= 2.6.11)
Can not write a buffer beyond 2 giga bytes in a C program (if kernel version ...
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Stephen Tweedie
Brian Brock
local system
Depends On:
  Show dependency treegraph
Reported: 2005-08-16 11:28 EDT by ZHOU Xiaodong
Modified: 2007-11-30 17:11 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2005-08-30 11:21:39 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description ZHOU Xiaodong 2005-08-16 11:28:32 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.7.8) Gecko/20050718 Firefox/1.0.4 (Debian package 1.0.4-2sarge1)

Description of problem:
After updating the kernel version >= 2.6.11 (so FC3 has the
same problem), under a C program, when I use write() or fwrite()
(or some other low level C write fonction) to write a buffer
beyond 2147483648 bytes, program fails with error message
: invalid argument.

Solution I used now is just downgrading to a 2.6.10 kernel.

My Athlon64 box config: 4 Giga bytes memory with a athlon64 3500+.

Sample program:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>

int main (int argc, char *argv[])
  int fd;
  size_t size_to_wrt;
  void *buffer_to_wrt;
  ssize_t size_wrten;

  size_to_wrt = 2147483648; /* can not beyond 2147483647 bytes */
  buffer_to_wrt = malloc(size_to_wrt);
  if(buffer_to_wrt == NULL) printf("Malloc error\n");

  fd  = open(argv[1],  O_CREAT | O_RDWR, S_IRUSR|S_IWUSR);
  size_wrten = write(fd, buffer_to_wrt, size_to_wrt);
  if(size_wrten != size_to_wrt)
   printf("Error message: %s\n", strerror(errno));
  return 0;

Version-Release number of selected component (if applicable):
if kernel version >= 2.6.11

How reproducible:

Steps to Reproduce:
1. A athlon64 PC has more that 2 Gbytes memory.
2. Compile ths sample C program. 
   (cc -o test_64bit_write test_64bit_write.c)
3. Execute with: test_64bit_write foo.


Actual Results:  error message
test_64bit_write : invalid argument.

Expected Results:  Program pass.

Additional info:
Comment 1 Stephen Tweedie 2005-08-30 07:34:08 EDT
Is there any genuine need for this?  This change was a deliberate upstream
decision, made after one or two code paths in the kernel were found which didn't
handle >=2GB IOs correctly.  Our policy is to follow upstream behaviour in all
such cases.
Comment 2 ZHOU Xiaodong 2005-08-30 08:18:27 EDT
In many industrial code we do need some kind of the
IO stream copy between the memory and storage device,
specially for a 64 bit program this kind of the
application comes soonly.

Certainly we can loop the I/O by buffer of 2Gb, but
this should not be done in a high level application,
but at some low level system IO menagement.

Thanks for any opinion.
Comment 3 ZHOU Xiaodong 2005-08-30 08:35:18 EDT
Anyway, the error message should be more explit
--- I had spend hours to found this.
Comment 4 Stephen Tweedie 2005-08-30 11:21:39 EDT
POSIX simply does not provide such fine-grained error codes; EINVAL is the
closest one available.  Unix in general just doesn't have a mechanism for
reporting errors with such specificity.

Our behaviour in this case is going to continue to match upstream.  It's
certainly worth asking on the linux kernel mailing lists if you think this
restriction is inappropriate.

Note You need to log in before you can comment on or make changes to this bug.