Bug 166067 - Can not write a buffer beyond 2 giga bytes in a C program (if kernel version >= 2.6.11)
Summary: Can not write a buffer beyond 2 giga bytes in a C program (if kernel version ...
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 4
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Stephen Tweedie
QA Contact: Brian Brock
URL: local system
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2005-08-16 15:28 UTC by ZHOU Xiaodong
Modified: 2007-11-30 22:11 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2005-08-30 15:21:39 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description ZHOU Xiaodong 2005-08-16 15:28:32 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.7.8) Gecko/20050718 Firefox/1.0.4 (Debian package 1.0.4-2sarge1)

Description of problem:
After updating the kernel version >= 2.6.11 (so FC3 has the
same problem), under a C program, when I use write() or fwrite()
(or some other low level C write fonction) to write a buffer
beyond 2147483648 bytes, program fails with error message
: invalid argument.

Solution I used now is just downgrading to a 2.6.10 kernel.

My Athlon64 box config: 4 Giga bytes memory with a athlon64 3500+.


Sample program:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>


int main (int argc, char *argv[])
{
  int fd;
  size_t size_to_wrt;
  void *buffer_to_wrt;
  ssize_t size_wrten;

  size_to_wrt = 2147483648; /* can not beyond 2147483647 bytes */
  buffer_to_wrt = malloc(size_to_wrt);
  if(buffer_to_wrt == NULL) printf("Malloc error\n");

      
  fd  = open(argv[1],  O_CREAT | O_RDWR, S_IRUSR|S_IWUSR);
  size_wrten = write(fd, buffer_to_wrt, size_to_wrt);
  if(size_wrten != size_to_wrt)
   printf("Error message: %s\n", strerror(errno));
  free(buffer_to_wrt);
  close(fd);
  
  return 0;
}





Version-Release number of selected component (if applicable):
if kernel version >= 2.6.11

How reproducible:
Always

Steps to Reproduce:
1. A athlon64 PC has more that 2 Gbytes memory.
2. Compile ths sample C program. 
   (cc -o test_64bit_write test_64bit_write.c)
3. Execute with: test_64bit_write foo.

  

Actual Results:  error message
test_64bit_write : invalid argument.


Expected Results:  Program pass.

Additional info:

Comment 1 Stephen Tweedie 2005-08-30 11:34:08 UTC
Is there any genuine need for this?  This change was a deliberate upstream
decision, made after one or two code paths in the kernel were found which didn't
handle >=2GB IOs correctly.  Our policy is to follow upstream behaviour in all
such cases.


Comment 2 ZHOU Xiaodong 2005-08-30 12:18:27 UTC
In many industrial code we do need some kind of the
IO stream copy between the memory and storage device,
specially for a 64 bit program this kind of the
application comes soonly.

Certainly we can loop the I/O by buffer of 2Gb, but
this should not be done in a high level application,
but at some low level system IO menagement.

Thanks for any opinion.

Comment 3 ZHOU Xiaodong 2005-08-30 12:35:18 UTC
Anyway, the error message should be more explit
--- I had spend hours to found this.

Comment 4 Stephen Tweedie 2005-08-30 15:21:39 UTC
POSIX simply does not provide such fine-grained error codes; EINVAL is the
closest one available.  Unix in general just doesn't have a mechanism for
reporting errors with such specificity.

Our behaviour in this case is going to continue to match upstream.  It's
certainly worth asking on the linux kernel mailing lists if you think this
restriction is inappropriate.



Note You need to log in before you can comment on or make changes to this bug.