Bug 141167 - find: .: Value too large for defined data type
find: .: Value too large for defined data type
Status: CLOSED INSUFFICIENT_DATA
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
4
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Steve Dickson
Brian Brock
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2004-11-29 13:43 EST by Carrie Knox
Modified: 2007-11-30 17:10 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-05-05 17:18:23 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
strace find . -mtime 1 -print 2>/tmp/trace (5.22 KB, text/plain)
2004-11-30 09:30 EST, Carrie Knox
no flags Details
ltrace find . -mtime 1 -print 2>/tmp/ltrace (10.52 KB, text/plain)
2004-11-30 13:18 EST, Carrie Knox
no flags Details
tcpdump -v -w /tmp/nfs.pcap -s 200 host mink (5.01 KB, text/plain)
2004-12-02 13:10 EST, Carrie Knox
no flags Details
Simple test program which trips the problem (401 bytes, text/plain)
2005-02-09 18:42 EST, Robin Holt
no flags Details
strace output when run against Irix 6.5.8 NFS server (2.36 KB, text/plain)
2005-02-09 18:45 EST, Robin Holt
no flags Details
strace output when run against Irix 6.5.24 NFS server (2.08 KB, text/plain)
2005-02-09 18:47 EST, Robin Holt
no flags Details

  None (edit)
Description Carrie Knox 2004-11-29 13:43:03 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.3)
Gecko/20041020

Description of problem:
When running the find command on an NFS mounted filesystem, the error
"Value too large for defined data type" is returned.  The error does
not happen for all NFS mounted filesystems.  Suspecting large size or
number of files is the issue.  Could re-create the error on another
large NFS mounted filesystem, but not on a smaller one. 

lnx-gx270-6$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
mink:/export/dist     133G  116G   17G  88% /data/cf/dist

lnx-gx270-7$ find . -mtime -1 -print
find: .: Value too large for defined data type

When I downgrade to findutils-4.1.7-25 from the FC2 distribution, the
error no longer occurs.  I do receive the same error, "Value too large
for defined data type",  with findutils-4.1.20-3.i386.rpm from FC3-test3.


Version-Release number of selected component (if applicable):
findutils-4.1.20-7

How reproducible:
Always

Steps to Reproduce:
1.NFS mount 133 GB filesystem
2.find . -mtime -1 -print
3.
    

Actual Results:  find: .: Value too large for defined data type

Expected Results:  files modified within the last day should have been
listed.

Additional info:
Comment 1 Tim Waugh 2004-11-30 05:52:12 EST
Tried with a 135Gb NFS filesystem but couldn't reproduce it.

1. What options is the NFS filesystem mounted with?
2. Please attach 'trace' from 'strace find . -mtime 1 -print 2>trace'.

Thanks.
Comment 2 Carrie Knox 2004-11-30 09:30:58 EST
Created attachment 107624 [details]
strace find . -mtime 1 -print 2>/tmp/trace

strace output attached
Comment 3 Carrie Knox 2004-11-30 09:32:44 EST
[root@lnx-gx270 mnt]# mkdir mink
[root@lnx-gx270 mnt]# mount mink:/export/dist/linux /mnt/mink
[root@lnx-gx270 mnt]# df | grep mink
mink:/export/dist/linux
                     138732992 123869376  14863616  90% /mnt/mink
[root@lnx-gx270 mnt]# mount | grep mink
mink:/export/dist/linux on /mnt/mink type nfs (rw,addr=137.38.224.3)
[root@lnx-gx270 mnt]# cd /mnt/mink
[root@lnx-gx270 mink]# find . -mtime -1 -print
find: .: Value too large for defined data type
Comment 4 Tim Waugh 2004-11-30 12:11:49 EST
How about 'ltrace find . -mtime 1 -print 2>/tmp/ltrace'?  I haven't tracked down
what the problem is yet -- the system calls all seem to have succeeded!
Comment 5 Carrie Knox 2004-11-30 13:18:45 EST
Created attachment 107644 [details]
ltrace find . -mtime 1 -print 2>/tmp/ltrace

find ltrace output is attached
Comment 6 Tim Waugh 2004-12-01 07:51:02 EST
What's 'ls -l' say in that directory?
Comment 7 Carrie Knox 2004-12-01 09:05:05 EST
$ cd /mnt/mink
$ ls -l
total 71888
drwxr-xr-x   2 distadm sdss_adm     4096 Sep 20  2002 62_apps
lrwxr-xr-x   1 root    root           13 Sep 20  2002 62_standard_apps ->
standard_apps
drwxr-xr-x   2 root    root            9 Sep 20  2002 62_updates
drwxr-xr-x   2 distadm sdss_adm     4096 Apr  2  2003 72_apps
drwxr-xr-x   2 root    root         4096 Apr  2  2003 72_rpms
drwxrwxr-x   2 distadm sdss_adm     4096 Sep 20  2002 72_rpms_NOT_USED
drwxr-xr-x   3 root    root         4096 Jan 28  2003 72_standard_apps
drwxrwxr-x   2 root    cfccn         265 Apr  4  2003 72_updates
drwxr-xr-x   2 root    root            9 Nov 19  2002 80_standard_apps
drwxr-xr-x   3 root    root         4096 Nov 19  2002 80_standard_apps.org
drwxr-xr-x   2 root    root            9 Nov 18  2002 80_updates
drwxr-xr-x   2 root    root         4096 Oct 28 10:29 90_apps
drwxr-xr-x   2 root    root         4096 Oct 14  2003 90_standard_apps
drwxr-xr-x   2 root    root         4096 Sep 22  2003 90_standard_apps.jim
drwxr-xr-x   6 root    root         4096 May 12  2004 90_updates
drwxr-xr-x   2 root    root            9 May 30  2003 90_updates.jim
lrwxr-xr-x   1 root    root            7 Sep 20  2002 apps -> 72_apps
drwxr-xr-x   2 root    root           34 Sep 20  2002 bin
drwxr-xr-x   3 root    root           21 Jan 17  2003 boot
drwxr-xr-x   3 distadm sdss_adm       26 Sep 20  2002 extras
drwxr-xr-x   2 root    root          199 Aug 12 16:41 fc2_apps
drwxr-xr-x   2 root    root         4096 Aug 12 13:16 fc2_standard_apps
drwxr-xr-x   2 root    root          368 Sep 30 14:42 fc2_updates
drwxr-xr-x   2 root    root          345 Nov 30 15:42 fc3_apps
drwxr-xr-x   2 knox    root         4096 Nov 12 15:23 fc3_standard_apps
drwxr-xr-x   2 knox    root         4096 Oct 19 08:39 fc3_standard_apps.O
drwxr-xr-x   3 root    root          196 Nov  3 08:01 fc3t3_updates
drwxr-xr-x   4 root    root          264 Nov 30 15:42 fc3_updates
drwxr-xr-x   2 root    root          108 Oct 14 08:36 fc3_updates.O
drwxr-xr-x   7 knox    root           87 Nov 30 09:43 ISO_images
drwxrwxrwx   5 distadm sdss_adm    16384 Nov 29 09:04 kickstart-hosts
-rw-r--r--   1 root    root     73512960 Oct  8  2003 lnx-xni.tar
drwxr-xr-x   2 knox    root         4096 Sep 24  2003 My90_standard_apps
drwxr-xr-x   4 knox    root         4096 Oct 10  2003 My90_updates
drwxrwxrwx   2 distadm sdss_adm      205 Sep 20  2002 notes
drwxr-xr-x   8 distadm sdss_adm      163 Sep 20  2002 propack
drwxrwxrwx  18 distadm sdss_adm      368 Nov 29 12:59 redhat
drwxrwxr-x   2 distadm sdss_adm     4096 Jan 15  2004 rpms
drwxrwxrwx   4 distadm sdss_adm     4096 Sep 27 14:57 src
drwxrwxrwx   2 distadm sdss_adm     4096 Sep 20  2002 standard_apps
drwxr-xr-x   8 distadm root          103 Sep 20  2002 standard_dist
drwxr-xr-x   5 knox    root          381 Sep 27 15:28 test
drwxr-xr-x   2 distadm root         4096 Sep 20  2002 test_standard_apps
drwxrwxrwt   3 root    root          337 Jul 25  2003 tmp
drwxr-xr-x   2 root    root           96 Sep 27  2002 updated_72_rpms
drwxr-xr-x   6 knox    root          189 Apr  4  2003 updates_c640
drwxr-xr-x   4 root    root           93 Mar 10  2003 updates_gx260
drwxr-xr-x   2 knox    root           88 Sep 20  2002 VMWare3.0_1455
drwxrwxrwx   4 distadm root           35 Sep 20  2002 xfs
-rw-r--r--   1 root    root          953 Dec  5  2001 xircom_serial.c
drwxr-xr-x   2 knox    root           33 Sep 20  2002 xni
$
Comment 8 Tim Waugh 2004-12-01 09:29:34 EST
Thanks.  Also, what sort of device is mink, and what underlying filesystem is
that export?  What does it have in common with the other NFS export you tried?

I haven't been able to reproduce this at all, but it seems to be due to
readdir() setting errno to EOVERFLOW at some point.  Previous versions of
findutils did not check errno after readdir() returned, but readdir() is quite
entitled to set it even when otherwise appearing successful.
Comment 9 Carrie Knox 2004-12-01 10:24:27 EST
The filesystem mounted from mink is an xlv on an IRIX 6.5.x server.  RAID 5 device. 
Filesystem             Type  blocks     use     avail  %use Mounted
on/dev/xlv/dist           xfs 277465984 238976576 38489408  87  /export/dist

mink-675# xfs_growfs -n /export/dist
meta-data=/export/dist           isize=512    agcount=134, agsize=258840 blks
data     =                       bsize=4096   blocks=34684416, imaxpct=25
         =                       sunit=8      swidth=8 blks, unwritten=1
naming   =version 1              bsize=4096
log      =internal               bsize=4096   blocks=1168
realtime =none                   extsz=65536  blocks=0, rtextents=0

--------------------------
The second filesystem that fails is also an xlv xfs type filesystem mounted from
an IRIX 6.5.x server.  RAID 5 device.

[root@lnx-gx270 ~]# mkdir /mnt/ecad
[root@lnx-gx270 ~]# mount ecad.engr:/ecad /mnt/ecad
[root@lnx-gx270 ~]# cd /mnt/ecad
[root@lnx-gx270 ecad]# df -k .
Filesystem           1K-blocks      Used Available Use% Mounted on
ecad.engr:/ecad      174016352 169716192   4300160  98% /mnt/ecad
[root@lnx-gx270 ecad]# find . -mtime -1 -print
find: .: Value too large for defined data type

# xfs_growfs -n /ecad
meta-data=/ecad                  isize=256    agcount=168, agsize=260293 blks
data     =                       bsize=4096   blocks=43505086, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 1              bsize=4096
log      =internal               bsize=4096   blocks=1000
realtime =none                   extsz=65536  blocks=0, rtextents=0

-----------------------
Numerous other xlv filesystems that are the same type xlv/xfs/RAID5 mounted from
an IRIX server report no error.  A find in the following NFS mounted filesystem
works OK.

[root@lnx-gx270 ecad]# cd
[root@lnx-gx270 ~]# mkdir /mnt/bobcat
[root@lnx-gx270 ~]# mount bobcat:/export/sw_ibm /mnt/bobcat
[root@lnx-gx270 ~]# cd /mnt/bobcat
[root@lnx-gx270 bobcat]# df -k .
Filesystem           1K-blocks      Used Available Use% Mounted on
bobcat:/export/sw_ibm
                      65528000  57470752   8057248  88% /mnt/bobcat

[root@lnx-gx270 bobcat]# find . -mtime -1 -print
.
./test

bobcat-152# xfs_growfs -n /export/sw_ibm
meta-data=/export/sw_ibm         isize=512    agcount=63, agsize=262144 blks
data     =                       bsize=4096   blocks=16384000, imaxpct=25
         =                       sunit=8      swidth=8 blks, unwritten=1
naming   =version 2              bsize=4096
log      =internal               bsize=4096   blocks=2000
realtime =none                   extsz=65536  blocks=0, rtextents=0



Comment 10 Tim Waugh 2004-12-02 04:54:34 EST
See also:
  http://savannah.gnu.org/bugs/?func=detailitem&item_id=8455

Could you please try running this in that directory?:

ls -R1 | awk '(length > 200) {print length, $0;}' | sort -n | tail

What does it say?
Comment 12 Carrie Knox 2004-12-02 10:02:14 EST
[root@lnx-gx270 mink]# cd /mnt/mink
[root@lnx-gx270 mink]# pwd
/mnt/mink
[root@lnx-gx270 mink]# ls -R1 | awk '(length > 200) {print length, $0;}' | sort
-n | tail
root@lnx-gx270 mink]# 

Nothing reported...
Comment 13 Tim Waugh 2004-12-02 12:14:04 EST
Can you try capturing the NFS traffic:

1. tcpdump -v -w nfs.pcap -s 200 host mink
2. find /mnt/mink -mtime -1 -print

and attaching the nfs.pcap file?  Just press control-C on the tcpdump when it's
captured the NFS traffic from running 'find'.
Comment 14 Carrie Knox 2004-12-02 13:10:37 EST
Created attachment 107782 [details]
tcpdump -v -w /tmp/nfs.pcap -s 200 host mink

tcpdump output file is attached
Comment 15 Tim Waugh 2004-12-02 13:24:24 EST
Okay, that looks like NFSv3, which puts an end to the theory that it was a
combination of 2Gb files and NFSv2 causing the problem.
Comment 16 Tim Waugh 2004-12-06 04:34:58 EST
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=180065

This seems to indicate that it is in fact a kernel bug.
Comment 18 Carrie Knox 2004-12-14 08:54:45 EST
I've tried 2.6.9-1.698_FC3 and the problem still exists with findutils-4.1.20-7. 
Comment 19 Scott A. Friedman 2005-01-04 16:58:01 EST
kernel 2.6.9-1.724_FC3 still has this problem as well. Is there a
status update on this or a workaround I can implement locally?
Comment 20 Robin Holt 2005-02-09 18:42:08 EST
Created attachment 110904 [details]
Simple test program which trips the problem

This simple test program fails when run against an Irix 6.5.8 NFS server,
but works against an Irix 6.5.24 server.
Comment 21 Robin Holt 2005-02-09 18:45:18 EST
Created attachment 110905 [details]
strace output when run against Irix 6.5.8 NFS server
Comment 22 Robin Holt 2005-02-09 18:47:09 EST
Created attachment 110906 [details]
strace output when run against Irix 6.5.24 NFS server
Comment 23 Robin Holt 2005-02-09 19:54:40 EST
I looked into the Irix bug database.  This has been identified as
an incompatiblity issue between the NFS protocol standard and how
glibc expects 64bit system calls to work.  If the XFS filesystem
on the server was created with XFS version 2, the problem does not
occur.  A technical service bulletin was issued a couple years ago
identifying the symptons of this problem and the workaround.

I am working with our administrators to update the server that I
was having issues with.  I will add to this bug the results of
further testing.
Comment 24 Robin Holt 2005-02-09 19:58:21 EST
One more data point.  I tested this with a straight bitkeeper
export of 2.6.10 and the same problems were seen.
Comment 26 Orion Poplawski 2005-02-17 11:53:16 EST
We're seeing the same with an Irix 6.5.19m server.  Does anyone know if this is
fixed in 6.5.22 (the latest public maintenance release)?  Otherwise, workarounds
would be appreciated.  Thanks!
Comment 27 Robin Holt 2005-02-21 07:55:57 EST
Using XFS version 2 logs on our 6.5.8m box did work.  There was a discussion
about this problem on the linux kernel mailing list.

http://marc.theaimsgroup.com/?l=linux-kernel&m=110803989703599&w=2
Comment 29 Jakub Jelinek 2005-03-15 02:06:39 EST
This is not a glibc bug, but depending on how NFS protocol specifies this
either Irix NFS server bug, or Linux NFS client code bug.
struct dirent's d_off field has type off_t, which is a signed, not unsigned
type.  Therefore, when widening this to off64_t, it should be sign-extended,
not zero-extended.  The glibc side is not going to change to work around
bugs in kernel or Irix server, wherever it is.
Comment 30 Orion Poplawski 2005-03-16 09:27:45 EST
Moving to XFS 2 on the Irix server fixed this for me too.
Comment 31 Dave Jones 2005-07-15 13:52:38 EDT
An update has been released for Fedora Core 3 (kernel-2.6.12-1.1372_FC3) which
may contain a fix for your problem.   Please update to this new kernel, and
report whether or not it fixes your problem.

If you have updated to Fedora Core 4 since this bug was opened, and the problem
still occurs with the latest updates for that release, please change the version
field of this bug to 'fc4'.

Thank you.
Comment 32 Thomas J. Baker 2005-08-16 10:42:02 EDT
FWIW, I'm had the same problem with an RHEL4 client and an IRIX 6.5.25m server.
"Converting" (backup, remake, restore) the exported filesystem to XFS 2 fixed
the problem.
Comment 33 Eric Smith 2005-09-01 14:40:23 EDT
I have the same problem without any NFS being involved.  I'm running FC3 with
kernel 2.6.11-1.27_FC3, and Apache httpd-20.52-3.1, using only ext3 file
systems (no NFS).  One of my web directories contains a DVD .iso file larger
than 2GB.  Attempting to fetch it from the web server results in the client
being told that there is a permission error.  The error log says:

[Thu Sep 01 11:31:22 2005] [error] [client 67.111.12.152] (75)Value too large
for defined data type: access to
/~eric/software/fedora/fc4/stentz-dvd-i386/FC4-i386-DVD.iso failed

I'm updating the kernel now to 2.6.12-1.1376_FC3, and will report here later
today on whether it has fixed the problem.
Comment 34 Eric Smith 2005-09-01 15:26:29 EDT
Now running 2.6.12-1.1376_FC3, and the problem still occurs.  Same message to
log as before.
Comment 35 Steve Dickson 2005-09-01 20:56:13 EDT
I'm just curious if making the daemons in the nfs-utils rpm
64bit applications (i.e. defining the __USE_FILE_OFFSET64
compile flag) will help with this problem.

In http://people.redhat.com/steved/bz141167 is an nfs-utils rpm
that is compiled with __USE_FILE_OFFSET64 flag defined.
If your able, please try this experimental rpms to if it helps.
Comment 36 Dave Jones 2006-01-16 17:19:44 EST
This is a mass-update to all currently open Fedora Core 3 kernel bugs.

Fedora Core 3 support has transitioned to the Fedora Legacy project.
Due to the limited resources of this project, typically only
updates for new security issues are released.

As this bug isn't security related, it has been migrated to a
Fedora Core 4 bug.  Please upgrade to this newer release, and
test if this bug is still present there.

This bug has been placed in NEEDINFO_REPORTER state.
Due to the large volume of inactive bugs in bugzilla, if this bug is
still in this state in two weeks time, it will be closed.

Should this bug still be relevant after this period, the reporter
can reopen the bug at any time. Any other users on the Cc: list
of this bug can request that the bug be reopened by adding a
comment to the bug.

Thank you.
Comment 37 Dave Jones 2006-02-03 00:23:47 EST
This is a mass-update to all currently open kernel bugs.

A new kernel update has been released (Version: 2.6.15-1.1830_FC4)
based upon a new upstream kernel release.

Please retest against this new kernel, as a large number of patches
go into each upstream release, possibly including changes that
may address this problem.

This bug has been placed in NEEDINFO_REPORTER state.
Due to the large volume of inactive bugs in bugzilla, if this bug is
still in this state in two weeks time, it will be closed.

Should this bug still be relevant after this period, the reporter
can reopen the bug at any time. Any other users on the Cc: list
of this bug can request that the bug be reopened by adding a
comment to the bug.

If this bug is a problem preventing you from installing the
release this version is filed against, please see bug 169613.

Thank you.
Comment 38 John Thacker 2006-05-05 17:18:23 EDT
Closing per previous comment.

Note You need to log in before you can comment on or make changes to this bug.