+++ This bug was initially created as a clone of Bug #829216 +++ Description of problem: After opening a file, a stat call on the file sometimes returns an inode number with the high 32 bits all ones. Subsequently it returns the normal inode number with high bits all zeroes. A read between the open and stat calls seems to greatly increase the likelyhood of the inode mangling occurring. Certain utilities depend on the inode number of a file staying constant. For example, find uses inode numbers to determine whether it arrived back in the same directory after visiting a subdirectory. If the inode number has changed, find gets confused and halts the search prematurely. Another effect is that stat calls can fail on 32-bit systems. The system call interface always uses a 64-bit struct stat, and for backwards (binary) compatibility glibc provides a 32-bit wrapper. This wrapper will return EOVERFLOW if some of the values in the 64-bit struct, such as inode number of size, are too large for the 32-bit fields. Presently I'm seeing this problem in libexiv2, which fails to extract metadata from images because it can't determine the size of the file. How reproducible: Always reproduces with the same access pattern on the same file. Not all files are affected. Steps to Reproduce: 1. execute a stat syscall on a file 2. open the file 3. read some data from the file (optional, increases likelyhood of bug occurrence) 4. execute the same stat syscall again Actual results: The second stat returns an st_ino which is 0xffffffff00000000|st_ino of the first call Expected results: Both stat calls return the same st_ino Additional info: Here's a sequence of system calls from a test program: stat64("stattest", {st_dev=makedev(0, 17), st_ino=1056645660, st_mode=S_IFREG|0755, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=16, st_size=5902, st_atime=2012/06/06-11:38:47, st_mtime=2012/06/06-11:38:46, st_ctime=2012/06/06-11:38:46}) = 0 open("stattest", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\2\0\3\0\1\0\0\0`\204\4\0104\0\0\0"..., 4096) = 4096 fstat64(3, {st_dev=makedev(0, 17), st_ino=18446744070471229980, st_mode=S_IFREG|0755, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=16, st_size=5902, st_atime=2012/06/06-11:38:47, st_mtime=2012/06/06-11:38:46, st_ctime=2012/06/06-11:38:46}) = 0 stat64("stattest", {st_dev=makedev(0, 17), st_ino=1056645660, st_mode=S_IFREG|0755, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=16, st_size=5902, st_atime=2012/06/06-11:38:47, st_mtime=2012/06/06-11:38:46, st_ctime=2012/06/06-11:38:46}) = 0 close(3) = 0 And another from exiv2: stat64("IMG_3722.CR2", {st_dev=makedev(0, 17), st_ino=2273457493, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0 open("IMG_3722.CR2", O_RDONLY) = 3 fstat64(3, {st_dev=makedev(0, 17), st_ino=2273457493, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0 read(3, "II*\0\20\0\0\0CR\2\0J\255\0\0\21\0\0\1\3\0\1\0\0\0\360\25\0\0\1\1"..., 131072) = 131072 _llseek(3, -131070, [2], SEEK_CUR) = 0 _llseek(3, 0, [0], SEEK_SET) = 0 read(3, "II*\0\20\0\0\0CR\2\0J\255\0\0\21\0\0\1\3\0\1\0\0\0\360\25\0\0\1\1"..., 131072) = 131072 _llseek(3, 131072, [131072], SEEK_SET) = 0 _llseek(3, 131072, [131072], SEEK_SET) = 0 _llseek(3, 131072, [131072], SEEK_SET) = 0 _llseek(3, 131072, [131072], SEEK_SET) = 0 close(3) = 0 open("IMG_3722.CR2", O_RDONLY) = 3 fstat64(3, {st_dev=makedev(0, 17), st_ino=2273457493, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0 read(3, "II*\0\20\0\0\0CR\2\0J\255\0\0\21\0\0\1\3\0\1\0\0\0\360\25\0\0\1\1"..., 131072) = 131072 _llseek(3, -131056, [16], SEEK_CUR) = 0 _llseek(3, 0, [0], SEEK_SET) = 0 close(3) = 0 open("IMG_3722.CR2", O_RDONLY) = 3 fstat64(3, {st_dev=makedev(0, 17), st_ino=2273457493, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0 read(3, "II*\0\20\0\0\0CR\2\0J\255\0\0\21\0\0\1\3\0\1\0\0\0\360\25\0\0\1\1"..., 131072) = 131072 _llseek(3, -131056, [16], SEEK_CUR) = 0 _llseek(3, 0, [0], SEEK_SET) = 0 stat64("IMG_3722.CR2", {st_dev=makedev(0, 17), st_ino=18446744071688041813, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0 stat64("IMG_3722.CR2", {st_dev=makedev(0, 17), st_ino=2273457493, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0 mmap2(NULL, 28232633, PROT_READ, MAP_SHARED, 3, 0) = 0xb5601000 close(3) = 0 GlusterFS volume info: Volume Name: home Type: Distribute Status: Started Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: capybara:/home Options Reconfigured: performance.io-cache: on nfs.enable-ino32: off --- Additional comment from tdb on 2012-06-06 08:22:20 EDT --- Some additional data points: Mounting the filesystem with --attribute-timeout=0 makes it consistently report the inode numbers with high bits all ones. The 32-bit stat wrapper can be worked around by recompiling affected software with -D_FILE_OFFSET_BITS=64.
This bug is not seen in current master branch (which will get branched as RHS 2.1.0 soon). To consider it for fixing, want to make sure this bug still exists in RHS servers. If not reproduced, would like to close this.
With the latest git version of glusterfs (as of maybe an hour ago), I can no longer reproduce the bug on my Debian system.
closed as per the comment #3 Thanks for confirming Mikko.