Bug 829216 - stat reports a different inode number after open
stat reports a different inode number after open
Status: CLOSED WORKSFORME
Product: GlusterFS
Classification: Community
Component: fuse (Show other bugs)
3.2.6
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: shishir gowda
: Triaged
Depends On:
Blocks: 848345
  Show dependency treegraph
 
Reported: 2012-06-06 04:55 EDT by Mikko Rasa
Modified: 2013-12-08 20:32 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 848345 (view as bug list)
Environment:
Last Closed: 2012-09-11 04:45:31 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Mikko Rasa 2012-06-06 04:55:31 EDT
Description of problem:
After opening a file, a stat call on the file sometimes returns an inode number with the high 32 bits all ones.  Subsequently it returns the normal inode number with high bits all zeroes.  A read between the open and stat calls seems to greatly increase the likelyhood of the inode mangling occurring.

Certain utilities depend on the inode number of a file staying constant.  For example, find uses inode numbers to determine whether it arrived back in the same directory after visiting a subdirectory.  If the inode number has changed, find gets confused and halts the search prematurely.

Another effect is that stat calls can fail on 32-bit systems.  The system call interface always uses a 64-bit struct stat, and for backwards (binary) compatibility glibc provides a 32-bit wrapper.  This wrapper will return EOVERFLOW if some of the values in the 64-bit struct, such as inode number of size, are too large for the 32-bit fields.  Presently I'm seeing this problem in libexiv2, which fails to extract metadata from images because it can't determine the size of the file.

How reproducible:
Always reproduces with the same access pattern on the same file.  Not all files are affected.

Steps to Reproduce:
1. execute a stat syscall on a file
2. open the file
3. read some data from the file (optional, increases likelyhood of bug occurrence)
4. execute the same stat syscall again
  
Actual results:
The second stat returns an st_ino which is 0xffffffff00000000|st_ino of the first call

Expected results:
Both stat calls return the same st_ino

Additional info:
Here's a sequence of system calls from a test program:
stat64("stattest", {st_dev=makedev(0, 17), st_ino=1056645660, st_mode=S_IFREG|0755, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=16, st_size=5902, st_atime=2012/06/06-11:38:47, st_mtime=2012/06/06-11:38:46, st_ctime=2012/06/06-11:38:46}) = 0
open("stattest", O_RDONLY)              = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\2\0\3\0\1\0\0\0`\204\4\0104\0\0\0"..., 4096) = 4096
fstat64(3, {st_dev=makedev(0, 17), st_ino=18446744070471229980, st_mode=S_IFREG|0755, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=16, st_size=5902, st_atime=2012/06/06-11:38:47, st_mtime=2012/06/06-11:38:46, st_ctime=2012/06/06-11:38:46}) = 0
stat64("stattest", {st_dev=makedev(0, 17), st_ino=1056645660, st_mode=S_IFREG|0755, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=16, st_size=5902, st_atime=2012/06/06-11:38:47, st_mtime=2012/06/06-11:38:46, st_ctime=2012/06/06-11:38:46}) = 0
close(3)                                = 0

And another from exiv2:
stat64("IMG_3722.CR2", {st_dev=makedev(0, 17), st_ino=2273457493, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0
open("IMG_3722.CR2", O_RDONLY)          = 3
fstat64(3, {st_dev=makedev(0, 17), st_ino=2273457493, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0
read(3, "II*\0\20\0\0\0CR\2\0J\255\0\0\21\0\0\1\3\0\1\0\0\0\360\25\0\0\1\1"..., 131072) = 131072
_llseek(3, -131070, [2], SEEK_CUR)      = 0
_llseek(3, 0, [0], SEEK_SET)            = 0
read(3, "II*\0\20\0\0\0CR\2\0J\255\0\0\21\0\0\1\3\0\1\0\0\0\360\25\0\0\1\1"..., 131072) = 131072
_llseek(3, 131072, [131072], SEEK_SET)  = 0
_llseek(3, 131072, [131072], SEEK_SET)  = 0
_llseek(3, 131072, [131072], SEEK_SET)  = 0
_llseek(3, 131072, [131072], SEEK_SET)  = 0
close(3)                                = 0
open("IMG_3722.CR2", O_RDONLY)          = 3
fstat64(3, {st_dev=makedev(0, 17), st_ino=2273457493, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0
read(3, "II*\0\20\0\0\0CR\2\0J\255\0\0\21\0\0\1\3\0\1\0\0\0\360\25\0\0\1\1"..., 131072) = 131072
_llseek(3, -131056, [16], SEEK_CUR)     = 0
_llseek(3, 0, [0], SEEK_SET)            = 0
close(3)                                = 0
open("IMG_3722.CR2", O_RDONLY)          = 3
fstat64(3, {st_dev=makedev(0, 17), st_ino=2273457493, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0
read(3, "II*\0\20\0\0\0CR\2\0J\255\0\0\21\0\0\1\3\0\1\0\0\0\360\25\0\0\1\1"..., 131072) = 131072
_llseek(3, -131056, [16], SEEK_CUR)     = 0
_llseek(3, 0, [0], SEEK_SET)            = 0
stat64("IMG_3722.CR2", {st_dev=makedev(0, 17), st_ino=18446744071688041813, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0
stat64("IMG_3722.CR2", {st_dev=makedev(0, 17), st_ino=2273457493, st_mode=S_IFREG|0600, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=131072, st_blocks=55144, st_size=28232633, st_atime=2012/06/05-22:43:31, st_mtime=2012/04/21-14:43:46, st_ctime=2012/06/05-22:43:00}) = 0
mmap2(NULL, 28232633, PROT_READ, MAP_SHARED, 3, 0) = 0xb5601000
close(3)                                = 0

GlusterFS volume info:
Volume Name: home
Type: Distribute
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: capybara:/home
Options Reconfigured:
performance.io-cache: on
nfs.enable-ino32: off
Comment 1 Mikko Rasa 2012-06-06 08:22:20 EDT
Some additional data points:
Mounting the filesystem with --attribute-timeout=0 makes it consistently report the inode numbers with high bits all ones.

The 32-bit stat wrapper can be worked around by recompiling affected software with -D_FILE_OFFSET_BITS=64.
Comment 2 Amar Tumballi 2012-09-11 04:45:31 EDT
Mikko, I am also closing this issue as you confirmed in bug 848345 (com#3)

Note You need to log in before you can comment on or make changes to this bug.