Bug 764774 (GLUSTER-3042)

Summary: inode number should be constant on storage
Product: [Community] GlusterFS Reporter: Amar Tumballi <amarts>
Component: coreAssignee: Amar Tumballi <amarts>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: medium    
Version: mainlineCC: aavati, gluster-bugs, jdarcy, saurabh, vraman
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Jeff Darcy 2011-06-16 11:04:53 UTC
As an extension to this, I'd like to propose that the GFID-based inode number be returned all the way up through AFR (which can return different values from different replicas) and DHT (which can return different values whenever bricks are added/removed because of the way dht_*itransform works).  Having inode numbers that are stable despite underlying configuration changes would be awesome.

Comment 1 Amar Tumballi 2011-06-16 11:17:55 UTC
so, to achieve this, consider getting the inode number from gfid itself, which is guaranteed to be uniq.

Let posix itself send the 'st_ino' calculated from 'gfid'. This will protect the inode number returning logic of glusterfs even in case of backend inode number reallocation in backend filesystems.

Comment 2 Anand Avati 2011-06-16 12:08:34 UTC
(In reply to comment #1)
> As an extension to this, I'd like to propose that the GFID-based inode number
> be returned all the way up through AFR (which can return different values from
> different replicas) and DHT (which can return different values whenever bricks
> are added/removed because of the way dht_*itransform works).  Having inode
> numbers that are stable despite underlying configuration changes would be
> awesome.

http://patches.gluster.com/patch/7524/ does exactly that.

Comment 3 Jeff Darcy 2011-06-16 12:38:34 UTC
(In reply to comment #2) 
> http://patches.gluster.com/patch/7524/ does exactly that.

Most excellent.

Comment 4 Anand Avati 2011-06-17 03:25:00 UTC
PATCH: http://patches.gluster.com/patch/7524 in master (core: fill 'ia_ino' from 'ia_gfid' in 'storage/posix' to preserve same ino number)

Comment 5 Amar Tumballi 2011-06-17 03:30:29 UTC
now the inode number (st_ino) of the stat structure is derived from gfid, hence, we are _almost_ guaranteed to have unique inode in cluster without any 'itransform' or anything. Which will help in preserving inode numbers even after a scale-out or scale-down operation.

Comment 6 Saurabh 2011-07-22 07:38:47 UTC
the inodes are found to be different from different clients,

on client1,
glusterfs#10.1.12.134:/dr3 on /mnt/fuse-test type fuse (rw,allow_other,default_permissions,max_read=131072)
[root@centos-qa-client-3 ~]# ls -li /mnt/fuse-test/
total 280
18446744053835159371 -rw-r--r-- 1 root root 1024 Jul 22 01:19 f.1
18446744055578181635 -rw-r--r-- 1 root root 1024 Jul 22 01:13 f.10
18446744058569784671 -rw-r--r-- 1 root root 1024 Jul 22 01:13 f.11
18446744054862085589 -rw-r--r-- 1 root root 1024 Jul 22 01:13 f.12
          3611390631 -rw-r--r-- 1 root root 1024 Jul 22 01:19 f.13


on client2,
[root@centos-qa-client-2 ~]# ls -li /mnt/quota_nfs
total 280
18446744070397152908 -rw-r--r-- 1 root root 1024 Jul 22 01:19 f.1
18446744070687656619 -rw-r--r-- 1 root root 1024 Jul 22 01:19 f.10
18446744071186257125 -rw-r--r-- 1 root root 1024 Jul 22 01:19 f.11
18446744070568307278 -rw-r--r-- 1 root root 1024 Jul 22 01:19 f.12
           601898438 -rw-r--r-- 1 root root 1024 Jul 22 01:19 f.13



even on client1,

re-mounting gave different values,
[root@centos-qa-client-3 ~]# umount /mnt/fuse-test
[root@centos-qa-client-3 ~]# mount -t glusterfs 10.1.12.134:/dr3 /mnt/fuse-test/
[root@centos-qa-client-3 ~]# ls -li /mnt/fuse-test/
total 280
18446744053835159368 -rw-r--r-- 1 root root 1024 Jul 22 01:13 f.1
18446744055578181638 -rw-r--r-- 1 root root 1024 Jul 22 01:19 f.10
18446744058569784674 -rw-r--r-- 1 root root 1024 Jul 22 01:19 f.11
18446744054862085592 -rw-r--r-- 1 root root 1024 Jul 22 01:19 f.12
          3611390628 -rw-r--r-- 1 root root 1024 Jul 22 01:13 f.13

Comment 7 Saurabh 2011-07-22 07:40:20 UTC
for the above test, the master is sync'd to this point,

commit 37b4b7ccad5a479ed2ce96598e902a12fd457bd5
Author: Raghavendra Bhat <raghavendrabhat>
Date:   Wed Jul 20 12:21:12 2011 +0530

    debug/io-stats: allow log-level to be set to INFO
    
    Change-Id: Ieba6591b0641dcb7dad724c1d8199dc5e91b6bfd

Comment 8 Amar Tumballi 2011-07-26 02:08:02 UTC
Tried the tests to check if there are changes in inode number across re-mounts, different mounts etc. In all tests it was same. (Tried it in master branch).

Comment 9 Saurabh 2011-07-26 08:32:33 UTC
with the latest sync to the repository , I am also not finding this issue, even after replace brick the inode number does not change.


But I don't how come the issue was seen earlier!

Comment 10 Saurabh 2011-07-26 08:32:49 UTC
verified it

Comment 11 Anand Avati 2011-07-26 08:56:56 UTC
(In reply to comment #9)
> with the latest sync to the repository , I am also not finding this issue, even
> after replace brick the inode number does not change.
> 
> 
> But I don't how come the issue was seen earlier!

Can you not rollback to the old code snapshot and check if it actually was dude to the presence/absence of the stable inode number patch?

Comment 12 Anand Avati 2011-10-24 03:00:51 UTC
CHANGE: http://review.gluster.com/624 (take the least significant 64bit from gfid and assign it to 'ia_ino',) merged in release-3.2 by Vijay Bellur (vijay)

Comment 13 Anand Avati 2011-11-09 03:10:12 UTC
CHANGE: http://review.gluster.com/632 (as posix layer itself sends inode numbers based on gfid now) merged in release-3.2 by Vijay Bellur (vijay)