Bug 1684969 - New GFID file recreated in a replica set after a GFID mismatch resolution
Summary: New GFID file recreated in a replica set after a GFID mismatch resolution
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: distribute
Version: 6
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact:
URL:
Whiteboard:
Depends On: 1661258 1670259
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-04 03:13 UTC by Nithya Balachandran
Modified: 2019-03-04 03:18 UTC (History)
1 user (show)

Fixed In Version:
Clone Of: 1670259
Environment:
Last Closed: 2019-03-04 03:18:27 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:

Attachments (Terms of Use)

Description Nithya Balachandran 2019-03-04 03:13:48 UTC 
+++ This bug was initially created as a clone of Bug #1670259 +++

If the gfid is not set for a directory on one brick, a lookup on that directory will cause a different GFID to be set on it, leading to a GFID mismatch.

--- Additional comment from Nithya Balachandran on 2019-01-29 04:56:49 UTC ---

Steps to reproduce the problem:

1. Create a 3 brick distribute volume
2. Fuse mount the volume and create some directories.
cd /mnt/fuse1
mkdir gfid-mismatch
mkdir gfid-mismatch/dir-1

3. delete the gfid and .glusterfs handle from the hashed brick

[root@rhgs313-6 brick1]# setfattr -x trusted.gfid vol1-1/gfid-mismatch/dir-1
[root@rhgs313-6 brick1]# unlink vol1-1/.glusterfs/8e/6c/8e6c686c-93e9-4bd7-ac5e-98bbf852a62b
[root@rhgs313-6 brick1]# 
[root@rhgs313-6 brick1]# 
[root@rhgs313-6 brick1]# 
[root@rhgs313-6 brick1]# getx vol1-*/gfid-mismatch/dir-1
# file: vol1-1/gfid-mismatch/dir-1
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.glusterfs.dht=0x00000000000000000000000055555554
trusted.glusterfs.dht.mds=0x00000000
trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3

# file: vol1-2/gfid-mismatch/dir-1
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.gfid=0x8e6c686c93e94bd7ac5e98bbf852a62b
trusted.glusterfs.dht=0x000000000000000055555555aaaaaaa9
trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3

# file: vol1-3/gfid-mismatch/dir-1
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.gfid=0x8e6c686c93e94bd7ac5e98bbf852a62b
trusted.glusterfs.dht=0x0000000000000000aaaaaaaaffffffff
trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3


4. Unmount and remount the fuse mount and list the contents of gfid-mismatch

[root@rhgs313-6 ~]# umount -f /mnt/fuse1; mount -t glusterfs -s 192.168.122.6:/vol1 /mnt/fuse1
[root@rhgs313-6 ~]# cd /mnt/fuse1
[root@rhgs313-6 fuse1]# cd gfid-mismatch/
[root@rhgs313-6 gfid-mismatch]# l
total 20K
drwxr-xr-x. 2 root root 4.0K Jan 29 09:20 dir-1


5. Check the gfid for dir-1 on the backend bricks.

[root@rhgs313-6 brick1]# getx vol1-*/gfid-mismatch/dir-1
# file: vol1-1/gfid-mismatch/dir-1
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.gfid=0x0c3a4860f93f416cb5261c3b2b06f52d    < ---- GFID is DIFFERENT!!
trusted.glusterfs.dht=0x00000000000000000000000055555554
trusted.glusterfs.dht.mds=0x00000000
trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3

# file: vol1-2/gfid-mismatch/dir-1
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.gfid=0x8e6c686c93e94bd7ac5e98bbf852a62b
trusted.glusterfs.dht=0x000000000000000055555555aaaaaaa9
trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3

# file: vol1-3/gfid-mismatch/dir-1
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.gfid=0x8e6c686c93e94bd7ac5e98bbf852a62b
trusted.glusterfs.dht=0x0000000000000000aaaaaaaaffffffff
trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3



The GFID on brick vol1-1 is set to 
trusted.gfid=0x0c3a4860f93f416cb5261c3b2b06f52d 

The original GFID was:
trusted.gfid=0x8e6c686c93e94bd7ac5e98bbf852a62b

--- Additional comment from Nithya Balachandran on 2019-01-29 04:58:11 UTC ---

Upstream patch:
https://review.gluster.org/#/c/glusterfs/+/22112/

--- Additional comment from Worker Ant on 2019-02-02 03:10:15 UTC ---

REVIEW: https://review.gluster.org/22112 (cluster/dht: Do not use gfid-req in fresh lookup) merged (#7) on master by Amar Tumballi
Note You need to log in before you can comment on or make changes to this bug.