If the gfid is not set for a directory on one brick, a lookup on that directory will cause a different GFID to be set on it, leading to a GFID mismatch.
Steps to reproduce the problem: 1. Create a 3 brick distribute volume 2. Fuse mount the volume and create some directories. cd /mnt/fuse1 mkdir gfid-mismatch mkdir gfid-mismatch/dir-1 3. delete the gfid and .glusterfs handle from the hashed brick [root@rhgs313-6 brick1]# setfattr -x trusted.gfid vol1-1/gfid-mismatch/dir-1 [root@rhgs313-6 brick1]# unlink vol1-1/.glusterfs/8e/6c/8e6c686c-93e9-4bd7-ac5e-98bbf852a62b [root@rhgs313-6 brick1]# [root@rhgs313-6 brick1]# [root@rhgs313-6 brick1]# [root@rhgs313-6 brick1]# getx vol1-*/gfid-mismatch/dir-1 # file: vol1-1/gfid-mismatch/dir-1 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000 trusted.glusterfs.dht=0x00000000000000000000000055555554 trusted.glusterfs.dht.mds=0x00000000 trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3 # file: vol1-2/gfid-mismatch/dir-1 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000 trusted.gfid=0x8e6c686c93e94bd7ac5e98bbf852a62b trusted.glusterfs.dht=0x000000000000000055555555aaaaaaa9 trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3 # file: vol1-3/gfid-mismatch/dir-1 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000 trusted.gfid=0x8e6c686c93e94bd7ac5e98bbf852a62b trusted.glusterfs.dht=0x0000000000000000aaaaaaaaffffffff trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3 4. Unmount and remount the fuse mount and list the contents of gfid-mismatch [root@rhgs313-6 ~]# umount -f /mnt/fuse1; mount -t glusterfs -s 192.168.122.6:/vol1 /mnt/fuse1 [root@rhgs313-6 ~]# cd /mnt/fuse1 [root@rhgs313-6 fuse1]# cd gfid-mismatch/ [root@rhgs313-6 gfid-mismatch]# l total 20K drwxr-xr-x. 2 root root 4.0K Jan 29 09:20 dir-1 5. Check the gfid for dir-1 on the backend bricks. [root@rhgs313-6 brick1]# getx vol1-*/gfid-mismatch/dir-1 # file: vol1-1/gfid-mismatch/dir-1 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000 trusted.gfid=0x0c3a4860f93f416cb5261c3b2b06f52d < ---- GFID is DIFFERENT!! trusted.glusterfs.dht=0x00000000000000000000000055555554 trusted.glusterfs.dht.mds=0x00000000 trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3 # file: vol1-2/gfid-mismatch/dir-1 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000 trusted.gfid=0x8e6c686c93e94bd7ac5e98bbf852a62b trusted.glusterfs.dht=0x000000000000000055555555aaaaaaa9 trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3 # file: vol1-3/gfid-mismatch/dir-1 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a756e6c6162656c65645f743a733000 trusted.gfid=0x8e6c686c93e94bd7ac5e98bbf852a62b trusted.glusterfs.dht=0x0000000000000000aaaaaaaaffffffff trusted.glusterfs.mdata=0x010000000000000000000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3000000005c4fcd7500000000017863f3 The GFID on brick vol1-1 is set to trusted.gfid=0x0c3a4860f93f416cb5261c3b2b06f52d The original GFID was: trusted.gfid=0x8e6c686c93e94bd7ac5e98bbf852a62b
Upstream patch: https://review.gluster.org/#/c/glusterfs/+/22112/
REVIEW: https://review.gluster.org/22112 (cluster/dht: Do not use gfid-req in fresh lookup) merged (#7) on master by Amar Tumballi
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-6.0, please open a new bug report. glusterfs-6.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] https://lists.gluster.org/pipermail/announce/2019-March/000120.html [2] https://www.gluster.org/pipermail/gluster-users/