Bug 764773 (GLUSTER-3041)

Summary:	[glusterfs-3.1.5qa2]: stat gives EINVAL
Product:	[Community] GlusterFS	Reporter:	Raghavendra Bhat <rabhat>
Component:	stat-prefetch	Assignee:	Raghavendra G <raghavendra>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	medium	Docs Contact:
Priority:	medium
Version:	3.1.4	CC:	anush, jdarcy, rahulcs
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	RTA	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Raghavendra Bhat 2011-06-16 09:28:37 UTC

distributed replicate setup. (2 servers => 2 bricks on each server). 1 fuse client and 1 nfs client.

running test as a normal user by chowning a directory on the mount point to the user. On fuse client dbench 100 and on nfs client clone a glusterfs git repository, autogen, configure and make.

On the fuse client after dbench is complete stating glusterfs.git directory says Invalid argument.


This is what the client log file in debug mode says.

[2011-06-16 07:14:17.727953] D [client3_1-fops.c:1937:client3_1_lookup_cbk] 2-mirror-client-2: gfid changed for /dir/glusterfs.git
[2011-06-16 07:14:17.728045] D [client3_1-fops.c:1937:client3_1_lookup_cbk] 2-mirror-client-3: gfid changed for /dir/glusterfs.git
[2011-06-16 07:14:17.728069] I [dht-common.c:240:dht_revalidate_cbk] 2-mirror-dht: subvolume mirror-replicate-1 for /dir/glusterfs.git returned -1 (Invalid argument)

Attaching gdb to the mount point and having break point to client3_1_lookup_cbk reveals that the gfid of the inode stored in the frame->local.loc and the gfid in the inode attribute for the inode obtained from the server are different.

Turning off stat-prefetch seemed to correct the problem and EINVAL was not observed upo stating the glusterfs.git directory. But rm -rf on the mount point (fuse) of the glusterfs.git again gave EINVAL. rm -rf worked fine on the nfs mount point.

Running the same test with stat-prefetch turned off from the beginning did not reproduce the issue.

Comment 1 Anand Avati 2011-06-22 12:41:00 UTC

PATCH: http://patches.gluster.com/patch/7588 in master (fuse: consider a lookup as revalidate even if the inode is present in new graph.)

Comment 2 Anand Avati 2011-06-22 12:41:44 UTC

PATCH: http://patches.gluster.com/patch/7576 in release-3.1 (fuse: consider a lookup as revalidate even if the inode is present in new graph.)

Comment 3 Anand Avati 2011-06-22 12:42:28 UTC

PATCH: http://patches.gluster.com/patch/7589 in release-3.2 (fuse: consider a lookup as revalidate even if the inode is present in new graph.)

Comment 4 Raghavendra G 2011-06-23 00:58:50 UTC

A simple test to reproduce the bug would be:

1. create a directory on mount and cd into it - /mnt/gluster1/dir

2. have another mount and create a file in that directory

/mnt/gluster2/dir# touch file

3. do ls on the first mount

/mnt/gluster1/dir# ls file

4. create a new graph in both clients by turning off a translator

#gluster volume ptop performance.write-behind off

5. on client2, remove and recreate file

/mnt/gluster2/dir# rm -f file && touch file

6. on client1, ls on the file should produce junk output or some error

/mnt/gluster1/dir# ls file

Comment 5 Rahul C S 2011-07-05 08:59:33 UTC

Works with 3.2.2qa1

Comment 6 Vijay Bellur 2011-07-07 10:03:56 UTC

PATCH: http://patches.gluster.com/patch/7786 in release-3.2 (mnt/fuse: generate uuids in fuse_lookup, not in fuse_lookup_resume.)

Comment 7 Anand Avati 2011-07-12 10:59:33 UTC

PATCH: http://patches.gluster.com/patch/7769 in master (mnt/fuse: generate uuids in fuse_lookup, not in fuse_lookup_resume.)

Comment 8 Anand Avati 2011-07-12 10:59:53 UTC

PATCH: http://patches.gluster.com/patch/7770 in release-3.1 (mnt/fuse: generate uuids in fuse_lookup, not in fuse_lookup_resume.)

Comment 9 Rahul C S 2011-08-05 12:28:38 UTC

Tried to reproduce with Du's comment on reproduce the bug.
Tested against 0c01d96a065deb992a8c3a0b24c74a70ab99ea86 on 3.1 branch