Bug 1244613

Summary: using fop's dict for resolving causes problems
Product: [Community] GlusterFS Reporter: Raghavendra Bhat <rabhat>
Component: protocolAssignee: Raghavendra Bhat <rabhat>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: mainlineCC: bugs, gluster-bugs
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.8rc2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1244100
: 1252348 1255669 (view as bug list) Environment:
Last Closed: 2016-06-16 13:25:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1244100    
Bug Blocks: 1252348, 1255669    

Description Raghavendra Bhat 2015-07-20 07:20:58 UTC
+++ This bug was initially created as a clone of Bug #1244100 +++

Description of problem:

protocol/server tries to resolve the inodes (of both parent and the entry dipeneding upon the fop) whenever a fop comes to it, before continuing the fop. If it cannot find the inode in the inode table for a gfid (soft resolve), then a lookup is sent on it (hard resolve) to build the inode into the inode table. For sending the lookup as part of resolve it uses same xdata as that of the fop. This causes problems in the below situation.

Lru limit has reached. Because of that some of the inodes have been purged. If all the inodes for the dentries present in a directory are purged, then there is no one holding the ref on the parent directory's inode. So soon it also gets moved to the lru list from the active list and might get purged as well if no one holds a ref on that (means, no new entries are created in it and old entries present in it are not looked up).

Now if a create operation comes within that directory, protocol/server is not able to find the inode for the parent directory's gfid and sends a lookup on it. Now if some xlator wants to get some extended attributes as part of lookup, then they add their xattr names in the xdata (ex: bit-rot-stub adding version, sign and bad-object keys into the dict. And the xdata used here is same as that of the create fop's xdata). The lookup succeeds and now create happens with extra xattr names added in the xdata and posix as part of create sets those xattrs. Since bad-object key is also present, this leads to the object that is created to be treated as bad object for rest of its life and will not allow any i/o on it.

Version-Release number of selected component (if applicable):


How reproducible:
Easilu if inode-lru-limit is set to 1.

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

http://review.gluster.org/11661 has been sent to master and accepted.

--- Additional comment from Anand Avati on 2015-07-17 02:37:49 EDT ---

REVIEW: http://review.gluster.org/11703 (protocol/server: use different dict for resolving) posted (#1) for review on release-3.7 by Raghavendra Bhat (raghavendra)

Comment 1 Anand Avati 2015-07-20 10:44:15 UTC
REVIEW: http://review.gluster.org/11718 (tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy created) posted (#1) for review on master by Raghavendra Bhat (raghavendra)

Comment 2 Anand Avati 2015-07-21 11:28:58 UTC
REVIEW: http://review.gluster.org/11718 (tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy created) posted (#2) for review on master by Raghavendra Bhat (raghavendra)

Comment 3 Anand Avati 2015-07-24 13:44:47 UTC
REVIEW: http://review.gluster.org/11718 (tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy created) posted (#3) for review on master by Raghavendra Bhat (raghavendra)

Comment 4 Anand Avati 2015-07-27 06:44:59 UTC
REVIEW: http://review.gluster.org/11718 (tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy created) posted (#4) for review on master by Raghavendra Bhat (raghavendra)

Comment 5 Anand Avati 2015-07-29 07:03:31 UTC
REVIEW: http://review.gluster.org/11718 (tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy created) posted (#5) for review on master by Venky Shankar (vshankar)

Comment 6 Anand Avati 2015-07-31 05:32:42 UTC
REVIEW: http://review.gluster.org/11718 (tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy created) posted (#6) for review on master by Venky Shankar (vshankar)

Comment 7 Anand Avati 2015-08-03 05:59:56 UTC
REVIEW: http://review.gluster.org/11718 (tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy created) posted (#7) for review on master by Venky Shankar (vshankar)

Comment 8 Anand Avati 2015-08-04 09:04:32 UTC
REVIEW: http://review.gluster.org/11718 (tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy created) posted (#8) for review on master by Raghavendra Bhat (raghavendra)

Comment 9 Anand Avati 2015-08-10 11:48:01 UTC
REVIEW: http://review.gluster.org/11718 (tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy created) posted (#9) for review on master by Raghavendra Bhat (raghavendra)

Comment 10 Anand Avati 2015-08-11 04:22:13 UTC
COMMIT: http://review.gluster.org/11718 committed in master by Venky Shankar (vshankar) 
------
commit c0da330c32312edc4233fe6f660ce8e218638fb5
Author: Raghavendra Bhat <raghavendra>
Date:   Mon Jul 20 16:03:40 2015 +0530

    tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy created
    
    This test sets the lru limit of the inode table to 1 and checks if inode forgets
    and resolve cause any problem with bit-rot xattrs (especially bad-file xattr).
    
    Change-Id: I1fa25fa2d31dda8d26e8192562e896e5bddd0381
    BUG: 1244613
    Signed-off-by: Raghavendra Bhat <raghavendra>
    Reviewed-on: http://review.gluster.org/11718
    Tested-by: NetBSD Build System <jenkins.org>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Venky Shankar <vshankar>

Comment 11 Nagaprasad Sathyanarayana 2015-10-25 14:59:43 UTC
Fix for this BZ is already present in a GlusterFS release. You can find clone of this BZ, fixed in a GlusterFS release and closed. Hence closing this mainline BZ as well.

Comment 12 Niels de Vos 2016-06-16 13:25:33 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user