Bug 1621981

Summary:	dht: File rename removes the .glusterfs handle for linkto file
Product:	[Community] GlusterFS	Reporter:	Nithya Balachandran <nbalacha>
Component:	distribute	Assignee:	Nithya Balachandran <nbalacha>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	mainline	CC:	bugs
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-5.0	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1622001 1625134 (view as bug list)		Environment:
Last Closed:	2018-10-23 15:17:29 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1619623, 1622001, 1625134

Description Nithya Balachandran 2018-08-24 05:40:39 UTC

Description of problem:

dht_rename_cbk sends a linkfile create in parallel with the unlinks to clean up the original src linkto files. If these fops reach the brick out of order, we end up a linkto file without a gfid handle


Version-Release number of selected component (if applicable):


How reproducible:
Often

Steps to Reproduce:
1. Run glusterfs/tests/bugs/posix/bug-1619720.t over and over again and check the file listing on both bricks.


Actual results:

/bricks/brick1/backends/patchy0/tmp/:
total 0
---------T. 1 root root 0 Aug 23 20:04 file-3  <-- Only one link

/bricks/brick1/backends/patchy1/tmp/:
total 0
-rw-r--r--. 2 root root 0 Aug 23 20:04 file-3

Expected results:

/bricks/brick1/backends/patchy0/tmp/:
total 0
---------T. 2 root root 0 Aug 23 20:04 file-3

/bricks/brick1/backends/patchy1/tmp/:
total 0
-rw-r--r--. 2 root root 0 Aug 23 20:04 file-3

Additional info:

Comment 1 Nithya Balachandran 2018-08-24 05:46:43 UTC

Success case:

[2018-08-23 15:35:39.449830] E [MSGID: 113018] [posix-entry-ops.c:372:posix_mknod] 0-patchy-posix: debug: Calling mknod for /bricks/brick1/backends/patchy0/tmp/file-3
[2018-08-23 15:35:39.449874] W [MSGID: 113096] [posix-handle.c:997:posix_create_link_if_gfid_exists] 0-patchy-posix: debug: link if handle exists /bricks/brick1/backends/patchy0/.glusterfs/5d/1d/5d1d04c7-a64e-41dd-89e0-1cbd47123831 (5d1d04c7-a64e-41dd-89e0-1cbd47123831)
[2018-08-23 15:35:39.449898] W [MSGID: 113096] [posix-handle.c:1011:posix_create_link_if_gfid_exists] 0-patchy-posix: debug: link  /bricks/brick1/backends/patchy0/.glusterfs/5d/1d/5d1d04c7-a64e-41dd-89e0-1cbd47123831, /bricks/brick1/backends/patchy0/tmp/file-3 (0)
[2018-08-23 15:35:39.450396] E [MSGID: 113018] [posix-entry-ops.c:1119:posix_unlink] 0-patchy-posix: debug: Calling unlink for /bricks/brick1/backends/patchy0/tmp/file-1  
[2018-08-23 15:35:39.450437] E [MSGID: 113018] [posix-entry-ops.c:965:posix_unlink_gfid_handle_and_entry] 0-patchy-posix: debug: Calling unlink_gfid_handle for /bricks/brick1/backends/patchy0/tmp/file-1, 2



Failure case:

[2018-08-23 15:38:48.480424] E [MSGID: 113018] [posix-entry-ops.c:372:posix_mknod] 0-patchy-posix: debug: Calling mknod for /bricks/brick1/backends/patchy0/tmp/file-3
[2018-08-23 15:38:48.480437] E [MSGID: 113018] [posix-entry-ops.c:1119:posix_unlink] 0-patchy-posix: debug: Calling unlink for /bricks/brick1/backends/patchy0/tmp/file-1
[2018-08-23 15:38:48.480462] E [MSGID: 113018] [posix-entry-ops.c:965:posix_unlink_gfid_handle_and_entry] 0-patchy-posix: debug: Calling unlink_gfid_handle for /bricks/brick1/backends/patchy0/tmp/file-1, 1
[2018-08-23 15:38:48.480474] W [MSGID: 113096] [posix-handle.c:997:posix_create_link_if_gfid_exists] 0-patchy-posix: debug: link if handle exists /bricks/brick1/backends/patchy0/.glusterfs/8a/d4/8ad46424-529f-4ec8-a8c3-03b94168d470 (8ad46424-529f-4ec8-a8c3-03b94168d470)

Comment 2 Worker Ant 2018-08-28 06:49:00 UTC

REVIEW: https://review.gluster.org/21023 (cluster/dht: In rename, unlink after creating linkto file) posted (#1) for review on master by N Balachandran

Comment 3 Worker Ant 2018-09-03 05:42:42 UTC

COMMIT: https://review.gluster.org/21023 committed in master by "Raghavendra G" <rgowdapp> with a commit message- cluster/dht: In rename, unlink after creating linkto file

The linkto file creation for the dst was done in parallel with
the unlink of the old src linkto. If these operations reached
the brick out of order, we end up with a dst linkto file without
a .glusterfs handle.

Fixed by the unlinking only after the linkto file creation has
completed.

Change-Id: I4246f7655f5bc180f5ded7fd34d263b7828a8110
fixes: bz#1621981
Signed-off-by: N Balachandran <nbalacha>

Comment 4 Shyamsundar 2018-10-23 15:17:29 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-5.0, please open a new bug report.

glusterfs-5.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://lists.gluster.org/pipermail/announce/2018-October/000115.html
[2] https://www.gluster.org/pipermail/gluster-users/