Bug 1272949

Summary:	I/O failure on attaching tier on nfs client
Product:	[Community] GlusterFS	Reporter:	Mohammed Rafi KC <rkavunga>
Component:	tiering	Assignee:	bugs <bugs>
Status:	CLOSED CURRENTRELEASE	QA Contact:	bugs <bugs>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	mainline	CC:	bugs, sankarshan
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.8rc2	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1278399 (view as bug list)		Environment:
Last Closed:	2016-06-16 13:40:47 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1278399, 1279095

Description Mohammed Rafi KC 2015-10-19 09:44:32 UTC

Description of problem:

on going I/o's are failing when attaching tier

Version-Release number of selected component (if applicable):


How reproducible:

100

Steps to Reproduce:
1.create a dist-rep volume
2.mount and start linux untar on mount point
3.attach-tier

Actual results:

i/o failure

Expected results:

i/o should not fail

Additional info:

Comment 1 Vijay Bellur 2015-10-19 09:46:22 UTC

REVIEW: http://review.gluster.org/12376 (dht: heal directory path if the directory is nor present) posted (#2) for review on master by mohammed rafi  kc (rkavunga)

Comment 2 Vijay Bellur 2015-10-19 09:47:10 UTC

REVIEW: http://review.gluster.org/12375 (Revert "fuse: resolve complete path after a graph switch") posted (#3) for review on master by mohammed rafi  kc (rkavunga)

Comment 3 Vijay Bellur 2015-10-20 04:51:58 UTC

REVIEW: http://review.gluster.org/12375 (Revert "fuse: resolve complete path after a graph switch") posted (#4) for review on master by mohammed rafi  kc (rkavunga)

Comment 4 Vijay Bellur 2015-10-20 04:52:01 UTC

REVIEW: http://review.gluster.org/12376 (dht: heal directory path if the directory is nor present) posted (#3) for review on master by mohammed rafi  kc (rkavunga)

Comment 5 Vijay Bellur 2015-10-22 18:54:39 UTC

REVIEW: http://review.gluster.org/12375 (Revert "fuse: resolve complete path after a graph switch") posted (#5) for review on master by mohammed rafi  kc (rkavunga)

Comment 6 Vijay Bellur 2015-10-22 18:54:42 UTC

REVIEW: http://review.gluster.org/12376 (dht: heal directory path if the directory is nor present) posted (#4) for review on master by mohammed rafi  kc (rkavunga)

Comment 7 Vijay Bellur 2015-10-22 18:54:46 UTC

REVIEW: http://review.gluster.org/12414 (dht:heal layout after a nameless lookup) posted (#1) for review on master by mohammed rafi  kc (rkavunga)

Comment 8 Vijay Bellur 2015-10-28 19:52:44 UTC

REVIEW: http://review.gluster.org/12375 (Revert "fuse: resolve complete path after a graph switch") posted (#6) for review on master by mohammed rafi  kc (rkavunga)

Comment 9 Vijay Bellur 2015-10-28 19:52:47 UTC

REVIEW: http://review.gluster.org/12376 (dht: heal directory path if the directory is not present) posted (#5) for review on master by mohammed rafi  kc (rkavunga)

Comment 10 Vijay Bellur 2015-10-28 19:52:50 UTC

REVIEW: http://review.gluster.org/12449 (dht: update cached subvolume during readdirp cbk) posted (#1) for review on master by mohammed rafi  kc (rkavunga)

Comment 11 Vijay Bellur 2015-10-30 11:43:56 UTC

REVIEW: http://review.gluster.org/12375 (Revert "fuse: resolve complete path after a graph switch") posted (#7) for review on master by mohammed rafi  kc (rkavunga)

Comment 12 Vijay Bellur 2015-10-30 11:44:00 UTC

REVIEW: http://review.gluster.org/12449 (dht: update cached subvolume during readdirp cbk) posted (#2) for review on master by mohammed rafi  kc (rkavunga)

Comment 13 Vijay Bellur 2015-10-30 11:44:09 UTC

REVIEW: http://review.gluster.org/12376 (dht: heal directory path if the directory is not present) posted (#6) for review on master by mohammed rafi  kc (rkavunga)

Comment 14 Vijay Bellur 2015-10-30 12:58:17 UTC

REVIEW: http://review.gluster.org/12375 (Revert "fuse: resolve complete path after a graph switch") posted (#8) for review on master by mohammed rafi  kc (rkavunga)

Comment 15 Vijay Bellur 2015-10-30 12:58:20 UTC

REVIEW: http://review.gluster.org/12449 (dht: update cached subvolume during readdirp cbk) posted (#3) for review on master by mohammed rafi  kc (rkavunga)

Comment 16 Vijay Bellur 2015-10-30 12:58:23 UTC

REVIEW: http://review.gluster.org/12376 (dht: heal directory path if the directory is not present) posted (#7) for review on master by mohammed rafi  kc (rkavunga)

Comment 17 Vijay Bellur 2015-11-02 06:28:43 UTC

REVIEW: http://review.gluster.org/12375 (Revert "fuse: resolve complete path after a graph switch") posted (#9) for review on master by mohammed rafi  kc (rkavunga)

Comment 18 Vijay Bellur 2015-11-02 06:28:46 UTC

REVIEW: http://review.gluster.org/12449 (dht: update cached subvolume during readdirp cbk) posted (#4) for review on master by mohammed rafi  kc (rkavunga)

Comment 19 Vijay Bellur 2015-11-02 06:28:49 UTC

REVIEW: http://review.gluster.org/12376 (dht: heal directory path if the directory is not present) posted (#8) for review on master by mohammed rafi  kc (rkavunga)

Comment 20 Vijay Bellur 2015-11-02 14:14:10 UTC

REVIEW: http://review.gluster.org/12375 (Revert "fuse: resolve complete path after a graph switch") posted (#10) for review on master by mohammed rafi  kc (rkavunga)

Comment 21 Vijay Bellur 2015-11-02 14:14:13 UTC

REVIEW: http://review.gluster.org/12449 (dht: update cached subvolume during readdirp cbk) posted (#5) for review on master by mohammed rafi  kc (rkavunga)

Comment 22 Vijay Bellur 2015-11-02 14:14:16 UTC

REVIEW: http://review.gluster.org/12376 (dht: heal directory path if the directory is not present) posted (#9) for review on master by mohammed rafi  kc (rkavunga)

Comment 23 Vijay Bellur 2015-11-03 06:24:02 UTC

REVIEW: http://review.gluster.org/12375 (Revert "fuse: resolve complete path after a graph switch") posted (#11) for review on master by mohammed rafi  kc (rkavunga)

Comment 24 Vijay Bellur 2015-11-03 06:24:05 UTC

REVIEW: http://review.gluster.org/12449 (dht: update cached subvolume during readdirp cbk) posted (#6) for review on master by mohammed rafi  kc (rkavunga)

Comment 25 Vijay Bellur 2015-11-03 06:24:07 UTC

REVIEW: http://review.gluster.org/12376 (dht: heal directory path if the directory is not present) posted (#10) for review on master by mohammed rafi  kc (rkavunga)

Comment 26 Vijay Bellur 2015-11-03 19:01:40 UTC

REVIEW: http://review.gluster.org/12375 (Revert "fuse: resolve complete path after a graph switch") posted (#12) for review on master by mohammed rafi  kc (rkavunga)

Comment 27 Mohammed Rafi KC 2015-11-10 10:03:05 UTC

I/O's were failed after attaching the tier is because, the fix-layout was not complete for some directories. So the directory structure was not proper on hot tier, and then trying to access such directories will result a failure.

Fix : 

after a nameless lookup if we get an incomplete layout, we will trigger a healing after getting full path from the server.

Comment 28 Vijay Bellur 2016-05-13 06:11:49 UTC

REVIEW: http://review.gluster.org/14319 (cluster/distribute: heal layout in discover codepath too) posted (#1) for review on master by Raghavendra G (rgowdapp)

Comment 29 Vijay Bellur 2016-05-16 05:48:41 UTC

REVIEW: http://review.gluster.org/14319 (cluster/distribute: heal layout in discover codepath too) posted (#2) for review on master by Raghavendra G (rgowdapp)

Comment 30 Niels de Vos 2016-06-16 13:40:47 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user