Bug 1344595 - [disperse] mkdir after re balance give Input/Output Error
Summary: [disperse] mkdir after re balance give Input/Output Error
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: disperse
Version: 3.7.12
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Ashish Pandey
QA Contact:
URL:
Whiteboard:
Depends On: 1344277
Blocks: 1344278
TreeView+ depends on / blocked
 
Reported: 2016-06-10 07:00 UTC by Ashish Pandey
Modified: 2016-06-28 12:19 UTC (History)
2 users (show)

Fixed In Version: glusterfs-3.7.12
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1344277
Environment:
Last Closed: 2016-06-28 12:19:47 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Ashish Pandey 2016-06-10 07:00:09 UTC
+++ This bug was initially created as a clone of Bug #1344277 +++

Description of problem:

For EC volume and nfs mount, creation of directory is giving EIO error.   


Version-Release number of selected component (if applicable):
glusterfs 3.7.9 built on Jun  7 2016 12:00:32
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.


How reproducible:
100%

Steps to Reproduce:
1. Create a EC volume, Mount it through nfs on 2-3 differnet mount points.
2. Start IO. Create lots file in a for loop. 
3. When IO's are going on, do add-brick and re balance.
4 - stop IO's and try to do mkdir on same point - It gives EIO.
 
Actual results:

mkdir giving EIO

Expected results:

mkdir should be successful 

Additional info:

--- Additional comment from Ashish Pandey on 2016-06-09 06:37:27 EDT ---

RCA - 
In case of mkdir failure, dht expects error information so that it can act accordingly. 
In this case, after re balance, layout has been changed. mkdir with old layout  returns EIO. EC gets this error in cbk->xdata but does not pass it back to dht. In this case dht will not be able to take corrective action.

--- Additional comment from Vijay Bellur on 2016-06-09 08:42:17 EDT ---

REVIEW: http://review.gluster.org/14679 (cluster/ec: Pass xdata to dht in case of error) posted (#1) for review on master by Ashish Pandey (aspandey)

--- Additional comment from Vijay Bellur on 2016-06-09 11:44:50 EDT ---

REVIEW: http://review.gluster.org/14679 (cluster/ec: Pass xdata to dht in case of error) posted (#2) for review on master by Ashish Pandey (aspandey)

Comment 1 Vijay Bellur 2016-06-10 07:04:31 UTC
REVIEW: http://review.gluster.org/14689 (cluster/ec: Pass xdata to dht in case of error) posted (#1) for review on release-3.7 by Ashish Pandey (aspandey)

Comment 2 Vijay Bellur 2016-06-13 08:30:24 UTC
COMMIT: http://review.gluster.org/14689 committed in release-3.7 by Pranith Kumar Karampuri (pkarampu) 
------
commit fb7adf2e1f8ab582173fa38e5aaa584659ddd2a5
Author: Ashish Pandey <aspandey>
Date:   Thu Jun 9 16:19:37 2016 +0530

    cluster/ec: Pass xdata to dht in case of error
    
    Problem: In case of mkdir failure, dht expects
    error information so that it can act accordingly.
    Aftre adding bricks and re balance, layout gets
    changed. Fop "mkdir" with old layout returns EIO.
    EC gets this error in xdata but does not pass it
    back to dht. In this case dht will not be able to
    take corrective action.
    
    Solution: Return xdata back to dht
    
    master -
    http://review.gluster.org/#/c/14679/
    
    Change-Id: I24def8038e6880607689b7b046dc6428f564c6ab
    BUG: 1344595
    Signed-off-by: Ashish Pandey <aspandey>
    Reviewed-on: http://review.gluster.org/14689
    Reviewed-by: Xavier Hernandez <xhernandez>
    Smoke: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>

Comment 3 Kaushal 2016-06-28 12:19:47 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.12, please open a new bug report.

glusterfs-3.7.12 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://www.gluster.org/pipermail/gluster-devel/2016-June/049918.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user


Note You need to log in before you can comment on or make changes to this bug.