Bug 1231166 - Disperse volume : fuse mount hung on renames on a distributed disperse volume
Summary: Disperse volume : fuse mount hung on renames on a distributed disperse volume
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: disperse
Version: rhgs-3.1
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: ---
: RHGS 3.1.0
Assignee: Pranith Kumar K
QA Contact: Bhaskarakiran
URL:
Whiteboard:
Depends On:
Blocks: 1202842 1232420 1232983
TreeView+ depends on / blocked
 
Reported: 2015-06-12 10:15 UTC by Bhaskarakiran
Modified: 2016-11-23 23:12 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.7.1-4
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1232420 (view as bug list)
Environment:
Last Closed: 2015-07-29 05:02:43 UTC
Target Upstream Version:


Attachments (Terms of Use)
client fuse log (14.10 MB, application/zip)
2015-06-16 10:42 UTC, Bhaskarakiran
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:1495 0 normal SHIPPED_LIVE Important: Red Hat Gluster Storage 3.1 update 2015-07-29 08:26:26 UTC

Description Bhaskarakiran 2015-06-12 10:15:24 UTC
Description of problem:
=======================

fuse mount hung while doing renames from / to a directory inside. created around 1000 files and moved them to a directory with 'mv' command.

volume info:
============
[root@interstellar ~]# gluster v info vol1
 
Volume Name: vol1
Type: Distributed-Disperse
Volume ID: 8136d188-7ab5-4cb5-97c2-5138431c3dec
Status: Started
Number of Bricks: 2 x (8 + 4) = 24
Transport-type: tcp
Bricks:
Brick1: transformers:/rhs/brick1/b1
Brick2: interstellar:/rhs/brick1/b2
Brick3: ninja:/rhs/brick1/b3
Brick4: transformers:/rhs/brick2/b4
Brick5: interstellar:/rhs/brick2/b5
Brick6: ninja:/rhs/brick2/b6
Brick7: transformers:/rhs/brick3/b7
Brick8: interstellar:/rhs/brick3/b8
Brick9: ninja:/rhs/brick3/b9
Brick10: transformers:/rhs/brick4/b10
Brick11: interstellar:/rhs/brick4/b11
Brick12: ninja:/rhs/brick4/b12
Brick13: transformers:/rhs/brick1/b3
Brick14: interstellar:/rhs/brick1/b14
Brick15: ninja:/rhs/brick1/b15
Brick16: transformers:/rhs/brick2/b16
Brick17: interstellar:/rhs/brick2/b17
Brick18: ninja:/rhs/brick2/b18
Brick19: transformers:/rhs/brick3/b19
Brick20: interstellar:/rhs/brick3/b20
Brick21: ninja:/rhs/brick3/b21
Brick22: transformers:/rhs/brick4/b22
Brick23: interstellar:/rhs/brick4/b23
Brick24: ninja:/rhs/brick4/b24
Options Reconfigured:
features.barrier: disable
cluster.disperse-self-heal-daemon: enable
features.inode-quota: on
features.quota: on
features.uss: on
server.event-threads: 2
client.event-threads: 2
performance.readdir-ahead: on
features.quota-deem-statfs: on
[root@interstellar ~]# 


log snippet:
============
[2015-06-12 09:09:03.509960] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-13: remote operation failed: Invalid argument
[2015-06-12 09:09:03.510033] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-16: remote operation failed: Invalid argument
[2015-06-12 09:09:03.510111] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-12: remote operation failed: Invalid argument
[2015-06-12 09:09:03.510237] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-19: remote operation failed: Invalid argument
[2015-06-12 09:09:03.510312] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-15: remote operation failed: Invalid argument
[2015-06-12 09:09:03.510361] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-14: remote operation failed: Invalid argument
[2015-06-12 09:09:03.510500] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-22: remote operation failed: Invalid argument
[2015-06-12 09:09:03.510572] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-18: remote operation failed: Invalid argument
[2015-06-12 09:09:03.510673] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-23: remote operation failed: Invalid argument
[2015-06-12 09:09:03.510728] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-17: remote operation failed: Invalid argument
[2015-06-12 09:09:03.510809] W [ec-combine.c:860:ec_combine_check] 0-vol1-disperse-0: Mismatching xdata in answers of 'LOOKUP'
[2015-06-12 09:09:03.510886] W [ec-combine.c:860:ec_combine_check] 0-vol1-disperse-0: Mismatching xdata in answers of 'LOOKUP'
[2015-06-12 09:09:03.510941] W [ec-combine.c:860:ec_combine_check] 0-vol1-disperse-0: Mismatching xdata in answers of 'LOOKUP'
[2015-06-12 09:09:03.511040] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-21: remote operation failed: Invalid argument
[2015-06-12 09:09:03.511255] E [client-rpc-fops.c:1533:client3_3_inodelk_cbk] 0-vol1-client-20: remote operation failed: Invalid argument
[2015-06-12 09:09:03.511287] W [ec-common.c:1397:ec_unlocked] 0-vol1-disperse-1: entry/inode unlocking failed (RENAME)
[2015-06-12 09:09:03.511311] W [ec-combine.c:860:ec_combine_check] 0-vol1-disperse-1: Mismatching xdata in answers of 'LOOKUP'
[2015-06-12 09:09:03.512823] W [ec-combine.c:860:ec_combine_check] 0-vol1-disperse-0: Mismatching xdata in answers of 'LOOKUP'

Version-Release number of selected component (if applicable):
==============================================================
[root@transformers ~]# gluster --version
glusterfs 3.7.1 built on Jun 12 2015 00:21:18
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@transformers ~]# 

How reproducible:
=================
100%

Steps to Reproduce:
As in description

Actual results:


Expected results:


Additional info:

Attaching the clinet fuse log.

Comment 2 Bhaskarakiran 2015-06-16 10:42:29 UTC
Created attachment 1039366 [details]
client fuse log

Comment 6 Bhaskarakiran 2015-07-07 10:08:20 UTC
verified this on 3.7.1-7 build and didn't see the issue. Marking this as fixed.

Comment 7 errata-xmlrpc 2015-07-29 05:02:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1495.html


Note You need to log in before you can comment on or make changes to this bug.