Description of problem: ======================= On Distributed volume when one sub-volume was down , directory creation failed with error 'Invalid argument' Version-Release number of selected component (if applicable): ============================================================= 3.6.0.18-1.el6rhs.x86_64 How reproducible: ================ not always Steps to Reproduce: =================== 1.create and mount distributed volume(NFS) 2. bring one or more sub-volume down by killing brick process. 3. add 2 more brick to volume 4. start I/O on NFs mount(dir and file cration) Actual results: ============== Directory creation failed with error 'Invalid argument' as below cannot create directory `etc1': Invalid argument Expected results: ================= If hashed sub-volume is down then on NFs mount it should fail with "I/O error". not with 'Invalid argument' Additional info: log snippet [2014-06-17 08:18:48.220398] W [dht-layout.c:178:dht_layout_search] 0-test1-dht: no subvolume for hash (value) = 2326726385 [2014-06-17 08:18:48.220428] W [nfs3.c:2722:nfs3svc_mkdir_cbk] 0-nfs: d2438e2b: /1/etc1 => -1 (Invalid argument) [2014-06-17 08:18:48.220450] W [nfs3-helpers.c:3470:nfs3_log_newfh_res] 0-nfs-nfsv3: XID: d2438e2b, MKDIR: NFS: 22(Invalid argument for operation), POSIX: 22(Invalid argument), FH: exportid 00000000-0000-0000-0000-000000000000, gfid 00000000-0000-0000-0000-000000000000 [2014-06-17 08:18:48.224624] W [client-rpc-fops.c:306:client3_3_mkdir_cbk] 0-test1-client-4: remote operation failed: Stale file handle. Path: /1/2 [2014-06-17 08:18:48.224788] W [client-rpc-fops.c:306:client3_3_mkdir_cbk] 0-test1-client-3: remote operation failed: Stale file handle. Path: /1/2 [2014-06-17 08:18:48.225446] W [client-rpc-fops.c:1021:client3_3_setxattr_cbk] 0-test1-client-4: remote operation failed: Stale file handle [2014-06-17 08:18:48.225522] W [client-rpc-fops.c:1021:client3_3_setxattr_cbk] 0-test1-client-3: remote operation failed: Stale file handle
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions
Fix has gone into downstream as part of rebase of 3.7 branch. Patch can be found at http://review.gluster.org/13351
This issue is not seen on glusterfs build: 3.7.9-10.el7rhgs.x86_64. Here are the steps that were performed, 1. Created a distributed replica volume and NFS mounted the volume to a client. 2. Brought down few sub-vols by killing brick process. 3. Added few more bricks to the volume. 4. Started I/O on NFS mount(dir and file creation). During dir/file creation if the hashed subvol is not found, error "Input/output" is seen which is the fix. mkdir: cannot create directory ‘d70’: Input/output error dd: failed to open ‘file_17’: Input/output error Also, this was tested with FUSE mount and appropriate error is seen. Hence, marking this bug as Verified.
Thanks Prasad. As per comment#9, I am Closing this BZ.