Bug 1121573 - nfs: remote I/O error , issue only for one particular directory
Summary: nfs: remote I/O error , issue only for one particular directory
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: distribute
Version: rhgs-3.0
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: ---
Assignee: Nithya Balachandran
QA Contact:
URL:
Whiteboard: triaged
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-07-21 09:19 UTC by Saurabh
Modified: 2017-03-25 14:23 UTC (History)
7 users (show)

Fixed In Version: 3.7.9-10
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-03-25 14:23:48 UTC


Attachments (Terms of Use)

Description Saurabh 2014-07-21 09:19:41 UTC
Description of problem:
I was trying to delete the data inside the mount-point, and eventually saw the error as "Remote I/O Error". Although this issue is seen till now only for one directory. 
The data in cosideratin i.e. that is not getting deleted is created by a script called as rpc-coverage.sh


Version-Release number of selected component (if applicable):
glusterfs-3.6.0.24-1.el6rhs.x86_64

How reproducible:
happens only for one directory.

Steps to Reproduce:
1. try to remove the data earlier created by the rpc.sh script 
2.
3.

Actual results:
[root@rhsauto034 nfs-test]# rm -rf run*
rm: cannot remove `run12026/coverage/dir/hardlink': Remote I/O error
rm: cannot remove `run12026/coverage/dir/file': Remote I/O error


nfs.log,
[2014-07-20 21:58:46.916123] W [client-rpc-fops.c:622:client3_3_unlink_cbk] 0-dist-rep-client-12: remote operation failed: No data available
[2014-07-20 21:58:46.916308] W [client-rpc-fops.c:622:client3_3_unlink_cbk] 0-dist-rep-client-13: remote operation failed: No data available
[2014-07-20 21:58:46.916400] W [nfs3.c:3352:nfs3svc_remove_cbk] 0-nfs: b8474c35: <gfid:fcfb2a6c-19b8-45f8-98d0-04e97b6e33ce>/hardlink => -1 (No data available)
[2014-07-20 21:58:46.916473] W [nfs3-helpers.c:3401:nfs3_log_common_res] 0-nfs-nfsv3: XID: b8474c35, REMOVE: NFS: 10006(Error occurred on the server or IO Error), POSIX: 61(No data available)
[2014-07-20 21:58:46.921176] W [client-rpc-fops.c:622:client3_3_unlink_cbk] 0-dist-rep-client-12: remote operation failed: No data available
[2014-07-20 21:58:46.921644] W [client-rpc-fops.c:622:client3_3_unlink_cbk] 0-dist-rep-client-13: remote operation failed: No data available
[2014-07-20 21:58:46.921725] W [nfs3.c:3352:nfs3svc_remove_cbk] 0-nfs: ba474c35: <gfid:fcfb2a6c-19b8-45f8-98d0-04e97b6e33ce>/file => -1 (No data available)
[2014-07-20 21:58:46.921760] W [nfs3-helpers.c:3401:nfs3_log_common_res] 0-nfs-nfsv3: XID: ba474c35, REMOVE: NFS: 10006(Error occurred on the server or IO Error), POSIX: 61(No data available)


Expected results:
rm -rf should succeed

Additional info:

Comment 3 Saurabh 2014-07-21 10:39:39 UTC
Data could not be deleted from the glusterfs mount as well,


[root@rhsauto002 run12026]# ls -liR
.:
total 0
9720369732820320925 drwxr-xr-x. 3 root root 128 Jul 10 07:35 coverage

./coverage:
total 0
11011306489765573582 drwxr-xr-x. 2 root root 74 Jul 21 05:17 dir

./coverage/dir:
total 1
12627660069723972778 -rw-r--r--. 2 root root 6 Jul  7 09:03 file
12627660069723972778 -rw-r--r--. 2 root root 6 Jul  7 09:03 hardlink
[root@rhsauto002 run12026]# rm -rf *
rm: cannot remove ‘coverage/dir/hardlink’: No data available
rm: cannot remove ‘coverage/dir/file’: No data available
[root@rhsauto002 run12026]#

Comment 8 Prasad Desala 2016-10-18 13:24:04 UTC
This issue is no more seen with glusterfs version: 3.8.4-2.el7rhgs.x86_64.

Ran rpc-coverage.sh script and was able to remove the earlier data created by the script without any errors/issues.
Hence, moving this BZ to Verified.


Note You need to log in before you can comment on or make changes to this bug.