Created attachment 614742 [details] glustershd.log file Description of problem: "W [client3_1-fops.c:529:client3_1_readlink_cbk] 0-vol0-client-1: remote operation failed: No such file or directory" messages are logged in bulk for around 4839 times during self heal operation. Version-Release number of selected component (if applicable): [root@darrel ~]# gluster --version glusterfs 3.3.0rhs built on Sep 10 2012 00:49:11 (glusterfs-3.3.0rhs-28.el6rhs.x86_64) How reproducible: Steps Carried: 1. Created 2 replica setup (10.70.34.115:/b1 and 10.70.34.119:/b2). 2. Bring down brick b2 ( I performed network stop on 10.70.34.119) 3. Created 20420 entries (files and directories) from the mount point 4. Brought back the brick b2 up (restarted 10.70.34.119) 5. After 10 min Checked the xattr which were all zero(s) and bricks were self healed. 6. But glustershd.log from 10.70.34.115 continuously reported these warning messages. This message is logged around 4839 times from line 67 to 4906 of the attached logs. Actual results: [2012-09-18 04:52:48.805426] W [client3_1-fops.c:529:client3_1_readlink_cbk] 0-vol0-client-1: remote operation failed: No such file or directory [2012-09-18 04:52:48.808220] W [client3_1-fops.c:529:client3_1_readlink_cbk] 0-vol0-client-1: remote operation failed: No such file or directory [2012-09-18 04:52:48.810325] W [client3_1-fops.c:529:client3_1_readlink_cbk] 0-vol0-client-1: remote operation failed: No such file or directory [2012-09-18 04:52:48.810508] W [client3_1-fops.c:529:client3_1_readlink_cbk] 0-vol0-client-1: remote operation failed: No such file or directory Log file is attached. Expected results: Additional info
Created attachment 615361 [details] New glustershd logs
Raising the severity of this bug. Tried the same scenario with distributed-replicate setup and observed this message for 28962 times continuously. Total number of lines reported in glustershd.log : 29088 Number of lines reported this message continuously: 28962 Attached the new log
Pranith, Wanted to check if a explicit readlink() is called from self-heal code? otherwise, this issue looks strange to me, because readlink() seems to be called on an non-existing file.
http://review.gluster.org/#change,4153 has been submitted for review.
ON_QA for rhs-2.1.0 branch.
Let us use this bug itslef for handling similar ENOENT errors in the protocol/client xlator.
The product version of Red Hat Storage on which this issue was reported has reached End Of Life (EOL) [1], hence this bug report is being closed. If the issue is still observed on a current version of Red Hat Storage, please file a new bug report on the current version. [1] https://rhn.redhat.com/errata/RHSA-2014-0821.html