Bug 1030981

Summary: DHT : log has warnings 'remote operation failed: No such file or directory' but no details about which operation failed and on which file or Directory
Product: Red Hat Gluster Storage Reporter: Rachana Patel <racpatel>
Component: distributeAssignee: Susant Kumar Palai <spalai>
Status: CLOSED NOTABUG QA Contact: storage-qa-internal <storage-qa-internal>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 2.1CC: mzywusko, rwheeler, spalai, vbellur
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-18 09:36:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Rachana Patel 2013-11-15 12:47:08 UTC 
Description of problem:
========================
mount log has warnings
'remote operation failed: No such file or directory' but no details about operation or file directory.

e.g.
[2013-11-11 10:37:50.560423] W [fuse-bridge.c:1133:fuse_attr_cbk] 0-glusterfs-fuse: 42111: STAT() /mvtestdatau2 => -1 (No such file or 
directory)
[2013-11-11 10:37:50.561210] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-3: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.561274] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-0: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.561308] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-1: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.561339] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-2: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.565997] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-2: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.566080] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-3: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.566138] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-0: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.566181] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-1: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.571166] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-2: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.571232] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-1: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.571271] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-0: remote operation failed: No such file or dire
ctory
[2013-11-11 10:37:50.571304] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-3: remote operation failed: No such file or directory
[2013-11-11 10:39:12.161243] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-3: remote operation failed: No such file or directory
[2013-11-11 10:39:12.161363] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-1: remote operation failed: No such file or directory
[2013-11-11 10:39:12.161406] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-2: remote operation failed: No such file or directory
[2013-11-11 10:39:12.161440] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-0: remote operation failed: No such file or directory
[2013-11-11 10:40:33.585573] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-3: remote operation failed: No such file or directory
[2013-11-11 10:40:33.585690] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-0: remote operation failed: No such file or directory
[2013-11-11 10:40:33.585731] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-2: remote operation failed: No such file or directory
[2013-11-11 10:40:33.585769] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-1: remote operation failed: No such file or directory
[2013-11-11 10:40:33.589715] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-0: remote operation failed: No such file or directory
[2013-11-11 10:40:33.589785] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-1: remote operation failed: No such file or directory
[2013-11-11 10:40:33.589824] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-3: remote operation failed: No such file or directory
[2013-11-11 10:40:33.589941] W [client-rpc-fops.c:519:client3_3_stat_cbk] 0-dht-client-2: remote operation failed: No such file or directory


not able to understand which operation was failed and on which entity

Version-Release number of selected component (if applicable):
============================================================= 
3.4.0.44rhs-1.el6rhs.x86_64



How reproducible:
always

Steps to Reproduce:
1. while going through mount log, found these warning which doesn't talk about file name or operation.
Not sure which TC was executed for this.

Actual results:
 no details about which operation failed and on which file or Directory

Expected results:
log should have enough details.

Additional info:
Comment 2 Rachana Patel 2013-11-15 13:05:28 UTC 
these warnings are also present in nfs log :-

log snippet

[2013-11-14 03:35:45.165952] W [client-rpc-fops.c:1037:client3_3_setxattr_cbk] 0-dht-client-2: remote operation failed: No such file or
 directory
[2013-11-14 03:35:45.165998] W [client-rpc-fops.c:1037:client3_3_setxattr_cbk] 0-dht-client-3: remote operation failed: No such file or
 directory
[2013-11-14 03:35:45.166040] W [client-rpc-fops.c:1037:client3_3_setxattr_cbk] 0-dht-client-1: remote operation failed: No such file or
 directory
[2013-11-14 03:35:45.166479] W [client-rpc-fops.c:1037:client3_3_setxattr_cbk] 0-dht-client-0: remote operation failed: No such file or
 directory
Comment 4 Amar Tumballi 2013-12-02 10:04:42 UTC 
it is visible for me in the log that the operation which failed is 'stat()' (in description and 'setxattr()' in comment #2.

Doesn't seem to be any issue IMO.