Bug 1228729

Summary: nfs-ganesha: rmdir logs "remote operation failed: Stale file handle" even though the operation is successful
Product: [Community] GlusterFS Reporter: Niels de Vos <ndevos>
Component: protocolAssignee: Niels de Vos <ndevos>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: high    
Version: 3.7.1CC: bugs, gluster-bugs, kkeithle
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.7.2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-06-20 09:50:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1228731    
Bug Blocks: 1224215, 1227206    

Description Niels de Vos 2015-06-05 14:35:54 UTC
+++ This bug was initially created as a clone of Bug #1224215 +++

Description of problem:
Even though the operation like rmdir is successfully deleting the directory in consideration the gfapi.log gets updated by logs such as 
"remote operation failed: Stale file handle"

This is a spurious log, doesn't seem to be giving any meaning to any one.

Version-Release number of selected component (if applicable):
glusterfs-3.7.0-2.el6rhs.x86_64
nfs-ganesha-2.2.0-0.el6.x86_64

How reproducible:
always

Steps to Reproduce:
1. create a volume of type 6x2, start it
2. bring up nfs-ganesha, after completing the pre-requisites
3. mount the volume with vers=4 or vers=3
4. create a dir and rmdir the same dir

Actual results:
step4 is successful
but gfapi.log reports
[2015-05-22 10:37:52.314210] W [client-rpc-fops.c:506:client3_3_stat_cbk] 0-vol2-client-1: remote operation failed: Stale file handle
[2015-05-22 10:37:52.314461] W [client-rpc-fops.c:506:client3_3_stat_cbk] 0-vol2-client-3: remote operation failed: Stale file handle
[2015-05-22 10:37:52.314606] W [client-rpc-fops.c:506:client3_3_stat_cbk] 0-vol2-client-11: remote operation failed: Stale file handle
[2015-05-22 10:37:52.314761] W [client-rpc-fops.c:506:client3_3_stat_cbk] 0-vol2-client-7: remote operation failed: Stale file handle
[2015-05-22 10:37:52.314860] W [client-rpc-fops.c:506:client3_3_stat_cbk] 0-vol2-client-9: remote operation failed: Stale file handle
[2015-05-22 10:37:52.314978] W [client-rpc-fops.c:506:client3_3_stat_cbk] 0-vol2-client-5: remote operation failed: Stale file handle


^C


Expected results:
logs should be meaning ful, it should match the operation that it is doing, present logs give a wrong impression of the system

Additional info:


--- Additional comment from Niels de Vos on 2015-06-05 14:27:31 CEST ---

Need to investigate if logging at Warning is suitable, maybe this should be marked as Debug instead.

Ganesha does remove first, and then stats the file-handle to update the attributes (like hard-link count). The stale file handle is expected in this case.

Comment 1 Anand Avati 2015-06-12 11:52:19 UTC
REVIEW: http://review.gluster.org/11169 (logging: log "Stale filehandle" on the client as Debug) posted (#2) for review on release-3.7 by Kaleb KEITHLEY (kkeithle)

Comment 2 Anand Avati 2015-06-14 06:36:25 UTC
COMMIT: http://review.gluster.org/11169 committed in release-3.7 by Vijay Bellur (vbellur) 
------
commit 5ab2ec4ad0129fb58d5287fcb7731be1b0dcf56e
Author: Niels de Vos <ndevos>
Date:   Thu Jun 11 01:07:56 2015 +0200

    logging: log "Stale filehandle" on the client as Debug
    
    There are valid use-cases where a "Stale filehandle" is expected. One of
    these is deleting a file through NFS-Ganesha. When the "filename" is
    deleted, Ganesha will stat the file-handle to update its attributes. The
    file-handle would still be valid in the case where there are hardlinks.
    
    There is no need to log "Stale filehandle" as Warning. It is perfectly
    fine to have this logged as Debug.
    
    Cherry picked from commit ec7d0c3de11cbee1470308dcc10ec9f02e7fdfff:
    > Change-Id: Ib9006d95a5d1f7dd394bf5d5994ad70740ad4ab2
    > BUG: 1228731
    > Signed-off-by: Niels de Vos <ndevos>
    > Reviewed-on: http://review.gluster.org/11107
    > Reviewed-by: Kaleb KEITHLEY <kkeithle>
    > Tested-by: Gluster Build System <jenkins.com>
    > Reviewed-by: soumya k <skoduri>
    > Reviewed-by: jiffin tony Thottan <jthottan>
    
    Change-Id: Ib9006d95a5d1f7dd394bf5d5994ad70740ad4ab2
    BUG: 1228729
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/11169
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 3 Niels de Vos 2015-06-20 09:50:09 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.2, please reopen this bug report.

glusterfs-3.7.2 has been announced on the Gluster Packaging mailinglist [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://www.gluster.org/pipermail/packaging/2015-June/000006.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user