Bug 1327070

Summary: [snapshot]: stalefile handle and invalid arguments are reported if uss is enabled and no snapshot present
Product: Red Hat Gluster Storage Reporter: Rahul Hinduja <rhinduja>
Component: snapshotAssignee: Bug Updates Notification Mailing List <rhs-bugs>
Status: CLOSED WONTFIX QA Contact: storage-qa-internal <storage-qa-internal>
Severity: medium Docs Contact:
Priority: unspecified    
Version: rhgs-3.1CC: rhs-bugs
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-16 16:03:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Rahul Hinduja 2016-04-14 08:38:48 UTC 
Description of problem:
=======================

Found invalid arguments errors on geo-replication client logs and stale file handle warning in snapd lags. Uss is enabled on the volumes but no snapshot operations are performed. No lookup on .snaps is carried. 

These errors in geo-rep logs are ambiguous as no snapshot operation is performed by user. 

Geo-Replication client logs:
============================

[2016-04-13 18:24:58.299211] E [snapview-client.c:92:svc_inode_ctx_get] (-->/usr/lib64/glusterfs/3.7.9/xlator/performance/md-cache.so(mdc_lookup_cbk+0x12a) [0x7fa4031b4a3a] -->/usr/lib64/glusterfs/3.7.9/xlator/features/snapview-client.so(+0xa2ab) [0x7fa402fa62ab] -->/usr/lib64/glusterfs/3.7.9/xlator/features/snapview-client.so(svc_inode_ctx_get+0xa8) [0x7fa402f9ecd8] ) 0-vol0-snapview-client: invalid argument: inode [Invalid argument]
[2016-04-13 18:24:58.302570] E [snapview-client.c:92:svc_inode_ctx_get] (-->/usr/lib64/glusterfs/3.7.9/xlator/performance/md-cache.so(mdc_lookup_cbk+0x12a) [0x7fa4031b4a3a] -->/usr/lib64/glusterfs/3.7.9/xlator/features/snapview-client.so(+0xa2ab) [0x7fa402fa62ab] -->/usr/lib64/glusterfs/3.7.9/xlator/features/snapview-client.so(svc_inode_ctx_get+0xa8) [0x7fa402f9ecd8] ) 0-vol0-snapview-client: invalid argument: inode [Invalid argument]
[2016-04-13 18:24:58.310297] E [snapview-client.c:92:svc_inode_ctx_get] (-->/usr/lib64/glusterfs/3.7.9/xlator/performance/md-cache.so(mdc_lookup_cbk+0x12a) [0x7fa4031b4a3a] -->/usr/lib64/glusterfs/3.7.9/xlator/features/snapview-client.so(+0xa2ab) [0x7fa402fa62ab] -->/usr/lib64/glusterfs/3.7.9/xlator/features/snapview-client.so(svc_inode_ctx_get+0xa8) [0x7fa402f9ecd8] ) 0-vol0-snapview-client: invalid argument: inode [Invalid argument]


Snapd logs:
===========

[2016-04-13 18:24:58.333051] W [MSGID: 115005] [server-resolve.c:126:resolve_gfid_cbk] 0-vol0-server: 24455707-e959-470c-9ce8-169993fa89cd: failed to resolve (Stale file handle) [Stale file handle]
The message "W [MSGID: 115005] [server-resolve.c:126:resolve_gfid_cbk] 0-vol0-server: 24455707-e959-470c-9ce8-169993fa89cd: failed to resolve (Stale file handle) [Stale file handle]" repeated 5 times between [2016-04-13 18:24:58.333051] and [2016-04-13 18:24:58.340688]
[2016-04-13 18:24:58.345005] W [MSGID: 115005] [server-resolve.c:126:resolve_gfid_cbk] 0-vol0-server: 13bb5cca-f82b-4209-b404-0d1314859fb6: failed to resolve (Stale file handle) [Stale file handle]
The message "W [MSGID: 115005] [server-resolve.c:126:resolve_gfid_cbk] 0-vol0-server: cc135b54-ad15-4a56-8037-923956a06d3f: failed to resolve (Stale file handle) [Stale file handle]" repeated 4 times between [2016-04-13 18:24:57.841812] and [2016-04-13 18:24:58.356258]
[2016-04-13 18:24:58.359004] W [MSGID: 115005] [server-resolve.c:126:resolve_gfid_cbk] 0-vol0-server: eadff54f-6208-459b-877b-36eaab39deda: failed to resolve (Stale file handle) [Stale file handle]
[2016-04-13 18:24:58.360232] W [MSGID: 115005] [server-resolve.c:126:resolve_gfid_cbk] 0-vol0-server: cc135b54-ad15-4a56-8037-923956a06d3f: failed to resolve (Stale file handle) [Stale file handle]
[2016-04-13 18:24:58.361814] W [MSGID: 115005] [server-resolve.c:126:resolve_gfid_cbk] 0-vol0-server: eadff54f-6208-459b-877b-36eaab39deda: failed to resolve (Stale file handle) [Stale file handle]


These errors are reported in huge numbers.

Version-Release number of selected component (if applicable):
=============================================================

glusterfs-3.7.9-1.el7rhgs.x86_64


How reproducible:
=================
2/2

Steps Carried:
1. Ran geo-replication automated cases once with root and once with non-root. 
2. fops carried in order are create,chmod,chown,chgrp,symlink,hardlink,rename,truncate,remove
Comment 3 Rahul Hinduja 2016-06-02 17:35:48 UTC 
For reference: 

These errors are reported multiple times at slave log for every entry synced to slave when bricks on Master volume were offline.

[root@dhcp37-52 geo-replication-slaves]# grep -ri "snapview-client.c:92:svc_inode_ctx_get" e4348759-bb69-415f-b8f7-94fbcb3a2533\:gluster%3A%2F%2F127.0.0.1%3ASlave.gluster.log  | wc -l 
1243
[root@dhcp37-52 geo-replication-slaves]#