Bug 1441280 - [snapshot cifs]ls on .snaps directory is throwing input/output error over cifs mount
Summary: [snapshot cifs]ls on .snaps directory is throwing input/output error over cif...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: snapshot
Version: rhgs-3.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: RHGS 3.3.0
Assignee: Mohammed Rafi KC
QA Contact: Vivek Das
URL:
Whiteboard:
Depends On: 1447266 1463522
Blocks: 1417151
TreeView+ depends on / blocked
 
Reported: 2017-04-11 14:48 UTC by Vivek Das
Modified: 2017-09-21 04:37 UTC (History)
9 users (show)

Fixed In Version: glusterfs-3.8.4-26
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1447266 (view as bug list)
Environment:
Last Closed: 2017-09-21 04:37:54 UTC
Embargoed:


Attachments (Terms of Use)
gfapi-reproducer (1.27 KB, text/x-csrc)
2017-04-18 11:46 UTC, Anoop C S
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:2774 0 normal SHIPPED_LIVE glusterfs bug fix and enhancement update 2017-09-21 08:16:29 UTC

Description Vivek Das 2017-04-11 14:48:33 UTC
Description of problem:
ls on .snaps directory is throwing input/output error over cifs mount.
Listing works on fuse mount and windows mount.

Version-Release number of selected component (if applicable):
samba-client-4.6.2-0.el7rhgs.x86_64
glusterfs-3.8.4-22.el7rhgs.x86_64

How reproducible:
Always

Steps to Reproduce:
1.Enable uss & VSS plugins
2.Mount cifs on a samba ctdb gluster cluster
3.cd to the mount point
4. Do ll or ls over the cifs mount

Actual results:
ls: reading directory .snaps/: Input/output error

Expected results:
Should list out contents

Additional info:

Comment 6 Anoop C S 2017-04-18 11:45:40 UTC
I was able reproduce the bug easily. Following entries were seen in Samba logs:

[2017/04/12 15:42:25.377261,  5, pid=11231, effective(0, 0), real(0, 0)] ../source3/smbd/filename.c:644(unix_convert)
  unix_convert begin: name = .snaps/*, dirpath = .snaps, start = *
[2017/04/12 15:42:25.378556,  5, pid=11231, effective(0, 0), real(0, 0)] ../source3/smbd/filename.c:218(check_parent_exists)
  check_parent_exists: name = .snaps/*, dirpath = .snaps, start = *
[2017/04/12 15:42:25.378605, 10, pid=11231, effective(0, 0), real(0, 0)] ../source3/smbd/mangle_hash2.c:418(is_mangled)
  is_mangled * ?
[2017/04/12 15:42:25.378630, 10, pid=11231, effective(0, 0), real(0, 0)] ../source3/smbd/mangle_hash2.c:357(is_mangled_component)
  is_mangled_component * (len 1) ?
[2017/04/12 15:42:25.378653,  5, pid=11231, effective(0, 0), real(0, 0)] ../source3/smbd/filename.c:847(unix_convert)
  Wildcard *
[2017/04/12 15:42:25.378676, 10, pid=11231, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:1199(check_reduced_name)
  check_reduced_name: check_reduced_name [.snaps/*] [/]
[2017/04/12 15:42:25.380977, 10, pid=11231, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:1259(check_reduced_name)
  check_reduced_name realpath [.snaps/*] -> [/.snaps/*]
[2017/04/12 15:42:25.380999,  5, pid=11231, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:1370(check_reduced_name)
  check_reduced_name: .snaps/* reduced to /.snaps/*
[2017/04/12 15:42:25.381010,  5, pid=11231, effective(0, 0), real(0, 0)] ../source3/smbd/trans2.c:2741(call_trans2findfirst)
  dir=.snaps, mask = *
[2017/04/12 15:42:25.381026,  5, pid=11231, effective(0, 0), real(0, 0)] ../source3/smbd/dir.c:474(dptr_create)
  dptr_create dir=.snaps
[2017/04/12 15:42:25.381499, 10, pid=11231, effective(0, 0), real(0, 0)] ../source3/smbd/open.c:100(smbd_check_access_rights)
  smbd_check_access_rights: root override on .snaps. Granting 0x1
[2017/04/12 15:42:25.383028,  4, pid=11231, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:874(vfs_ChDir)
  vfs_ChDir to .snaps
[2017/04/12 15:42:25.383966,  1, pid=11231, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:921(vfs_GetWd)
  vfs_GetWd: couldn't stat "." error No such file or directory (NFS problem ?)
[2017/04/12 15:42:25.383985,  4, pid=11231, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:885(vfs_ChDir)
  vfs_ChDir got /.snaps
[2017/04/12 15:42:25.383998, 10, pid=11231, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:1199(check_reduced_name)
  check_reduced_name: check_reduced_name [.] [/]
[2017/04/12 15:42:25.385272,  3, pid=11231, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:1239(check_reduced_name)
  check_reduce_name: couldn't get realpath for . (NT_STATUS_OBJECT_PATH_NOT_FOUND)
[2017/04/12 15:42:25.385286,  5, pid=11231, effective(0, 0), real(0, 0)] ../source3/smbd/filename.c:1248(check_name)
  check_name: name . failed with NT_STATUS_OBJECT_PATH_NOT_FOUND

Judging from the above log it seems that stat and realpath vfs calls to glusterfs after changing directory to .snaps failed. Corresponding glusterfs client log entries:

[2017-04-12 10:12:25.379506] D [MSGID: 0] [client-rpc-fops.c:2936:client3_3_lookup_cbk] 0-stack-trace: stack-address: 0x55b8761a29b0, xcube-snapd-client returned -1 error: No such file or directory [No such file or directory]
[2017-04-12 10:12:25.379535] D [snapview-client.c:289:gf_svc_lookup_cbk] 0-xcube-snapview-client: Lookup failed on snapview graph with error No such file or directory
[2017-04-12 10:12:25.379554] D [MSGID: 0] [snapview-client.c:329:gf_svc_lookup_cbk] 0-stack-trace: stack-address: 0x55b8761a29b0, xcube-snapview-client returned -1 error: No such file or directory [No such file or directory]
[2017-04-12 10:12:25.379581] D [MSGID: 0] [io-stats.c:2213:io_stats_lookup_cbk] 0-stack-trace: stack-address: 0x55b8761a29b0, xcube returned -1 error: No such file or directory [No such file or directory]

I have attached a simple gfapi reproducer. We need to see why those calls are failing on .snaps directory.

Comment 7 Anoop C S 2017-04-18 11:46:27 UTC
Created attachment 1272289 [details]
gfapi-reproducer

Comment 8 Mohammed Rafi KC 2017-04-20 12:01:18 UTC
RCA:

Currently, snapview server is not handling the dentry name "." and ".." . So it fails for dentries pointing to entry_point ie, .snaps or the snapshot names.

Comment 17 Mohammed Rafi KC 2017-05-04 12:12:59 UTC
upstream master patch : https://review.gluster.org/17177

Comment 23 Vivek Das 2017-06-08 09:40:47 UTC
Followed the steps to reproduce and no Input/Output error seen accessing the .snaps directory on a cifs mount.

Version
-------
samba-client-4.6.3-2.el7rhgs.x86_64
glusterfs-3.8.4-27.el7rhgs.x86_64

As per comment 22 above not moving the bug to verified.

Comment 24 Vivek Das 2017-07-07 12:51:33 UTC
As the bug mentioned in Comment 22 is moved to verified. Revisited the bug, followed the steps to reproduce over version 
samba-client-4.6.3-3.el7rhgs.x86_64
glusterfs-3.8.4-32.el7rhgs.x86_64

And i am not seeing the Input/Output error over cifs mount.
Marking this as verified.

Comment 26 errata-xmlrpc 2017-09-21 04:37:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2774


Note You need to log in before you can comment on or make changes to this bug.