Bug 1191033 - [USS]: While accessing 256 snapshots under .snaps from fuse and NFS , some snapshots intermittently gave ENOENT from fuse and 'Invalid Argument' error from NFS
Summary: [USS]: While accessing 256 snapshots under .snaps from fuse and NFS , some sn...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: snapshot
Version: rhgs-3.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Bhavana
QA Contact: storage-qa-internal@redhat.com
URL:
Whiteboard: USS
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-02-10 10:17 UTC by senaik
Modified: 2016-09-17 12:57 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
Cause: When a large number of snapshots are activated and are accessed via USS, the inode limit for the mounted filesystem may get exhausted, causing deletion of inode entries from the inode table. Accessing .snaps when this happens may result in ENOENT errors in fuse, and "Invalid Argument" errors in NFS. Workaround: Clear the kernel VFS cache by executing the following command: # echo 3 >/proc/sys/vm/drop_caches
Clone Of:
Environment:
Last Closed: 2015-06-12 08:12:34 UTC
Embargoed:


Attachments (Terms of Use)

Description senaik 2015-02-10 10:17:49 UTC
Description of problem:
=======================
While accessing 256 snapshots under .snaps from fuse and NFS , some snapshots intermittently gave ENOENT from fuse and 'Invalid Argument' error from NFS


Version-Release number of selected component (if applicable):
=============================================================
glusterfs-3.6.0.42.1-1.el6rhs.x86_64

How reproducible:
================
Often


Steps to Reproduce:
===================
1.Create a 6x2 dist rep volume and start it 

2.Fuse and NFS mount the volume and create heavy IO 

3.Create 256 snapshots in loop and activate them from another node

4.From the client access .snaps and cd to every snap in .snaps directory from the / of the volume and from random sub-directories  

5.Some snaps intermittently gave 'ENOENT' from fuse and 'Invalid Argument' from NFS mount .

After clearing the cache , snapshots were successfully accessed under .snaps 

Actual results:
==============
Some snapshots intermittently gave ENOENT from fuse and 'Invalid Argument' error from NFS

Expected results:
================
There should be no errors seen while trying to access snapshots under .snaps 


Additional info:

Comment 2 Avra Sengupta 2015-02-10 11:37:02 UTC
Added doc text:
Cause: When a large number of snapshots are activated and are accessed via USS, the inode limit for the mounted filesystem may get exhausted, causing deletion of inode entries from the inode table. Accessing .snaps when this happens may result in ENOENT errors in fuse, and "Invalid Argument" errors in NFS.

Workaround: Clear the kernel VFS cache by executing the following command:
# echo 3 >/proc/sys/vm/drop_caches

Comment 4 Bhavana 2015-02-11 06:50:00 UTC
The information is added as a Known Issue in the "Release Notes":

http://docbuilder.usersys.redhat.com/22629/#Red_Hat_Storage

Do review the same and share your comments if any.

Comment 5 Avra Sengupta 2015-02-11 07:06:23 UTC
Looks great. Thanks Bhavana.

Comment 6 Bhavana 2015-06-12 08:12:34 UTC
This is fixed in the doc as verified in comment 5.

Hence closing this bug.


Note You need to log in before you can comment on or make changes to this bug.