Bug 1202758 - Disperse volume: brick logs are getting filled with "anonymous fd creation failed" messages [NEEDINFO]
Summary: Disperse volume: brick logs are getting filled with "anonymous fd creation fa...
Alias: None
Product: GlusterFS
Classification: Community
Component: disperse
Version: mainline
Hardware: All
OS: All
Target Milestone: ---
Assignee: Ashish Pandey
QA Contact:
Depends On:
Blocks: qe_tracker_everglades 1224136 1228100
TreeView+ depends on / blocked
Reported: 2015-03-17 12:07 UTC by Bhaskarakiran
Modified: 2016-11-23 23:13 UTC (History)
7 users (show)

Fixed In Version: glusterfs-3.8rc2
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1224136 1228100 (view as bug list)
Last Closed: 2016-06-16 12:43:55 UTC
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
aspandey: needinfo? (byarlaga)

Attachments (Terms of Use)
sosreport of client (8.13 MB, application/x-xz)
2015-03-17 12:07 UTC, Bhaskarakiran
no flags Details

Description Bhaskarakiran 2015-03-17 12:07:47 UTC
Created attachment 1002781 [details]
sosreport of client

Description of problem:

Brick logs are getting filled with below messages and the filesystem is full within no time.

[2015-03-17 11:56:04.171990] W [server-resolve.c:425:resolve_anonfd_simple] 0-server: inode for the gfid (ea84b493-b049-487a-825b-cb4bece296d7) is not found. anonymous fd creation failed
[2015-03-17 11:56:04.172063] W [server-resolve.c:425:resolve_anonfd_simple] 0-server: inode for the gfid (ea84b493-b049-487a-825b-cb4bece296d7) is not found. anonymous fd creation failed
[2015-03-17 11:56:04.172489] W [server-resolve.c:425:resolve_anonfd_simple] 0-server: inode for the gfid (b8bb35ff-1641-49c8-8ce2-b07f35fd4ef7) is not found. anonymous fd creation failed
[2015-03-17 11:56:04.172567] W [server-resolve.c:425:resolve_anonfd_simple] 0-server: inode for the gfid (b8bb35ff-1641-49c8-8ce2-b07f35fd4ef7) is not found. anonymous fd creation failed

Version-Release number of selected component (if applicable):
[root@vertigo tmp]# gluster --version
glusterfs 3.7dev built on Mar 12 2015 01:40:59
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@vertigo tmp]# 

How reproducible:
Tried once

Steps to Reproduce:
1. Create a 1x(8+4) disperse volume and nfs/fuse mount on the client
2. Untar linux kernel images in parallel of about 20 instances
3. Bring down 4 bricks for some time and bring them back up with "gluster volume start force"
4. Trigger full heal from server
5. On the client do a 'du -sh  ." on the untarred directory. (Not sure if this has to do anything with log getting filled up)

Actual results:

Expected results:

Additional info:
sosreports of server and clients will be attached.

Comment 1 Ashish Pandey 2015-05-12 11:58:04 UTC
I tried to reproduce the issue with latest code but could not see such  ("anonymous fd creation failed") logs and failures.
During these steps logs were getting created but I could not find anything which may cause logs filled.
Please check with latest code and see if you still see the above logs or any other problematic logs.

Comment 2 Anand Avati 2015-05-19 11:37:53 UTC
REVIEW: http://review.gluster.org/10829 (Changing log level from Warning to Debug) posted (#1) for review on master by Ashish Pandey (aspandey@redhat.com)

Comment 3 Anand Avati 2015-05-29 02:36:13 UTC
COMMIT: http://review.gluster.org/10829 committed in master by Pranith Kumar Karampuri (pkarampu@redhat.com) 
commit e6934debda3a67b5666f1aa30b33c5267de49d84
Author: Ashish Pandey <aspandey@redhat.com>
Date:   Tue May 19 17:04:55 2015 +0530

    Changing log level from Warning to Debug
    Change-Id: Ic4306ebb5a58c2ca310e973bb562be1ca3e4885f
    BUG: 1202758
    Signed-off-by: Ashish Pandey <aspandey@redhat.com>
    Reviewed-on: http://review.gluster.org/10829
    Tested-by: NetBSD Build System
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

Comment 4 Nagaprasad Sathyanarayana 2015-10-25 14:49:46 UTC
Fix for this BZ is already present in a GlusterFS release. You can find clone of this BZ, fixed in a GlusterFS release and closed. Hence closing this mainline BZ as well.

Comment 5 Niels de Vos 2016-06-16 12:43:55 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Note You need to log in before you can comment on or make changes to this bug.