Bug 1202758

Summary: Disperse volume: brick logs are getting filled with "anonymous fd creation failed" messages
Product: [Community] GlusterFS Reporter: Bhaskarakiran <byarlaga>
Component: disperseAssignee: Ashish Pandey <aspandey>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: medium    
Version: mainlineCC: aspandey, bugs, byarlaga, gluster-bugs, mzywusko, pkarampu, rkavunga
Target Milestone: ---Keywords: Reopened, Triaged
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: glusterfs-3.8rc2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1224136 1228100 (view as bug list) Environment:
Last Closed: 2016-06-16 12:43:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1186580, 1224136, 1228100    
Attachments:
Description Flags
sosreport of client none

Description Bhaskarakiran 2015-03-17 12:07:47 UTC 
Created attachment 1002781 [details]
sosreport of client

Description of problem:
======================= 

Brick logs are getting filled with below messages and the filesystem is full within no time.

[2015-03-17 11:56:04.171990] W [server-resolve.c:425:resolve_anonfd_simple] 0-server: inode for the gfid (ea84b493-b049-487a-825b-cb4bece296d7) is not found. anonymous fd creation failed
[2015-03-17 11:56:04.172063] W [server-resolve.c:425:resolve_anonfd_simple] 0-server: inode for the gfid (ea84b493-b049-487a-825b-cb4bece296d7) is not found. anonymous fd creation failed
[2015-03-17 11:56:04.172489] W [server-resolve.c:425:resolve_anonfd_simple] 0-server: inode for the gfid (b8bb35ff-1641-49c8-8ce2-b07f35fd4ef7) is not found. anonymous fd creation failed
[2015-03-17 11:56:04.172567] W [server-resolve.c:425:resolve_anonfd_simple] 0-server: inode for the gfid (b8bb35ff-1641-49c8-8ce2-b07f35fd4ef7) is not found. anonymous fd creation failed

Version-Release number of selected component (if applicable):
=============================================================
[root@vertigo tmp]# gluster --version
glusterfs 3.7dev built on Mar 12 2015 01:40:59
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@vertigo tmp]# 

How reproducible:
=================
Tried once


Steps to Reproduce:
1. Create a 1x(8+4) disperse volume and nfs/fuse mount on the client
2. Untar linux kernel images in parallel of about 20 instances
3. Bring down 4 bricks for some time and bring them back up with "gluster volume start force"
4. Trigger full heal from server
5. On the client do a 'du -sh  ." on the untarred directory. (Not sure if this has to do anything with log getting filled up)

Actual results:


Expected results:


Additional info:
================
sosreports of server and clients will be attached.
Comment 1 Ashish Pandey 2015-05-12 11:58:04 UTC 
I tried to reproduce the issue with latest code but could not see such  ("anonymous fd creation failed") logs and failures.
During these steps logs were getting created but I could not find anything which may cause logs filled.
Please check with latest code and see if you still see the above logs or any other problematic logs.
Comment 2 Anand Avati 2015-05-19 11:37:53 UTC 
REVIEW: http://review.gluster.org/10829 (Changing log level from Warning to Debug) posted (#1) for review on master by Ashish Pandey (aspandey)
Comment 3 Anand Avati 2015-05-29 02:36:13 UTC 
COMMIT: http://review.gluster.org/10829 committed in master by Pranith Kumar Karampuri (pkarampu) 
------
commit e6934debda3a67b5666f1aa30b33c5267de49d84
Author: Ashish Pandey <aspandey>
Date:   Tue May 19 17:04:55 2015 +0530

    Changing log level from Warning to Debug
    
    Change-Id: Ic4306ebb5a58c2ca310e973bb562be1ca3e4885f
    BUG: 1202758
    Signed-off-by: Ashish Pandey <aspandey>
    Reviewed-on: http://review.gluster.org/10829
    Tested-by: NetBSD Build System
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>
Comment 4 Nagaprasad Sathyanarayana 2015-10-25 14:49:46 UTC 
Fix for this BZ is already present in a GlusterFS release. You can find clone of this BZ, fixed in a GlusterFS release and closed. Hence closing this mainline BZ as well.
Comment 5 Niels de Vos 2016-06-16 12:43:55 UTC 
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user
Comment 6 Red Hat Bugzilla 2023-09-14 02:56:21 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days