Bug 1045271

Summary:	nfs: DRC failed to detect duplicates, nfsd is killed
Product:	Red Hat Gluster Storage	Reporter:	Saurabh <saujain>
Component:	gluster-nfs	Assignee:	Niels de Vos <ndevos>
Status:	CLOSED DEFERRED	QA Contact:	storage-qa-internal <storage-qa-internal>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	2.1	CC:	mzywusko, spradhan, vagarwal, vbellur
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-11-25 10:24:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Bug Depends On:
Bug Blocks:	1046576

Description Saurabh 2013-12-20 04:01:23 UTC

Description of problem:
NFS server killed and nfs.log reports "DRC failed to detect duplicates"
Test inluded creating data of random size files, inside several directories. Directories are having quota limits set.

Version-Release number of selected component (if applicable):
glusterfs-3.4.0.49rhs-1.el6rhs.x86_64

How reproducible:
happened on this build.

Steps to Reproduce:
1. create a 6x2 volume, start it
2. mount over nfs, create 10 directories
3. set limit of 1TB on root of the volume and 10GB on each directory
4. start creating data inside each directory.

Actual results:
nfsd is killed

nfs.log reports,
[2013-12-19 21:40:01.826319] E [rpc-drc.c:499:rpcsvc_add_op_to_cache] 0-rpc-service: DRC failed to detect duplicates
[2013-12-19 21:40:01.839258] E [rpc-drc.c:499:rpcsvc_add_op_to_cache] 0-rpc-service: DRC failed to detect duplicates
[2013-12-19 21:40:01.848096] E [rpc-drc.c:499:rpcsvc_add_op_to_cache] 0-rpc-service: DRC failed to detect duplicates
[2013-12-19 21:40:01.880223] E [rpc-drc.c:499:rpcsvc_add_op_to_cache] 0-rpc-service: DRC failed to detect duplicates
[2013-12-19 21:40:01.890184] E [rpc-drc.c:499:rpcsvc_add_op_to_cache] 0-rpc-service: DRC failed to detect duplicates


Expected results:
DRC should detect duplicate requests, 
test mentioned above is just about creating data, this should pass.

Additional info:

Comment 3 ssamanta 2013-12-23 13:59:04 UTC

The log should also include the diagnostic message why the DRC failed along with the cache-hit/miss count etc for better usability purpose.

Comment 4 Saurabh 2013-12-26 12:27:11 UTC

This issue is seen even without quota enabled on volume.

Comment 5 Saurabh 2013-12-26 12:29:36 UTC

(In reply to Saurabh from comment #4)
> This issue is seen even without quota enabled on volume.

Here, the "DRC failed to detect duplicates" logs are seen in numbers though nfsd is still running. if it gets killed will update the BZ, as the script to created data is still running.

Comment 7 Saurabh 2013-12-27 10:24:30 UTC

(In reply to Saurabh from comment #5)
> (In reply to Saurabh from comment #4)
> > This issue is seen even without quota enabled on volume.
> 
> Here, the "DRC failed to detect duplicates" logs are seen in numbers though
> nfsd is still running. if it gets killed will update the BZ, as the script
> to created data is still running.

I kept running the script for creating data, but the nfsd is not killed. This time the quota was not enabled.

Comment 10 Niels de Vos 2015-11-25 10:24:40 UTC

Newer versions of RH(G)S have DRC disabled by default due to instability issues. We are currently not planning to resolve these issues in Gluster/NFS. When progress is made in upstream bug 1105883, we can revisit this decision.

Comment 11 Niels de Vos 2015-11-25 10:32:05 UTC

*** Bug 1049215 has been marked as a duplicate of this bug. ***

Comment 12 Niels de Vos 2015-11-25 10:33:49 UTC

*** Bug 1044909 has been marked as a duplicate of this bug. ***