Bug 1327773

Summary:	ganesha+Tiering: ganesha crashes with segfault error while large 100KB files creation in progress.
Product:	[Red Hat Storage] Red Hat Gluster Storage	Reporter:	Shashank Raj <sraj>
Component:	nfs-ganesha	Assignee:	Soumya Koduri <skoduri>
Status:	CLOSED WONTFIX	QA Contact:	Manisha Saini <msaini>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	rhgs-3.1	CC:	jthottan, kkeithle, mzywusko, ndevos, nlevinki, skoduri
Target Milestone:	---	Keywords:	ZStream
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-04-16 18:19:00 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Shashank Raj 2016-04-15 21:53:16 UTC

Description of problem:

ganesha crashes on the mounting node with segfault error while large (~100000) 100KB files creation in progress.

Version-Release number of selected component (if applicable):

glusterfs-3.7.9-1
nfs-ganesha-2.3.1-3

How reproducible:
Once

Steps to Reproduce:
1.Create a 4 node cluster and configure ganesha on it
2.Create a tiered volume, enable quota and enable ganesha on the volume.
3.Mount the volume with version 4 on 2 clients.
4.Start creating (50000) 100KB files from each mount point.
5.Observe that while IO was in progress, ganesha service on the mounted node fails with below message in logs

Apr 15 16:05:12 dhcp37-141 kernel: ganesha.nfsd[29870]: segfault at 7ff5d4071108 ip 00007ff5eea90b80 sp 00007ff5c45cf380 error 4 in libglusterfs.so.0.0.1[7ff5eea49000+c6000]
Apr 15 16:05:12 dhcp37-141 systemd: nfs-ganesha.service: main process exited, code=killed, status=11/SEGV
Apr 15 16:05:12 dhcp37-141 systemd: Unit nfs-ganesha.service entered failed state.
Apr 15 16:05:12 dhcp37-141 systemd: nfs-ganesha.service failed

6. No core is generated after the failure.

Actual results:

ganesha crashes on the mounting node with segfault error while large (~100000) 100KB files creation in progress.

Expected results:

ganesha service should not crash.

Additional info:

this seems like the same issue as (https://bugzilla.redhat.com/show_bug.cgi?id=1325975) which is filed earlier during refresh config scenario on ganesha.

Filing a separate bug as it is seen in case of tiered volume.

Comment 2 Shashank Raj 2016-04-15 22:02:03 UTC

sosreport and logs are placed under http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1327773

Comment 3 Soumya Koduri 2016-04-17 12:18:13 UTC

Shashank,
request you to provide the core to confirm that it is indeed duplicate of bug1325975.

Comment 4 Shashank Raj 2016-04-27 17:35:56 UTC

Since there was no core generated when this issue was seen, this needs to be reproduced again to confirm the segfault issue whether its same as bug1325975. 

During the recent testing, not able to reproduce this by following the exact steps mentioned in description of bug.

Will keep an eye on this bug and update bugzilla accordingly.

Comment 6 Kaleb KEITHLEY 2016-06-20 12:24:46 UTC

retest on 3.2

Comment 7 Shashank Raj 2016-07-05 06:25:20 UTC

This issue is not seen during recent RHGS 3.1.3 testing and also during 7.3 qualification with RHGS 3.1.3.

Can be closed for now. Will reopen or file a new bug if we see it again.