Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1729950

Summary:	Deadlock when generating statedumps
Product:	[Community] GlusterFS	Reporter:	Xavi Hernandez <jahernan>
Component:	core	Assignee:	Xavi Hernandez <jahernan>
Status:	CLOSED NEXTRELEASE	QA Contact:
Severity:	high	Docs Contact:
Priority:	medium
Version:	7	CC:	bugs, pasik
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1727068	Environment:
Last Closed:	2019-07-20 07:43:32 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1727068
Bug Blocks:

Description Xavi Hernandez 2019-07-15 12:04:28 UTC

+++ This bug was initially created as a clone of Bug #1727068 +++

Description of problem:

There seems to be a deadlock between statedump generation and creation of an anonymous fd.

When a statedump is generated and it's going to dump inode information, it takes the inode table lock and then each individual inode lock.

On the other side, when an anonymous fd is created, first it's acquired the lock from the inode, and then the lock from the inode table when the refcount of the inode is incremented.

This causes a deadlock because of reverse acquisition order of the locks.

This deadlock may not be exclusive of statedump, but there are few places where we take both locks at the same time, and for sure none of them does that in a so "massive" way.

Version-Release number of selected component (if applicable): mainline


How reproducible:

It's a race hard to reproduce.

Steps to Reproduce:
1.
2.
3.

Actual results:

Generation of statedump hangs and mount point becomes unresponsiveness.

Expected results:

Generation of statedump should complete successfully and mount point continue working normally.


Additional info:

Comment 1 Worker Ant 2019-07-15 12:06:16 UTC

REVIEW: https://review.gluster.org/23046 (core: fix deadlock between statedump and fd_anonymous()) posted (#1) for review on release-7 by Xavi Hernandez

Comment 2 Worker Ant 2019-07-20 07:43:32 UTC

REVIEW: https://review.gluster.org/23046 (core: fix deadlock between statedump and fd_anonymous()) merged (#2) on release-7 by Rinku Kothiya