Bug 1729950 - Deadlock when generating statedumps
Summary: Deadlock when generating statedumps
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: core
Version: 7
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
Assignee: Xavi Hernandez
QA Contact:
URL:
Whiteboard:
Depends On: 1727068
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-15 12:04 UTC by Xavi Hernandez
Modified: 2019-07-20 07:43 UTC (History)
2 users (show)

Fixed In Version:
Clone Of: 1727068
Environment:
Last Closed: 2019-07-20 07:43:32 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Gluster.org Gerrit 23046 0 None Merged core: fix deadlock between statedump and fd_anonymous() 2019-07-20 07:43:31 UTC

Description Xavi Hernandez 2019-07-15 12:04:28 UTC
+++ This bug was initially created as a clone of Bug #1727068 +++

Description of problem:

There seems to be a deadlock between statedump generation and creation of an anonymous fd.

When a statedump is generated and it's going to dump inode information, it takes the inode table lock and then each individual inode lock.

On the other side, when an anonymous fd is created, first it's acquired the lock from the inode, and then the lock from the inode table when the refcount of the inode is incremented.

This causes a deadlock because of reverse acquisition order of the locks.

This deadlock may not be exclusive of statedump, but there are few places where we take both locks at the same time, and for sure none of them does that in a so "massive" way.

Version-Release number of selected component (if applicable): mainline


How reproducible:

It's a race hard to reproduce.

Steps to Reproduce:
1.
2.
3.

Actual results:

Generation of statedump hangs and mount point becomes unresponsiveness.

Expected results:

Generation of statedump should complete successfully and mount point continue working normally.


Additional info:

Comment 1 Worker Ant 2019-07-15 12:06:16 UTC
REVIEW: https://review.gluster.org/23046 (core: fix deadlock between statedump and fd_anonymous()) posted (#1) for review on release-7 by Xavi Hernandez

Comment 2 Worker Ant 2019-07-20 07:43:32 UTC
REVIEW: https://review.gluster.org/23046 (core: fix deadlock between statedump and fd_anonymous()) merged (#2) on release-7 by Rinku Kothiya


Note You need to log in before you can comment on or make changes to this bug.