Bug 1637802 - data-self-heal in arbiter volume results in stale locks.
Summary: data-self-heal in arbiter volume results in stale locks.
Alias: None
Product: GlusterFS
Classification: Community
Component: replicate
Version: mainline
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Ravishankar N
QA Contact:
: 1664223 (view as bug list)
Depends On:
Blocks: 1636902 1637953 1637989 1638026 1638159
TreeView+ depends on / blocked
Reported: 2018-10-10 06:41 UTC by Ravishankar N
Modified: 2019-03-25 16:31 UTC (History)
4 users (show)

Fixed In Version: glusterfs-6.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1637953 1637989 1638159 (view as bug list)
Last Closed: 2018-11-20 08:50:48 UTC
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:

Attachments (Terms of Use)

Description Ravishankar N 2018-10-10 06:41:53 UTC
Description of problem:
commit eb472d82a083883335bc494b87ea175ac43471ff in master introduced a bug where a data-self-heal on a file in arbiter leaves a stale inodelk behind on the bricks. Thus any new write to the file from a client can hang

How reproducible:

Steps to Reproduce:
1. Create 1x (2+1) arbiter, fuse mount it and create a file
2. Kill arbiter brick, write to the file, bring back arbiter and let self-heal complete.
3. Next write to the file from mount will hang because the inodelk gets blocked because of the previous stale locks left behind from self-heal

Additional info:
Downstream bug which found the issue: BZ 1636902

Comment 1 Worker Ant 2018-10-10 06:56:21 UTC
REVIEW: https://review.gluster.org/21380 (afr: prevent winding inodelks twice for arbiter volumes) posted (#1) for review on master by Ravishankar N

Comment 2 Worker Ant 2018-10-10 16:19:31 UTC
COMMIT: https://review.gluster.org/21380 committed in master by "Amar Tumballi" <amarts@redhat.com> with a commit message- afr: prevent winding inodelks twice for arbiter volumes

In an arbiter volume, if there is a pending data heal of a file only on
arbiter brick, self-heal takes inodelks twice due to a code-bug but unlocks
it only once, leaving behind a stale lock on the brick. This causes
the next write to the file to hang.

Fix the code-bug to take lock only once. This bug was introduced master
with commit eb472d82a083883335bc494b87ea175ac43471ff

Thanks to  Pranith Kumar K <pkarampu@redhat.com> for finding the RCA.

fixes: bz#1637802
Change-Id: I15ad969e10a6a3c4bd255e2948b6be6dcddc61e1
Signed-off-by: Ravishankar N <ravishankar@redhat.com>

Comment 3 Karthik U S 2019-02-27 10:28:14 UTC
*** Bug 1664223 has been marked as a duplicate of this bug. ***

Comment 4 Shyamsundar 2019-03-25 16:31:17 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-6.0, please open a new bug report.

glusterfs-6.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://lists.gluster.org/pipermail/announce/2019-March/000120.html
[2] https://www.gluster.org/pipermail/gluster-users/

Note You need to log in before you can comment on or make changes to this bug.