Bug 1659334 - FUSE mount seems to be hung and not accessible
Summary: FUSE mount seems to be hung and not accessible
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: fuse
Version: mainline
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact:
URL:
Whiteboard:
: 1702268 (view as bug list)
Depends On:
Blocks: 1662838 1698728 1702270 1702271
TreeView+ depends on / blocked
 
Reported: 2018-12-14 06:05 UTC by Prasad Desala
Modified: 2019-08-20 09:58 UTC (History)
7 users (show)

Fixed In Version: glusterfs-7.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1662838 1698728 (view as bug list)
Environment:
Last Closed: 2019-07-05 07:36:13 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Gluster.org Gerrit 22554 0 None Open core: handle memory accounting correctly 2019-04-22 03:54:56 UTC

Description Prasad Desala 2018-12-14 06:05:50 UTC
Description of problem:
=======================
When glusto automation script[1] is ran on master branch (6dev-0.370.git8293d21.el7.x86_64) the test script is getting hung at below glusto log,
2018-12-14 11:18:52,556 INFO (run) root.42.139 (cp): getfattr --absolute-names --only-values -n 'trusted.glusterfs.pathinfo' /mnt/testvol_distributed-replicated_glusterfs

It is not able to access the mount point due to which the test script is getting stuck at that point. I even logged into the client machine to see if we can access it manually but the commands(df -h, cd <mountpoint>) are getting hung.

[1] https://review.gluster.org/#/c/glusto-tests/+/21826/

Version-Release number of selected component (if applicable):
6dev-0.370.git8293d21.el7.x86_64

How reproducible:
always (when ran this glusto patch)

Steps to Reproduce:
===================
1) Run the glusto patch (https://review.gluster.org/#/c/glusto-tests/+/21826/) on master build.

Actual results:
===============
FUSE mount is not accessible.

Expected results:
================
FUSE mount should accessible without any hangs.

Comment 4 Yaniv Kaul 2019-04-17 07:44:53 UTC
What's the status of this BZ?

Comment 5 Nithya Balachandran 2019-04-17 12:31:29 UTC
(In reply to Yaniv Kaul from comment #4)
> What's the status of this BZ?

Susant (spalai) is looking into the issue downstream. The fix will be posted upstream.

Comment 6 Xavi Hernandez 2019-04-17 15:52:27 UTC
The hang is caused by a log message sent from inside a locked region in a memory allocation function, which also makes use of dynamic memory, causing a deadlock (Susant has already posted a patch [1] to avoid the deadlock)

However the log message should never be triggered because it means that something is not working fine. I found another case where this also happens [2]. Debugging it, I found an issue in memory accounting management. I fixed it in another patch [3].

[1] https://review.gluster.org/c/glusterfs/+/22589
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1663375
[3] https://review.gluster.org/c/glusterfs/+/22554

Comment 7 Xavi Hernandez 2019-04-17 15:59:33 UTC
To be clear, the referenced bug is not related to this issue, but I found it when I started debugging the original problem.

Comment 8 Worker Ant 2019-04-17 16:03:22 UTC
REVIEW: https://review.gluster.org/22554 (core: handle memory accounting correctly) posted (#4) for review on master by Xavi Hernandez

Comment 9 Xavi Hernandez 2019-04-17 16:05:12 UTC
I've referenced this bug in my patch. I think Sasant's patch should also be added.

Comment 10 Worker Ant 2019-04-22 03:54:57 UTC
REVIEW: https://review.gluster.org/22554 (core: handle memory accounting correctly) merged (#5) on master by Atin Mukherjee

Comment 11 Xavi Hernandez 2019-04-23 11:55:51 UTC
*** Bug 1702268 has been marked as a duplicate of this bug. ***

Comment 12 Amar Tumballi 2019-07-05 07:36:13 UTC
https://review.gluster.org/#/c/glusterfs/+/22600/ merged.


Note You need to log in before you can comment on or make changes to this bug.