Bug 1429330 - [crawler]: auxiliary mount remains even after crawler finishes
Summary: [crawler]: auxiliary mount remains even after crawler finishes
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: quota
Version: mainline
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Sanoj Unnikrishnan
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 1428302
TreeView+ depends on / blocked
 
Reported: 2017-03-06 06:22 UTC by Sanoj Unnikrishnan
Modified: 2017-05-30 18:46 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.11.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1428302
Environment:
Last Closed: 2017-05-30 18:46:48 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Sanoj Unnikrishnan 2017-03-06 06:22:12 UTC
Description of problem:

 Auxiliary mount remains even after crawler finishes crawling , when quota is enabled when bricks are down.

How reproducible:

1/1

Steps to Reproduce:
1. Kill one of the brick in each sub-volume
2. Enable quota 
3. 

Actual results:

[root@rhs-arch-srv3 tmp]# ps -ef | grep quota
root      7762     1  0 07:01 ?        00:00:00 /usr/sbin/glusterfs --volfile-server localhost --volfile-id clone2 -l /var/log/glusterfs/quota-mount-clone2.log -p /var/run/gluster/clone2.pid --client-pid -5 /var/run/gluster/clone2/
root     14877  6664  0 09:20 pts/0    00:00:00 grep quota
root     15095     1  0 Mar01 ?        00:00:01 /usr/sbin/glusterfs -s localhost --volfile-id client_per_brick/clone2.client.10.70.36.3.var-run-gluster-snaps-0c286cc539c34063aae2d17fdb66fd71-brick2-b2.vol --use-readdirp=yes --client-pid -100 -l /var/log/glusterfs/quota_crawl/var-run-gluster-snaps-0c286cc539c34063aae2d17fdb66fd71-brick2-b2.log /var/run/gluster/tmp/mntRlTPnv
root     18829     1  0 Mar01 ?        00:00:02 /usr/sbin/glusterfs -s localhost --volfile-id gluster/quotad -p /var/lib/glusterd/quotad/run/quotad.pid -l /var/log/glusterfs/quotad.log -S /var/run/gluster/3b3b5f5b2555b948d2dbb20210fbc9d6.socket --xlator-option *replicate*.data-self-heal=off --xlator-option *replicate*.metadata-self-heal=off --xlator-option *replicate*.entry-self-heal=off


glusterd logs:

 [2017-03-01 06:02:36.883481] W [MSGID: 106033] [glusterd-quota.c:331:_glusterd_quota_initiate_fs_crawl] 0-management: chdir /var/run/gluster/tmp/mntRlTPnv failed [Transport endpoint is not connected]


Since the brick is down we get transport_disconnected on chdir to mountpoint.

In the function _glusterd_quota_initiate_fs_crawl, we exit if chdir fails without doing a umount. 

                ret = chdir (mountdir);
                if (ret == -1) {
                        gf_msg (THIS->name, GF_LOG_WARNING, errno,
                                GD_MSG_DIR_OP_FAILED, "chdir %s failed",
                                mountdir);
                        exit (EXIT_FAILURE);
                }

To fix this we need to do a lazy_umount before exit.

Comment 1 Worker Ant 2017-03-06 12:11:38 UTC
REVIEW: https://review.gluster.org/16853 (Fixes Stale auxiliary mount when crawler fails to spawn) posted (#1) for review on master by sanoj-unnikrishnan (sunnikri)

Comment 2 Worker Ant 2017-04-05 05:23:33 UTC
COMMIT: https://review.gluster.org/16853 committed in master by Jeff Darcy (jeff.us) 
------
commit 6b069086cbee39aec9da77b1831e88fe85258b0f
Author: Sanoj Unnikrishnan <sunnikri>
Date:   Mon Mar 6 12:36:02 2017 +0530

    Fixes Stale auxiliary mount when crawler fails to spawn
    
    The auxiliary mount created for crawling remains if the crawler was not
    successfully spawned due to transport disconnect or other such issues.
    The patch ensures the mount is cleared in those code paths as well.
    
    Change-Id: I659fcc1d1956f8e05a37b75ebe3f3a00c24693e8
    BUG: 1429330
    Signed-off-by: Sanoj Unnikrishnan <sunnikri>
    Reviewed-on: https://review.gluster.org/16853
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Zhou Zhengping <johnzzpcrystal>
    Reviewed-by: Manikandan Selvaganesh <manikandancs333>

Comment 3 Shyamsundar 2017-05-30 18:46:48 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.11.0, please open a new bug report.

glusterfs-3.11.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-May/000073.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.