Bug 1394482 - A hot tier brick becomes full, causing the entire volume to have issues and returns stale file handle and input/output error.
Summary: A hot tier brick becomes full, causing the entire volume to have issues and r...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: tiering
Version: 3.9
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
Assignee: Milind Changire
QA Contact: bugs@gluster.org
URL:
Whiteboard:
Depends On: 1366648
Blocks: 1361759
TreeView+ depends on / blocked
 
Reported: 2016-11-12 12:27 UTC by Milind Changire
Modified: 2017-03-08 10:20 UTC (History)
9 users (show)

Fixed In Version: glusterfs-3.9.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1366648
Environment:
Last Closed: 2017-03-08 10:19:47 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Comment 1 Worker Ant 2016-11-13 05:26:14 UTC
REVIEW: http://review.gluster.org/15835 (cluster/tier: handle fast demotions) posted (#1) for review on release-3.9 by Milind Changire (mchangir)

Comment 2 Worker Ant 2016-11-28 16:51:40 UTC
COMMIT: http://review.gluster.org/15835 committed in release-3.9 by Dan Lambright (dlambrig) 
------
commit 1001a4f7705f991f3ae1611997cf01b341ac453a
Author: Milind Changire <mchangir>
Date:   Sun Nov 13 10:52:03 2016 +0530

    cluster/tier: handle fast demotions
    
    Demote files on priority if hi-watermark has been breached and continue
    to demote until the watermark drops below hi-watermark.
    
    Monitor watermark more frequently.
    Trigger demotion as soon as hi-watermark is breached.
    Add cluster.tier-query-limit option to limit number
    of files returned from the database query for every iteration of
    tier_migrate_using_query_file(). If watermark hasn't dropped below
    hi-watermark during the first iteration, the next iteration will be
    triggered approximately 1 second after tier_demote() returns to the
    main tiering loop.
    Update changetimerecorder xlator to handle query for emergency demote
    mode.
    
    Add tier-ctr-interface.h:
    Move tier and ctr interface specific macros and struct definition from
    libglusterfs/src/gfdb/gfdb_data_store.h to new header
    libglusterfs/src/tier-ctr-interface.h
    
    > Reviewed-on: http://review.gluster.org/15158
    > Smoke: Gluster Build System <jenkins.org>
    > CentOS-regression: Gluster Build System <jenkins.org>
    > NetBSD-regression: NetBSD Build System <jenkins.org>
    > Reviewed-by: Dan Lambright <dlambrig>
    (cherry picked from commit 460016428cf27484c333227f534c2e2f73a37fb1)
    
    Change-Id: If56af78c6c81d37529b9b6e65ae606ba5c99a811
    BUG: 1394482
    Signed-off-by: Milind Changire <mchangir>
    Reviewed-on: http://review.gluster.org/15835
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Dan Lambright <dlambrig>

Comment 3 Worker Ant 2016-12-02 05:05:37 UTC
REVIEW: http://review.gluster.org/16000 (cluster/tier: fix op-version for tier-query-limit) posted (#1) for review on release-3.9 by Milind Changire (mchangir)

Comment 4 Worker Ant 2016-12-02 10:17:01 UTC
COMMIT: http://review.gluster.org/16000 committed in release-3.9 by Atin Mukherjee (amukherj) 
------
commit 89ee91e4256029014ccf2f0475d638e0558aa1d5
Author: Milind Changire <mchangir>
Date:   Fri Dec 2 10:29:51 2016 +0530

    cluster/tier: fix op-version for tier-query-limit
    
    Correct the op-version for tier-query-limit option from 3.9.0 to 3.9.1
    
    > BUG: 1366648
    > Reviewed-on: http://review.gluster.org/15990
    > Reviewed-by: Dan Lambright <dlambrig>
    > Smoke: Gluster Build System <jenkins.org>
    > NetBSD-regression: NetBSD Build System <jenkins.org>
    > CentOS-regression: Gluster Build System <jenkins.org>
    > Reviewed-by: Atin Mukherjee <amukherj>
    (cherry picked from commit 530453c78146e8ba4f13636e1dec1ea59849c783)
    
    Change-Id: I3a52a94c2708a97c18377e945d559a51d8025c41
    BUG: 1394482
    Signed-off-by: Milind Changire <mchangir>
    Reviewed-on: http://review.gluster.org/16000
    Reviewed-by: Atin Mukherjee <amukherj>
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 5 Kaushal 2017-03-08 10:19:47 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.9.1, please open a new bug report.

glusterfs-3.9.1 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2017-January/029725.html
[2] https://www.gluster.org/pipermail/gluster-users/

Comment 6 Kaushal 2017-03-08 10:20:58 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.9.1, please open a new bug report.

glusterfs-3.9.1 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2017-January/029725.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.