Bug 1306852

Summary:	Tiering threads can starve each other
Product:	[Community] GlusterFS	Reporter:	Dan Lambright <dlambrig>
Component:	tiering	Assignee:	Dan Lambright <dlambrig>
Status:	CLOSED CURRENTRELEASE	QA Contact:	bugs <bugs>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	mainline	CC:	bugs, sarumuga
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	Triaged
Fixed In Version:	glusterfs-3.8rc2	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-06-16 13:57:29 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Dan Lambright 2016-02-11 22:34:21 UTC

Description of problem:

It is possible the promotion thread will prevent the demotion thread from running. This can happen if the promotion thread is moving data at the same time the demotion thread should run.

Version-Release number of selected component (if applicable):

N/A

How reproducible:

Difficult

Steps to Reproduce:
1. Have a file on the hot tier that has become cold.
2. Promote many large files in a loop at the same time demotion is scheduled.
3. Observe demotion does not happen. 


Actual results:

File stays on hot tier.

Expected results:

File is demoted.

Additional info:

Comment 1 Vijay Bellur 2016-02-11 22:41:40 UTC

REVIEW: http://review.gluster.org/13433 (cluster/tier: make promotion and demotion independant (WIP)) posted (#1) for review on master by Dan Lambright (dlambrig)

Comment 2 Vijay Bellur 2016-02-16 18:37:44 UTC

REVIEW: http://review.gluster.org/13433 (cluster/tier: make promotion and demotion independant (WIP)) posted (#2) for review on master by Dan Lambright (dlambrig)

Comment 3 Vijay Bellur 2016-02-19 17:57:27 UTC

REVIEW: http://review.gluster.org/13433 (cluster/tier: make promotion and demotion independant) posted (#3) for review on master by Dan Lambright (dlambrig)

Comment 4 Vijay Bellur 2016-02-29 01:32:28 UTC

COMMIT: http://review.gluster.org/13433 committed in master by Dan Lambright (dlambrig) 
------
commit 87b19927ed41cfe47579c1c069cb6640fd47a732
Author: Dan Lambright <dlambrig>
Date:   Tue Feb 9 03:37:31 2016 +0000

    cluster/tier: make promotion and demotion independant
    
    Currently a main loop in tiering spawns promotion and demotion threads,
    and does a join to wait for them to complete. When one of the two
    threads takes a long time, the main thread waits for it
    before exiting the join. It may wait so long the scheduled time
    for the other thread is skipped. In the case of demotion, it may be
    a long time before another attempt. This patch fixes that by
    making the promotion and demotion activities independant. A side
    effect of this change is the logic is significantly simplified.
    
    Change-Id: I1196bd4bbfc95e8aa326a9bd4ebf395032369d1c
    BUG: 1306852
    Signed-off-by: Dan Lambright <dlambrig>
    Reviewed-on: http://review.gluster.org/13433
    Smoke: Gluster Build System <jenkins.com>
    Reviewed-by: Joseph Fernandes
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>

Comment 5 Niels de Vos 2016-06-16 13:57:29 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user