Bug 1476653

Summary:	cassandra fails on gluster-block with both replicate and ec volumes
Product:	[Community] GlusterFS	Reporter:	Pranith Kumar K <pkarampu>
Component:	core	Assignee:	bugs <bugs>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	high
Version:	3.12	CC:	bugs, vbellur
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.12.0	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1474190	Environment:
Last Closed:	2017-09-05 17:37:48 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1474190
Bug Blocks:

Description Pranith Kumar K 2017-07-31 05:17:44 UTC

+++ This bug was initially created as a clone of Bug #1474190 +++

Description of problem:
gluster-block file is opened with O_DIRECT but because block-profile has filter_O_DIRECT enabled, it is leading to high latency for fsync which leads to failures in cassandra. Disabling remote-dio fixed this issue. We need to change remote-dio to disabled in gluster-block.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

--- Additional comment from Worker Ant on 2017-07-24 02:25:45 EDT ---

REVIEW: https://review.gluster.org/17856 (extras: Disable remote-dio in gluster-block profile) posted (#1) for review on master by Pranith Kumar Karampuri (pkarampu)

--- Additional comment from Worker Ant on 2017-07-24 10:17:14 EDT ---

COMMIT: https://review.gluster.org/17856 committed in master by Pranith Kumar Karampuri (pkarampu) 
------
commit 0b3fec6924cad5c9f38941550ab4106972efa5cc
Author: Pranith Kumar K <pkarampu>
Date:   Mon Jul 24 11:31:56 2017 +0530

    extras: Disable remote-dio in gluster-block profile
    
    gluster-block file is opened with O_DIRECT but because block-profile has
    remote-dio enabled, it is leading to high latency for fsync which leads to
    failures in cassandra. Disabling remote-dio fixed this issue. We need to change
    remote-dio to disabled in gluster-block.
    
    BUG: 1474190
    Change-Id: Ifd845ea9cbdcc08dd6073faca6082682af376ca3
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: https://review.gluster.org/17856
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Krutika Dhananjay <kdhananj>
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Jeff Darcy <jeff.us>
    Reviewed-by: Niels de Vos <ndevos>

Comment 1 Worker Ant 2017-07-31 05:18:48 UTC

REVIEW: https://review.gluster.org/17918 (extras: Disable remote-dio in gluster-block profile) posted (#1) for review on release-3.12 by Pranith Kumar Karampuri (pkarampu)

Comment 2 Worker Ant 2017-07-31 16:30:02 UTC

COMMIT: https://review.gluster.org/17918 committed in release-3.12 by Shyamsundar Ranganathan (srangana) 
------
commit 46f52dc131f86c8b3d61c8d9ac2916bd1d979068
Author: Pranith Kumar K <pkarampu>
Date:   Mon Jul 24 11:31:56 2017 +0530

    extras: Disable remote-dio in gluster-block profile
    
    gluster-block file is opened with O_DIRECT but because block-profile has
    remote-dio enabled, it is leading to high latency for fsync which leads to
    failures in cassandra. Disabling remote-dio fixed this issue. We need to change
    remote-dio to disabled in gluster-block.
    
     >BUG: 1474190
     >Change-Id: Ifd845ea9cbdcc08dd6073faca6082682af376ca3
     >Signed-off-by: Pranith Kumar K <pkarampu>
     >Reviewed-on: https://review.gluster.org/17856
     >CentOS-regression: Gluster Build System <jenkins.org>
     >Reviewed-by: Krutika Dhananjay <kdhananj>
     >Smoke: Gluster Build System <jenkins.org>
     >Reviewed-by: Jeff Darcy <jeff.us>
     >Reviewed-by: Niels de Vos <ndevos>
     >(cherry picked from commit 0b3fec6924cad5c9f38941550ab4106972efa5cc)
    
    BUG: 1476653
    Change-Id: Ifd845ea9cbdcc08dd6073faca6082682af376ca3
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: https://review.gluster.org/17918
    CentOS-regression: Gluster Build System <jenkins.org>
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Krutika Dhananjay <kdhananj>
    Reviewed-by: Niels de Vos <ndevos>
    Reviewed-by: Shyamsundar Ranganathan <srangana>

Comment 3 Shyamsundar 2017-09-05 17:37:48 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.12.0, please open a new bug report.

glusterfs-3.12.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-September/000082.html
[2] https://www.gluster.org/pipermail/gluster-users/