Bug 1342171

Summary: O_DIRECT support for sharding
Product: [Community] GlusterFS Reporter: Krutika Dhananjay <kdhananj>
Component: shardingAssignee: Krutika Dhananjay <kdhananj>
Status: CLOSED CURRENTRELEASE QA Contact: bugs <bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: mainlineCC: bugs, jbyers
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.9.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1342903 1347553 (view as bug list) Environment:
Last Closed: 2017-03-27 18:13:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1342903, 1347553    

Description Krutika Dhananjay 2016-06-02 15:27:53 UTC
Description of problem:

This bug is to track work towards making odirect writes and reads work at the level of individual shards.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Vijay Bellur 2016-06-02 15:30:01 UTC
REVIEW: http://review.gluster.org/14191 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#7) for review on master by Krutika Dhananjay (kdhananj)

Comment 2 Vijay Bellur 2016-06-03 09:32:56 UTC
REVIEW: http://review.gluster.org/14191 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#8) for review on master by Krutika Dhananjay (kdhananj)

Comment 3 Vijay Bellur 2016-06-03 09:32:59 UTC
REVIEW: http://review.gluster.org/14639 (posix, shard: Use page-aligned buffer for o-direct reads) posted (#1) for review on master by Krutika Dhananjay (kdhananj)

Comment 4 Vijay Bellur 2016-06-03 10:11:08 UTC
REVIEW: http://review.gluster.org/14191 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#9) for review on master by Krutika Dhananjay (kdhananj)

Comment 5 Vijay Bellur 2016-06-03 10:11:11 UTC
REVIEW: http://review.gluster.org/14639 (posix, shard: Use page-aligned buffer for o-direct reads) posted (#2) for review on master by Krutika Dhananjay (kdhananj)

Comment 6 Vijay Bellur 2016-06-04 01:58:14 UTC
COMMIT: http://review.gluster.org/14639 committed in master by Pranith Kumar Karampuri (pkarampu) 
------
commit 9e0cdf1d2393d7bdb0eac019e70dce7c5f4f59c9
Author: Krutika Dhananjay <kdhananj>
Date:   Thu Jun 2 19:45:03 2016 +0530

    posix, shard: Use page-aligned buffer for o-direct reads
    
    and also make shard_readv_do() pass the correct flags when
    the original fd is opened with O_DIRECT.
    
    Change-Id: Ic2f8ad900743ed3f7cab56948bcf1358d247a311
    BUG: 1342171
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Reviewed-on: http://review.gluster.org/14639
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Smoke: Gluster Build System <jenkins.com>
    CentOS-regression: Gluster Build System <jenkins.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>

Comment 7 Vijay Bellur 2016-06-06 17:58:22 UTC
REVIEW: http://review.gluster.org/14191 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#10) for review on master by Pranith Kumar Karampuri (pkarampu)

Comment 8 Vijay Bellur 2016-06-06 19:01:27 UTC
REVIEW: http://review.gluster.org/14191 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#11) for review on master by Pranith Kumar Karampuri (pkarampu)

Comment 9 Vijay Bellur 2016-06-07 07:19:32 UTC
COMMIT: http://review.gluster.org/14191 committed in master by Pranith Kumar Karampuri (pkarampu) 
------
commit eb1744582d57d0f9fe08275781800c0c3459697f
Author: Krutika Dhananjay <kdhananj>
Date:   Mon May 2 16:51:10 2016 +0530

    core, shard: Make shards inherit main file's O_DIRECT flag if present
    
    If the application opens a file with O_DIRECT, the shards'
    anon fds would also need to inherit the flag. Towards this,
    shard xl would be passing the odirect flag in the @flags parameter
    to the WRITEV fop. This will be used in anon fd resolution
    and subsequent opening by posix xl.
    
    Change-Id: Iddb75c9ed14ce5a8c5d2128ad09b749f46e3b0c2
    BUG: 1342171
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Reviewed-on: http://review.gluster.org/14191
    Tested-by: Pranith Kumar Karampuri <pkarampu>
    Smoke: Gluster Build System <jenkins.com>
    CentOS-regression: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>

Comment 10 Vijay Bellur 2016-06-07 12:40:40 UTC
REVIEW: http://review.gluster.org/14665 (libglusterfs: Negate all but O_DIRECT flag if present on anon fds) posted (#1) for review on master by Krutika Dhananjay (kdhananj)

Comment 11 Vijay Bellur 2016-06-09 02:53:22 UTC
REVIEW: http://review.gluster.org/14665 (libglusterfs: Negate all but O_DIRECT flag if present on anon fds) posted (#2) for review on master by Krutika Dhananjay (kdhananj)

Comment 12 Vijay Bellur 2016-06-09 09:29:26 UTC
REVIEW: http://review.gluster.org/14665 (libglusterfs: Negate all but O_DIRECT flag if present on anon fds) posted (#3) for review on master by Krutika Dhananjay (kdhananj)

Comment 13 Vijay Bellur 2016-06-10 05:05:52 UTC
COMMIT: http://review.gluster.org/14665 committed in master by Atin Mukherjee (amukherj) 
------
commit 79dfe7d197ced45bd3b84f71d5c889e95f4dec1a
Author: Krutika Dhananjay <kdhananj>
Date:   Tue Jun 7 17:15:56 2016 +0530

    libglusterfs: Negate all but O_DIRECT flag if present on anon fds
    
    This is to prevent any unforeseen problems that might arise due to
    writevs and readvs being wound with @flag parameter containing
    O_TRUNC or O_APPEND especially wrt translators like sharding and ec
    where O_TRUNC write or O_APPEND write on individual shards/fragments
    is not the same as O_TRUNC write or O_APPEND write as expected by the
    application.
    
    Change-Id: I9e5206a6ce2b1b70df61ff23b1c961cf25bf7ff9
    BUG: 1342171
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Reviewed-on: http://review.gluster.org/14665
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>
    CentOS-regression: Gluster Build System <jenkins.com>
    Smoke: Gluster Build System <jenkins.com>
    Tested-by: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>

Comment 14 Shyamsundar 2017-03-27 18:13:07 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.9.0, please open a new bug report.

glusterfs-3.9.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2016-November/029281.html
[2] https://www.gluster.org/pipermail/gluster-users/