Bug 1347553 - O_DIRECT support for sharding
Summary: O_DIRECT support for sharding
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: sharding
Version: 3.8.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact: bugs@gluster.org
URL:
Whiteboard:
Depends On: 1342171 1350407
Blocks: 1342903
TreeView+ depends on / blocked
 
Reported: 2016-06-17 08:37 UTC by Pranith Kumar K
Modified: 2016-07-08 14:43 UTC (History)
2 users (show)

Fixed In Version: glusterfs-3.8.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1342171
Environment:
Last Closed:
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Pranith Kumar K 2016-06-17 08:37:45 UTC
+++ This bug was initially created as a clone of Bug #1342171 +++

Description of problem:

This bug is to track work towards making odirect writes and reads work at the level of individual shards.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

--- Additional comment from Vijay Bellur on 2016-06-02 11:30:01 EDT ---

REVIEW: http://review.gluster.org/14191 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#7) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Vijay Bellur on 2016-06-03 05:32:56 EDT ---

REVIEW: http://review.gluster.org/14191 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#8) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Vijay Bellur on 2016-06-03 05:32:59 EDT ---

REVIEW: http://review.gluster.org/14639 (posix, shard: Use page-aligned buffer for o-direct reads) posted (#1) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Vijay Bellur on 2016-06-03 06:11:08 EDT ---

REVIEW: http://review.gluster.org/14191 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#9) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Vijay Bellur on 2016-06-03 06:11:11 EDT ---

REVIEW: http://review.gluster.org/14639 (posix, shard: Use page-aligned buffer for o-direct reads) posted (#2) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Vijay Bellur on 2016-06-03 21:58:14 EDT ---

COMMIT: http://review.gluster.org/14639 committed in master by Pranith Kumar Karampuri (pkarampu) 
------
commit 9e0cdf1d2393d7bdb0eac019e70dce7c5f4f59c9
Author: Krutika Dhananjay <kdhananj>
Date:   Thu Jun 2 19:45:03 2016 +0530

    posix, shard: Use page-aligned buffer for o-direct reads
    
    and also make shard_readv_do() pass the correct flags when
    the original fd is opened with O_DIRECT.
    
    Change-Id: Ic2f8ad900743ed3f7cab56948bcf1358d247a311
    BUG: 1342171
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Reviewed-on: http://review.gluster.org/14639
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Smoke: Gluster Build System <jenkins.com>
    CentOS-regression: Gluster Build System <jenkins.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>

--- Additional comment from Vijay Bellur on 2016-06-06 13:58:22 EDT ---

REVIEW: http://review.gluster.org/14191 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#10) for review on master by Pranith Kumar Karampuri (pkarampu)

--- Additional comment from Vijay Bellur on 2016-06-06 15:01:27 EDT ---

REVIEW: http://review.gluster.org/14191 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#11) for review on master by Pranith Kumar Karampuri (pkarampu)

--- Additional comment from Vijay Bellur on 2016-06-07 03:19:32 EDT ---

COMMIT: http://review.gluster.org/14191 committed in master by Pranith Kumar Karampuri (pkarampu) 
------
commit eb1744582d57d0f9fe08275781800c0c3459697f
Author: Krutika Dhananjay <kdhananj>
Date:   Mon May 2 16:51:10 2016 +0530

    core, shard: Make shards inherit main file's O_DIRECT flag if present
    
    If the application opens a file with O_DIRECT, the shards'
    anon fds would also need to inherit the flag. Towards this,
    shard xl would be passing the odirect flag in the @flags parameter
    to the WRITEV fop. This will be used in anon fd resolution
    and subsequent opening by posix xl.
    
    Change-Id: Iddb75c9ed14ce5a8c5d2128ad09b749f46e3b0c2
    BUG: 1342171
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Reviewed-on: http://review.gluster.org/14191
    Tested-by: Pranith Kumar Karampuri <pkarampu>
    Smoke: Gluster Build System <jenkins.com>
    CentOS-regression: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>

--- Additional comment from Vijay Bellur on 2016-06-07 08:40:40 EDT ---

REVIEW: http://review.gluster.org/14665 (libglusterfs: Negate all but O_DIRECT flag if present on anon fds) posted (#1) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Vijay Bellur on 2016-06-08 22:53:22 EDT ---

REVIEW: http://review.gluster.org/14665 (libglusterfs: Negate all but O_DIRECT flag if present on anon fds) posted (#2) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Vijay Bellur on 2016-06-09 05:29:26 EDT ---

REVIEW: http://review.gluster.org/14665 (libglusterfs: Negate all but O_DIRECT flag if present on anon fds) posted (#3) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Vijay Bellur on 2016-06-10 01:05:52 EDT ---

COMMIT: http://review.gluster.org/14665 committed in master by Atin Mukherjee (amukherj) 
------
commit 79dfe7d197ced45bd3b84f71d5c889e95f4dec1a
Author: Krutika Dhananjay <kdhananj>
Date:   Tue Jun 7 17:15:56 2016 +0530

    libglusterfs: Negate all but O_DIRECT flag if present on anon fds
    
    This is to prevent any unforeseen problems that might arise due to
    writevs and readvs being wound with @flag parameter containing
    O_TRUNC or O_APPEND especially wrt translators like sharding and ec
    where O_TRUNC write or O_APPEND write on individual shards/fragments
    is not the same as O_TRUNC write or O_APPEND write as expected by the
    application.
    
    Change-Id: I9e5206a6ce2b1b70df61ff23b1c961cf25bf7ff9
    BUG: 1342171
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Reviewed-on: http://review.gluster.org/14665
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>
    CentOS-regression: Gluster Build System <jenkins.com>
    Smoke: Gluster Build System <jenkins.com>
    Tested-by: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>

Comment 1 Vijay Bellur 2016-06-17 08:59:10 UTC
REVIEW: http://review.gluster.org/14752 (features/shard: Don't modify readv size) posted (#1) for review on release-3.8 by Pranith Kumar Karampuri (pkarampu)

Comment 2 Vijay Bellur 2016-06-17 08:59:13 UTC
REVIEW: http://review.gluster.org/14753 (posix, shard: Use page-aligned buffer for o-direct reads) posted (#1) for review on release-3.8 by Pranith Kumar Karampuri (pkarampu)

Comment 3 Vijay Bellur 2016-06-17 08:59:16 UTC
REVIEW: http://review.gluster.org/14754 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#1) for review on release-3.8 by Pranith Kumar Karampuri (pkarampu)

Comment 4 Vijay Bellur 2016-06-17 08:59:18 UTC
REVIEW: http://review.gluster.org/14755 (libglusterfs: Negate all but O_DIRECT flag if present on anon fds) posted (#1) for review on release-3.8 by Pranith Kumar Karampuri (pkarampu)

Comment 5 Vijay Bellur 2016-06-22 16:27:22 UTC
REVIEW: http://review.gluster.org/14754 (core, shard: Make shards inherit main file's O_DIRECT flag if present) posted (#2) for review on release-3.8 by Krutika Dhananjay (kdhananj)

Comment 6 Vijay Bellur 2016-06-22 16:27:25 UTC
REVIEW: http://review.gluster.org/14755 (libglusterfs: Negate all but O_DIRECT flag if present on anon fds) posted (#2) for review on release-3.8 by Krutika Dhananjay (kdhananj)

Comment 7 Vijay Bellur 2016-06-22 16:27:27 UTC
REVIEW: http://review.gluster.org/14753 (posix, shard: Use page-aligned buffer for o-direct reads) posted (#2) for review on release-3.8 by Krutika Dhananjay (kdhananj)

Comment 8 Vijay Bellur 2016-06-27 15:17:15 UTC
COMMIT: http://review.gluster.org/14753 committed in release-3.8 by Pranith Kumar Karampuri (pkarampu) 
------
commit 7cbd3be5f8c5d70226e732a151a487eb047662b2
Author: Pranith Kumar K <pkarampu>
Date:   Fri Jun 17 14:14:22 2016 +0530

    posix, shard: Use page-aligned buffer for o-direct reads
    
            Backport of: http://review.gluster.org/#/c/14639/
    
    and also make shard_readv_do() pass the correct flags when
    the original fd is opened with O_DIRECT.
    
     >Change-Id: I253f74459656b047de652b6aace487c0e2ac71f6
     >BUG: 1342903
     >Signed-off-by: Krutika Dhananjay <kdhananj>
     >Reviewed-on: http://review.gluster.org/14650
     >Smoke: Gluster Build System <jenkins.org>
     >CentOS-regression: Gluster Build System <jenkins.org>
     >NetBSD-regression: NetBSD Build System <jenkins.org>
     >Reviewed-by: Pranith Kumar Karampuri <pkarampu>
    
    Change-Id: Ic774d2c21efc05566a1b842f7dce24509cbeaac4
    BUG: 1347553
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: http://review.gluster.org/14753
    Smoke: Gluster Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>

Comment 9 Vijay Bellur 2016-06-27 15:17:58 UTC
COMMIT: http://review.gluster.org/14754 committed in release-3.8 by Pranith Kumar Karampuri (pkarampu) 
------
commit 42d6a493570599277ed88b51ab0fb9760840ed1b
Author: Krutika Dhananjay <kdhananj>
Date:   Mon May 2 16:51:10 2016 +0530

    core, shard: Make shards inherit main file's O_DIRECT flag if present
    
            Backport of: http://review.gluster.org/14191
    
    If the application opens a file with O_DIRECT, the shards'
    anon fds would also need to inherit the flag. Towards this,
    shard xl would be passing the odirect flag in the @flags parameter
    to the WRITEV fop. This will be used in anon fd resolution
    and subsequent opening by posix xl.
    
     >Change-Id: I3a0593fa46cc25e390a5762a0354b469c2a1532d
     >BUG: 1342903
     >Signed-off-by: Krutika Dhananjay <kdhananj>
     >Reviewed-on: http://review.gluster.org/14663
     >Smoke: Gluster Build System <jenkins.org>
     >CentOS-regression: Gluster Build System <jenkins.org>
     >NetBSD-regression: NetBSD Build System <jenkins.org>
     >Reviewed-by: Jeff Darcy <jdarcy>
    
    Change-Id: Ibfc164aa7f9eecd6993255f1c03557f2ec35ac8c
    BUG: 1347553
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: http://review.gluster.org/14754
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 10 Vijay Bellur 2016-06-27 15:18:43 UTC
COMMIT: http://review.gluster.org/14752 committed in release-3.8 by Pranith Kumar Karampuri (pkarampu) 
------
commit cac16a0e5713f85a65f03b3eb5b62abfa894ac94
Author: Pranith Kumar K <pkarampu>
Date:   Fri Jun 3 04:22:27 2016 +0530

    features/shard: Don't modify readv size
    
            Backport of: http://review.gluster.org/14623
    
    For o-direct reads application sends aligned size which needs to be
    sent as is, otherwise o-direct writes where the file-size is not
    aligned fails.
    
     >Change-Id: I66afcba41f3484da11e9a12fe2671d2051fafc8a
     >BUG: 1342903
     >Signed-off-by: Pranith Kumar K <pkarampu>
     >Reviewed-on: http://review.gluster.org/14658
     >Tested-by: Krutika Dhananjay <kdhananj>
     >Smoke: Gluster Build System <jenkins.org>
     >CentOS-regression: Gluster Build System <jenkins.org>
     >Reviewed-by: Krutika Dhananjay <kdhananj>
     >NetBSD-regression: NetBSD Build System <jenkins.org>
    
    BUG: 1347553
    Change-Id: I1adcdaccbd5686986a057314efab9a646afa3d47
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: http://review.gluster.org/14752
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Krutika Dhananjay <kdhananj>
    Tested-by: Krutika Dhananjay <kdhananj>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 11 Vijay Bellur 2016-06-27 15:25:18 UTC
REVIEW: http://review.gluster.org/14755 (libglusterfs: Negate all but O_DIRECT flag if present on anon fds) posted (#3) for review on release-3.8 by Krutika Dhananjay (kdhananj)

Comment 12 Vijay Bellur 2016-06-28 05:57:14 UTC
COMMIT: http://review.gluster.org/14755 committed in release-3.8 by Pranith Kumar Karampuri (pkarampu) 
------
commit 37fa947743d5e56baa617fc2e72485a1907b39d2
Author: Krutika Dhananjay <kdhananj>
Date:   Tue Jun 7 17:15:56 2016 +0530

    libglusterfs: Negate all but O_DIRECT flag if present on anon fds
    
            Backport of: http://review.gluster.org/14665
    
    This is to prevent any unforeseen problems that might arise due to
    writevs and readvs being wound with @flag parameter containing
    O_TRUNC or O_APPEND especially wrt translators like sharding and ec
    where O_TRUNC write or O_APPEND write on individual shards/fragments
    is not the same as O_TRUNC write or O_APPEND write as expected by the
    application.
    
     >Change-Id: Ib0110731d6099bc888b7ada552a2abbea8e76051
     >BUG: 1342903
     >Signed-off-by: Krutika Dhananjay <kdhananj>
     >Reviewed-on: http://review.gluster.org/14735
     >Smoke: Gluster Build System <jenkins.org>
     >NetBSD-regression: NetBSD Build System <jenkins.org>
     >Reviewed-by: Pranith Kumar Karampuri <pkarampu>
     >CentOS-regression: Gluster Build System <jenkins.org>
    
    Change-Id: I7ffa4fa366f727f7e345ab0bf4c8eb009710074b
    BUG: 1347553
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: http://review.gluster.org/14755
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 13 Niels de Vos 2016-07-08 14:43:16 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.1, please open a new bug report.

glusterfs-3.8.1 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.packaging/156
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 14 Niels de Vos 2016-07-08 14:43:16 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.1, please open a new bug report.

glusterfs-3.8.1 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.packaging/156
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 15 Niels de Vos 2016-07-08 14:43:16 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.1, please open a new bug report.

glusterfs-3.8.1 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.packaging/156
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 16 Niels de Vos 2016-07-08 14:43:16 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.1, please open a new bug report.

glusterfs-3.8.1 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.packaging/156
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user


Note You need to log in before you can comment on or make changes to this bug.