Bug 1096047
| Summary: | [barrier] ls --color gets blocked on the fuse mount, with O_SYNC writes unless barrier was disabled | |||
|---|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | SATHEESARAN <sasundar> | |
| Component: | core | Assignee: | krishnan parthasarathi <kparthas> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | SATHEESARAN <sasundar> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | mainline | CC: | amukherj, bugs, gluster-bugs, kparthas, nsathyan | |
| Target Milestone: | --- | |||
| Target Release: | --- | |||
| Hardware: | x86_64 | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | glusterfs-3.6.0beta1 | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1101464 (view as bug list) | Environment: | ||
| Last Closed: | 2014-11-11 08:32:14 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1101464 | |||
REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above) posted (#1) for review on master by Krishnan Parthasarathi (kparthas) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#1) for review on master by Krishnan Parthasarathi (kparthas) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above.) posted (#2) for review on master by Krishnan Parthasarathi (kparthas) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#2) for review on master by Krishnan Parthasarathi (kparthas) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above.) posted (#3) for review on master by Krishnan Parthasarathi (kparthas) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#3) for review on master by Krishnan Parthasarathi (kparthas) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above) posted (#4) for review on master by Krishnan Parthasarathi (kparthas) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#4) for review on master by Krishnan Parthasarathi (kparthas) Root cause analysis: -------------------- Problem: In a GlusterFS volume, when barrier is enabled, accessed via FUSE mount point, "ls --color" hangs when O_SYNC writes are performed on the same mount concurrently. Cause: In the FUSE kernel module, readdirplus attempts an attribute update on a file on which write is under-way, with either the new file size or the new mtime (possibly caused by the same write, or otherwise) and the page invalidation is waiting for the on-going write to complete before returning. The on-going write would not complete until barrier is disabled. This causes "ls --color" to hang. Fix: To prevent readdirp from being blocked on the page invalidation due to the barrier'd O_SYNC write, we set direct-io-mode flag for the fd which prevents page caching in the FUSE kernel module. REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above.) posted (#5) for review on master by Krishnan Parthasarathi (kparthas) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#5) for review on master by Krishnan Parthasarathi (kparthas) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above.) posted (#6) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#6) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above.) posted (#7) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#7) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_open() should set fd->flgas before syncop_open()) posted (#1) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above.) posted (#8) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_open() should set fd->flgas before syncop_open()) posted (#2) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_open() should set fd->flags before syncop_open()) posted (#3) for review on master by Krishnan Parthasarathi (kparthas) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above) posted (#9) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#8) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_open() should set fd->flags before syncop_open()) posted (#4) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_open() should set fd->flgas before syncop_open()) posted (#5) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_open() should set fd->flags before syncop_open()) posted (#6) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above.) posted (#10) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_open() should set fd->flags before syncop_open()) posted (#7) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above) posted (#11) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#9) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_open() should set fd->flags before syncop_open()) posted (#8) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above.) posted (#12) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above) posted (#13) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#10) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_open() should set fd->flags before syncop_open()) posted (#9) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7872 (dht: pass xdata to xlators above.) posted (#14) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_h_creat()/glfs_open()/glfs_h_open() should set fd->flags before syncop_open()) posted (#10) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#11) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#12) for review on master by Atin Mukherjee (amukherj) REVIEW: http://review.gluster.org/7873 (barrier: enable FUSE direct-io-mode for barriered fds) posted (#13) for review on master by Atin Mukherjee (amukherj) COMMIT: http://review.gluster.org/7872 committed in master by Vijay Bellur (vbellur) ------ commit 187a7a926b6b1b57060046a030a5d9a44000fc4b Author: Krishnan Parthasarathi <kparthas> Date: Mon May 26 17:18:17 2014 +0530 dht: pass xdata to xlators above. Change-Id: I96e9feb88443fcd7da40c33c0e8c4e2645b1fcf3 BUG: 1096047 Signed-off-by: Krishnan Parthasarathi <kparthas> Reviewed-on: http://review.gluster.org/7872 Reviewed-by: Shyamsundar Ranganathan <srangana> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Vijay Bellur <vbellur> REVIEW: http://review.gluster.org/7999 (libgfapi : glfs_create()/glfs_h_creat()/glfs_open()/glfs_h_open() should set fd->flags before syncop_open()) posted (#11) for review on master by Atin Mukherjee (amukherj) COMMIT: http://review.gluster.org/7999 committed in master by Anand Avati (avati) ------ commit 67a6f402d4a4af2e7d9b8d80b25c94f1a08ef029 Author: Atin Mukherjee <amukherj> Date: Fri Jun 6 12:51:57 2014 +0530 libgfapi : glfs_create()/glfs_h_creat()/glfs_open()/glfs_h_open() should set fd->flags before syncop_open() glfs_create() and glfs_open() do not set fd->flags before calling syncop_open(). This patch addresses this problem and ensure the flags are set in fd before invoking syncop_open() Change-Id: I9ef3243b1de610e1dd1a3e37b66fc2f763a865f9 BUG: 1096047 Signed-off-by: Atin Mukherjee <amukherj> Reviewed-on: http://review.gluster.org/7999 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Krishnan Parthasarathi <kparthas> Reviewed-by: Humble Devassy Chirammal <humble.devassy> Reviewed-by: Anand Avati <avati> A beta release for GlusterFS 3.6.0 has been released. Please verify if the release solves this bug report for you. In case the glusterfs-3.6.0beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED. Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution. [1] http://supercolony.gluster.org/pipermail/gluster-users/2014-September/018836.html [2] http://supercolony.gluster.org/pipermail/gluster-users/ This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report. glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html [2] http://supercolony.gluster.org/mailman/listinfo/gluster-users |
Description of problem: ======================= "ls" command on the fuse mount gets blocked when O_SYNC writes was in progress on the same mount, when barrier was enabled Version-Release number of selected component (if applicable): ============================================================== mainline How reproducible: ================= Always Steps to Reproduce: =================== 1. Enable barrier 2. Start O_SYNC writes on the fuse mount 3. perform "ls" on the same mount Actual results: =============== "ls" command got blocked till the barrier was disabled Expected results: ================= "ls" command should not get blocked Additional info: ================= While repeating the test with other barrier class fops {unlink, rmdir, removexattr, fremovexattr, fsync, rename, O_SYNC write }, I see that "ls" command got blocked except for truncate and ftruncate calls, till barrier was enabled.