+++ This bug was initially created as a clone of Bug #1207603 +++ Description of problem: Metadata read fops like stat, fstat, lookup, readdirp etc return the file size and block count in their response. This means that for shard'ed files, it is necessary to look up each of the shards associated with the file and aggregate their block count and block_size before returning it to the translators above. But that would involve lot of network fops, increasing the latency of these fops. To get around this problem, the size and block count can be kept up to data in the form of an xattr on the first shard with each inode write operation (like writev, truncate, ftruncate, zerofill, discard, fallocate etc). Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: --- Additional comment from Anand Avati on 2015-03-31 06:11:22 EDT --- REVIEW: http://review.gluster.org/10066 (features/shard: Introduce file size xattr) posted (#1) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-01 07:50:38 EDT --- REVIEW: http://review.gluster.org/10097 (features/shard: Introduce file size xattr) posted (#1) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-01 07:50:41 EDT --- REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block count in metadata read ops) posted (#1) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-07 02:40:51 EDT --- REVIEW: http://review.gluster.org/10097 (features/shard: Introduce file size xattr) posted (#2) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-07 06:35:52 EDT --- REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block count in metadata read ops) posted (#2) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-07 06:35:55 EDT --- REVIEW: http://review.gluster.org/10097 (features/shard: Introduce file size xattr) posted (#3) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-07 11:44:13 EDT --- REVIEW: http://review.gluster.org/10097 (features/shard: Introduce file size xattr) posted (#4) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-07 21:08:00 EDT --- REVIEW: http://review.gluster.org/10158 (storage/posix: Introduce xattr_fill capability in posix_stat) posted (#1) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-08 02:11:26 EDT --- COMMIT: http://review.gluster.org/10158 committed in master by Pranith Kumar Karampuri (pkarampu) ------ commit 6a465e6e7e46940e00a387089cc995464975b53d Author: Krutika Dhananjay <kdhananj> Date: Tue Apr 7 21:45:49 2015 +0530 storage/posix: Introduce xattr_fill capability in posix_stat Change-Id: I2b6503ad9333f445ebdcd9fa660da20b861b985f BUG: 1207603 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/10158 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu> Tested-by: Pranith Kumar Karampuri <pkarampu> --- Additional comment from Anand Avati on 2015-04-08 11:24:42 EDT --- COMMIT: http://review.gluster.org/10097 committed in master by Vijay Bellur (vbellur) ------ commit 4be65bb376e2fffd7175f579724aae4c5718d57c Author: Krutika Dhananjay <kdhananj> Date: Wed Apr 1 16:00:05 2015 +0530 features/shard: Introduce file size xattr With each inode write FOP, the size and block count of the file will be updated within the xattr. There are two 64 byte fields that are intentionally left blank for now for future use when consistency guarantee is introduced later in sharding. Change-Id: I40a2e700150c1f199a6bf87909f063c84ab7bb43 BUG: 1207603 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/10097 Reviewed-by: Pranith Kumar Karampuri <pkarampu> Tested-by: Pranith Kumar Karampuri <pkarampu> Tested-by: Gluster Build System <jenkins.com> --- Additional comment from Anand Avati on 2015-04-09 10:38:09 EDT --- REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block count in metadata read ops) posted (#3) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-10 06:25:18 EDT --- REVIEW: http://review.gluster.org/10180 (storage/posix: Introduce xattr-fill on fds) posted (#2) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-13 02:35:28 EDT --- COMMIT: http://review.gluster.org/10180 committed in master by Pranith Kumar Karampuri (pkarampu) ------ commit f5220016c6525a6e166b83bcc24a036c5db0498b Author: Krutika Dhananjay <kdhananj> Date: Wed Apr 8 14:48:32 2015 +0530 storage/posix: Introduce xattr-fill on fds ... with some of the code borrowed from http://review.gluster.org/#/c/3904/ Change-Id: I4901ef14d6f843d8d69f102d43d21b60ba298092 BUG: 1207603 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/10180 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu> --- Additional comment from Anand Avati on 2015-04-13 05:39:22 EDT --- REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block count in metadata read ops) posted (#4) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-15 04:27:22 EDT --- REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block count in metadata read ops) posted (#5) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-15 05:49:26 EDT --- REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block count in metadata read ops) posted (#6) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-17 05:10:32 EDT --- REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block count in metadata read ops) posted (#7) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Anand Avati on 2015-04-22 01:38:25 EDT --- COMMIT: http://review.gluster.org/10098 committed in master by Vijay Bellur (vbellur) ------ commit f1bdc3f186576107fda4f3665c809c791b4cbe95 Author: Krutika Dhananjay <kdhananj> Date: Fri Mar 27 16:03:20 2015 +0530 features/shard: Consume size and block count in metadata read ops Metadata read fops like lookup, stat etc will now fetch the xattr that holds the size and block count information, extract the size and block count fields and set them in respective stbuf before unwinding the resultant iatt to the parent xlator. Change-Id: I881be8955092fa6b75f8b0e4f3deb01344cb638e BUG: 1207603 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/10098 Reviewed-by: Pranith Kumar Karampuri <pkarampu> Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Vijay Bellur <vbellur>
REVIEW: http://review.gluster.org/10331 (features/shard: Consume size and block count in metadata read ops) posted (#1) for review on release-3.7 by Krutika Dhananjay (kdhananj)
COMMIT: http://review.gluster.org/10331 committed in release-3.7 by Vijay Bellur (vbellur) ------ commit af9dfc5080fdabec41ba74a104208c3e0dd13430 Author: Krutika Dhananjay <kdhananj> Date: Fri Mar 27 16:03:20 2015 +0530 features/shard: Consume size and block count in metadata read ops Backport of : http://review.gluster.org/10098 Metadata read fops like lookup, stat etc will now fetch the xattr that holds the size and block count information, extract the size and block count fields and set them in respective stbuf before unwinding the resultant iatt to the parent xlator. Change-Id: If7d2c4af886f8d70dc69d7cb09f1f66be391f198 BUG: 1214248 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/10331 Reviewed-by: Pranith Kumar Karampuri <pkarampu> Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Vijay Bellur <vbellur>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report. glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939 [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user