Description of problem: When the user executes the following command to generate the client perf profile, # setxattr -n trusted.io-stats-dump -v /tmp/iostat.log /mnt/fuse if the client is on the same node as bricks, the bricks overwrite the profile info written by clients. Also xattr "trusted.io-stats-dump" gets set on the mount point. # getfattr -n trusted.io-stats-dump /d/backends/patchy getfattr: Removing leading '/' from absolute path names # file: d/backends/patchy trusted.io-stats-dump="/tmp/io-stats.log" Version-Release number of selected component (if applicable): master How reproducible: On any trusted storage pool, create a volume, mount(fuse) on one of the nodes, and execute `setxattr -n trusted.io-stats-dump -v <profile file> <fuse mount point>`. The content in the profile file will correspond to the brick not the mount point. and also on the mount point, the xattr "trusted.io-stats-dump" gets set Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
REVIEW: http://review.gluster.org/13872 (io-stats: Fix overwriting of client profile by the bricks) posted (#1) for review on master by Poornima G (pgurusid)
REVIEW: http://review.gluster.org/13872 (io-stats: Fix overwriting of client profile by the bricks) posted (#2) for review on master by Poornima G (pgurusid)
REVIEW: http://review.gluster.org/13872 (io-stats: Fix overwriting of client profile by the bricks) posted (#3) for review on master by Poornima G (pgurusid)
REVIEW: http://review.gluster.org/13872 (io-stats: Fix overwriting of client profile by the bricks) posted (#4) for review on master by Poornima G (pgurusid)
REVIEW: http://review.gluster.org/13872 (io-stats: Fix overwriting of client profile by the bricks) posted (#5) for review on master by Poornima G (pgurusid)
COMMIT: http://review.gluster.org/13872 committed in master by Jeff Darcy (jdarcy) ------ commit 0facb11220aea20a6573b656785922219c9650cf Author: Poornima G <pgurusid> Date: Thu Mar 31 06:29:29 2016 -0400 io-stats: Fix overwriting of client profile by the bricks Issue: When the user executes the following command to generate the client perf profile, if the client is on the same node as bricks, the bricks overwrite the profile info written by clients. Also xattr "trusted.io-stats-dump" gets set on the mount point. setxattr -n trusted.io-stats-dump -v /tmp/iostat.log /mnt/fuse Fix: Unwind from setxattr, when xattr is 'io-stats-dump' Change-Id: Iba0e5df2f25f4ba3b1399ac176a3f8a916ff372e BUG: 1322825 Signed-off-by: Poornima G <pgurusid> Reviewed-on: http://review.gluster.org/13872 NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.com> Reviewed-by: Ravishankar N <ravishankar> Smoke: Gluster Build System <jenkins.com> Reviewed-by: Jeff Darcy <jdarcy>
Reopening this based on observations and comments here, http://www.gluster.org/pipermail/gluster-devel/2016-May/049657.html In summary: - We need to trigger io-stat-dump for all instances in the graph when this attr is set - We need to write the output to different files, so that they do not overwrite each others data - We need to prevent this xattr from being set on the path that is used to trigger the io-stat-dump information Following this up with a patch for the above.
REVIEW: http://review.gluster.org/14552 (io-stats: Fix io-stat dump to dump at all levels) posted (#1) for review on master by Shyamsundar Ranganathan (srangana)
REVIEW: http://review.gluster.org/14552 (io-stats: Fix io-stat dump to dump at all levels) posted (#2) for review on master by Shyamsundar Ranganathan (srangana)
REVIEW: http://review.gluster.org/14552 (io-stats: Fix io-stat dump to dump at all levels) posted (#3) for review on master by Shyamsundar Ranganathan (srangana)
COMMIT: http://review.gluster.org/14552 committed in master by Jeff Darcy (jdarcy) ------ commit c04df79dc453ef5cb7b3a0ca8ba14598da6189ac Author: Shyam <srangana> Date: Fri May 27 14:00:40 2016 -0400 io-stats: Fix io-stat dump to dump at all levels Previous commit to fix the bug, where io-stat-dump was overwriting the dump file when the client and a brick was on the same host, failed to consider the existing behaviour where io-stats can help generate closely correlated set of stats across clients and bricks, by triggering the dump using the same command. This was introduced in commit: 0facb11220aea20a6573b656785922219c9650cf Further, by limiting the first io-stat to unwind the dump request, there is no way to trigger other io-stat xlators in the stack to dump their stat information. This bug hence is being fixed by this commit keeping the following in mind, - We need to trigger io-stat-dump for all instances in the graph when this attr is set - We need to write the output to different files, so that they do not overwrite each others data - We need to prevent this xattr from being set on the path that is used to trigger the io-stat-dump information Change-Id: I31ec380f0d85e10313a9d7b977da0e1ec74638a6 BUG: 1322825 Signed-off-by: Shyam <srangana> Reviewed-on: http://review.gluster.org/14552 Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> Reviewed-by: Ravishankar N <ravishankar> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Jeff Darcy <jdarcy>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report. glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/ [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user