Description of problem: `gluster volume heal volname statistics heal-count` command not showing the count of the entries to be healed. How reproducible: Always Steps to Reproduce: 1. Create 1x2 replica and fuse mount it. 2. Write data to a file from mount: `echo hello>/mnt/fuse_mnt/hello.txt` 3. Run `gluster vol heal volname info` 4. Kill brick 2 5. create 1000 files with data: `for i in `seq 1 1000`; do echo hi>/mnt/fuse_mnt/file_$i; done` 6.Run `gluster volume heal volname statistics heal-count` 7.Bring back brick 2 8.Self heal should kick in. Now again run `gluster volume heal volname statistics heal-count` Actual results: =============== Both at step 6 and step 8 for brick1: Brick 10.70.43.30:/brick/brick1 No gathered input for this brick Expected results: ================= At step 6: ---------- Brick 10.70.43.30:/brick/brick1 Number of entries: 1001------->including an entry for root dir. At step 8: ---------- Brick 10.70.43.30:/brick/brick1 Number of entries: 307----->depending on the number of pending entries to be healed. Additional info:
REVIEW: http://review.gluster.org/7897 (features/index: reset priv->to_be_healed_states) posted (#1) for review on release-3.5 by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/7897 (features/index: reset priv->to_be_healed_states) posted (#2) for review on release-3.5 by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/7897 (features/index: Don't delete current xattrop index.) posted (#3) for review on release-3.5 by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/7897 (features/index: Don't delete current xattrop index.) posted (#4) for review on release-3.5 by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/8119 (features/index: Don't delete current xattrop index.) posted (#1) for review on master by Ravishankar N (ravishankar)
COMMIT: http://review.gluster.org/8119 committed in master by Pranith Kumar Karampuri (pkarampu) ------ commit eba1fb05791f1424f68998cf2175cad1ce6098de Author: Ravishankar N <ravishankar> Date: Thu Jun 19 05:06:53 2014 +0000 features/index: Don't delete current xattrop index. Delete the base entry in indices/xattrop only when it is stale. Change-Id: I675c1510dd8293d068e31b552b0de48f50aac658 BUG: 1101647 Signed-off-by: Ravishankar N <ravishankar> Reviewed-on: http://review.gluster.org/8119 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu> Tested-by: Pranith Kumar Karampuri <pkarampu>
COMMIT: http://review.gluster.org/7897 committed in release-3.5 by Niels de Vos (ndevos) ------ commit b14ec5104f60c0f3baa744835c1214fec6099e7b Author: Ravishankar N <ravishankar> Date: Tue May 27 07:30:42 2014 +0000 features/index: Don't delete current xattrop index. Problem: `gluster v heal <volname> statistics heal-count` was not able to read the number of entries to be healed from the source brick because the base xattrop entries in indices/base_indices_holder and indices/xattrop were getting deleted after a successful heal and the code flow prevented them from creating it again. Fix: Don't delete the xattrop index unless it is stale (i.e. brick is restarted) Change-Id: Ief4eee0ddf42c4d8b711d00751be92bbbc7bbbb0 BUG: 1101647 Signed-off-by: Ravishankar N <ravishankar> Reviewed-on: http://review.gluster.org/7897 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu> Reviewed-by: Niels de Vos <ndevos>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.5.1, please reopen this bug report. glusterfs-3.5.1 has been announced on the Gluster Users mailinglist [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://supercolony.gluster.org/pipermail/gluster-users/2014-June/040723.html [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user
We still seeing this happening after upgraded to 3.5.1 [2014-06-26 03:45:31.990033] E [index.c:267:check_delete_stale_index_file] 0-gfs-index: Base index is not createdunder index/base_indices_holder [2014-06-26 03:55:32.064365] E [index.c:267:check_delete_stale_index_file] 0-gfs-index: Base index is not createdunder index/base_indices_holder [2014-06-26 03:55:32.064797] E [index.c:267:check_delete_stale_index_file] 0-gfs-index: Base index is not createdunder index/base_indices_holder
(In reply to Peter Auyeung from comment #9) > We still seeing this happening after upgraded to 3.5.1 > > [2014-06-26 03:45:31.990033] E [index.c:267:check_delete_stale_index_file] > 0-gfs-index: Base index is not createdunder index/base_indices_holder > [2014-06-26 03:55:32.064365] E [index.c:267:check_delete_stale_index_file] > 0-gfs-index: Base index is not createdunder index/base_indices_holder > [2014-06-26 03:55:32.064797] E [index.c:267:check_delete_stale_index_file] > 0-gfs-index: Base index is not createdunder index/base_indices_holder Are you sure you restarted the brick processes after upgrade?
You are right Ravishankar, just restarted the brick process and so far no more error on the brick logs. Thanks!! However: # gluster volume heal gfs Launching heal operation to perform index self heal on volume gfs has been successful Use heal info commands to check status # gluster volume heal gfs info Volume heal failed is that normal to see heal failed? -Peter
You probably need to check the glustershd.log and glfsheal-*.logs for details.