Bug 1486557

Summary:	Log entry of files skipped/failed during rebalance operation
Product:	[Community] GlusterFS	Reporter:	Nithya Balachandran <nbalacha>
Component:	distribute	Assignee:	Nithya Balachandran <nbalacha>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	high	Docs Contact:
Priority:	high
Version:	3.12	CC:	amukherj, bkunal, bugs, rhs-bugs, spalai, storage-qa-internal, tdesala
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	glusterfs-glusterfs-3.12.1	Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:	1480445	Environment:
Last Closed:	2017-09-14 07:42:56 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1480445
Bug Blocks:	1463114, 1488560, 1531936

Description Nithya Balachandran 2017-08-30 07:02:44 UTC

+++ This bug was initially created as a clone of Bug #1480445 +++

+++ This bug was initially created as a clone of Bug #1463114 +++

Description of problem:

As of now rebalance status just indicate the count of failed and skipped entries. This information should be more informative and should even give the list of files so that list can be used to manually operate and complete rebalancing.



  Good to have list of files in a separate log file. Preferable would be to have file path and not gfid. 
   We should log in such a way that it can be consumed easily even if the list is long.
   With this fix we should even create a way/script to consume the list and make the life of Admin easier. As of now doing it manually is a tough job.

So here we have 3 requirements :
   1) Get the list of files failed or skipped with error messages
   2) Automatically consume those file list to complete rebalance( This can be triggered manually)
   3) If not able to consume list, list the file with some reason of failure

--- Additional comment from Nithya Balachandran on 2017-08-11 02:44:32 EDT ---

Skipped files are now logged in the rebalance log :

2017-08-11 05:33:01.464620] I [MSGID: 109126] [dht-rebalance.c:2681:gf_defrag_migrate_single_file] 0-vol1-dht: File migration skipped for /NEWFILE-1.


Users can search for the msgid [109126] for the list of skipped files.

--- Additional comment from Worker Ant on 2017-08-11 02:50:34 EDT ---

REVIEW: https://review.gluster.org/18021 (cluster/dht:  Log files skipped by rebalance) posted (#1) for review on master by N Balachandran (nbalacha)

--- Additional comment from Worker Ant on 2017-08-30 01:56:19 EDT ---

COMMIT: https://review.gluster.org/18021 committed in master by Raghavendra G (rgowdapp) 
------
commit a4c43ba9374b8f75a48d38a032353a0c7d311a73
Author: N Balachandran <nbalacha>
Date:   Fri Aug 11 12:16:12 2017 +0530

    cluster/dht:  Log files skipped by rebalance
    
    There was no easy way to find out which files were
    skipped during a rebalance.
    Rebalance now logs a message for every skipped file
    using msgid 109126, making it easier to find
    all files that were skipped.
    
    Change-Id: I2cac7db7285e2f82354251f3ea4094827b0daf3e
    BUG: 1480445
    Signed-off-by: N Balachandran <nbalacha>
    Reviewed-on: https://review.gluster.org/18021
    Smoke: Gluster Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: hari gowtham <hari.gowtham005>
    Reviewed-by: Raghavendra G <rgowdapp>

Comment 1 Worker Ant 2017-08-30 07:07:27 UTC

REVIEW: https://review.gluster.org/18149 (cluster/dht:  Log files skipped by rebalance) posted (#1) for review on release-3.12 by N Balachandran (nbalacha)

Comment 2 Worker Ant 2017-09-07 07:09:24 UTC

COMMIT: https://review.gluster.org/18149 committed in release-3.12 by jiffin tony Thottan (jthottan) 
------
commit 30d787f05fef4c2bd7b0f4a293f683603dca28e6
Author: N Balachandran <nbalacha>
Date:   Wed Aug 30 12:36:11 2017 +0530

    cluster/dht:  Log files skipped by rebalance
    
    There was no easy way to find out which files were
    skipped during a rebalance.
    Rebalance now logs a message for every skipped file
    using msgid 109126, making it easier to find
    all files that were skipped.
    
    > BUG: 1480445
    > Signed-off-by: N Balachandran <nbalacha>
    > Reviewed-on: https://review.gluster.org/18021
    > Smoke: Gluster Build System <jenkins.org>
    > CentOS-regression: Gluster Build System <jenkins.org>
    > Reviewed-by: hari gowtham <hari.gowtham005>
    > Reviewed-by: Raghavendra G <rgowdapp>
    
    Change-Id: I2cac7db7285e2f82354251f3ea4094827b0daf3e
    BUG: 1486557
    (cherry picked from commit a4c43ba9374b8f75a48d38a032353a0c7d311a73)
    Signed-off-by: N Balachandran <nbalacha>
    Reviewed-on: https://review.gluster.org/18149
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Raghavendra G <rgowdapp>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 3 Jiffin 2017-09-14 07:42:56 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-glusterfs-3.12.1, please open a new bug report.

glusterfs-glusterfs-3.12.1 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2017-September/032441.html
[2] https://www.gluster.org/pipermail/gluster-users/