Bug 1466188

Summary:	Add scripts to analyze quota xattr in backend and identify accounting issues
Product:	[Community] GlusterFS	Reporter:	Sanoj Unnikrishnan <sunnikri>
Component:	quota	Assignee:	Sanoj Unnikrishnan <sunnikri>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	mainline	CC:	bugs
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.12.0	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-09-05 17:35:13 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Sanoj Unnikrishnan 2017-06-29 08:42:00 UTC

This bug is for introducing scripts in order to identify Accounting issues in marker as discussed originally in http://nongnu.13855.n7.nabble.com/Scripts-to-help-RCA-quota-accounting-issues-td222941.html,

1. Script that does a du -h on the directory tree and a quota list on the hierarchy and interleaves the two. We can then identify which directory(s) in FS has caused the accounting to go bad and try to investigate what fops happened on those directories. Also, currently for accounting issues we set dirty xattr on the FS tree as a whole. But, if we knew the origin of the accounting mismatch then we can set dirty xattr on a few directories on the brick and let the correction propagate by itself.

2. Another issue is that since the xattr are in hexadecimal it becomes tedious to analyze. we need a script that can recursively collect xattr values and dump it human readable form for quick analysis

Comment 1 Worker Ant 2017-06-29 12:28:56 UTC

REVIEW: https://review.gluster.org/17649 (Scripts to identify quota accouting issues) posted (#1) for review on master by sanoj-unnikrishnan (sunnikri)

Comment 2 Worker Ant 2017-06-30 08:39:08 UTC

COMMIT: https://review.gluster.org/17649 committed in master by Raghavendra G (rgowdapp) 
------
commit 50d72373fadf1174918e754e31bae3806aa4c214
Author: Sanoj Unnikrishnan <sunnikri>
Date:   Thu Jun 29 15:39:35 2017 +0530

    Scripts to identify quota accouting issues
    
    The patch contains 2 scripts:
    
    log_accounting.sh does a du -h on the FS hierarchy and a quota list
    on the hierarchy and interleaves the two output. We can then identify
    which directory(s) in FS has caused the accounting to go bad and try
    to investigate what fops happened on those directories. We can also
    limit the set of directories on which we need to set dirty xattr to
    correct accounting.
    xattr_analysis.py reads all the xattr of a brick and dumps it a human
    readable form to ease debugging.
    
    Change-Id: I2155561d10c08dc3ab9e8b09dbd258f0592b4d33
    BUG: 1466188
    Signed-off-by: Sanoj Unnikrishnan <sunnikri>
    Reviewed-on: https://review.gluster.org/17649
    CentOS-regression: Gluster Build System <jenkins.org>
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Raghavendra G <rgowdapp>

Comment 3 Shyamsundar 2017-09-05 17:35:13 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.12.0, please open a new bug report.

glusterfs-3.12.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-September/000082.html
[2] https://www.gluster.org/pipermail/gluster-users/