http://review.gluster.org/6369/ introduced an automatic cleanup of quota xattrs. If the user disables quota and immediately re-enables quota, inconsistent things seem to happen, including differences between df and du and brick crashes. I suspect this is due to the cleanup script having not yet completed. Since the user was unaware of the script's existence, he did not anticipate any problems. At the very least, a notice should be displayed from the CLI when quota is disabled telling the user what to look for to determine if the cleanup has finished.
One solution we can think of is to maintain version for quota-xattrs, with this we can enable/disable quota, while the cleanup script is still running.
Hi Joe, This issue has been fixed in 3.7 with versioning the xattrs.
As per comment# 2, problem of quota accounting inconsistency with disabling and enabling quota is solved with quota-xattr versioning feature introduced in 3.7. Also there is no plan for back-porting this feature to 3.5, so closing this bug