Description of problem: In a master/slave geo-replication setup, the slave node fills up the disk inodes. The issue happens because the ".processing" directory is not cleared up as it is on the master. This means that on the slave nodes that directory keeps all the CHANGELOG_* files. Version-Release number of selected component (if applicable): glusterfs-geo-replication-3.6.0-0.5.beta3.el6.x86_64 How reproducible: 2 local nodes, geo-replicating to 1 or more remote sites. Actual results: The slave server saves the CHANGELOG file in: /var/lib/misc/glusterfsd/<VOLNAME>/<REMOTESITE>/<ID>/.processing/ But it does not delete them after the changelog is applied. Expected results: The .processing directory gets cleared up as it is on the master ( CHANGELOG file gets moved in the .processed directory )
This is a known issue, but also could not find a bug for this yet. Venky is aware of the problem: - http://supercolony.gluster.org/pipermail/gluster-devel/2014-November/042887.html
Yep, that was me asking about it on the mailing list. He asked me to raise a bug report for it.
Aravinda, passive replica periodically brings it's local stime to cluster stime. Could we also purge accumulated changelogs too. Also, we'd need to purge processed changelogs too. A good idea would be to pack them into an archive periodically. Any thoughts?
REVIEW: http://review.gluster.org/9453 (geo-rep: [WIP] Archive Changelogs and avoid generating empty XSync changelogs) posted (#1) for review on master by Aravinda VK (avishwan)
REVIEW: http://review.gluster.org/9453 (geo-rep: Archive Changelogs and avoid generating empty XSync changelogs) posted (#2) for review on master by Aravinda VK (avishwan)
REVIEW: http://review.gluster.org/9453 (geo-rep: Archive Changelogs and avoid generating empty XSync changelogs) posted (#3) for review on master by Aravinda VK (avishwan)
COMMIT: http://review.gluster.org/9453 committed in master by Venky Shankar (vshankar) ------ commit 1226083d0ff5fcff21abd16b314effeee49ae770 Author: Aravinda VK <avishwan> Date: Thu Jan 15 15:19:50 2015 +0530 geo-rep: Archive Changelogs and avoid generating empty XSync changelogs With this patch, - Hybrid Crawl will not generate empty Changelogs - Archives Changelogs when processed(Hybrid(XSync), History, and Changelog Crawl - Passive worker cleans up its processing directory BUG: 1169331 Change-Id: I1383ffaed261cdf50da91b14260b4d43177657d1 Signed-off-by: Aravinda VK <avishwan> Reviewed-on: http://review.gluster.org/9453 Reviewed-by: Venky Shankar <vshankar> Tested-by: Venky Shankar <vshankar>
*** Bug 1188968 has been marked as a duplicate of this bug. ***
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report. glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939 [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user