Description of problem: Rebooted the brick2 and started renaming the files in a brick1 which is full. The brick2 didn't came online after the reboot. Errors were seen in the brick logs. "Creation of unlink directory failed" sosreport kept at rhsqe-repo.lab.eng.blr.redhat.com://var/www/html/sosreports/<bugid> Version-Release number of selected component (if applicable): How reproducible: 100% Steps to Reproduce: 1. Create replica 3 volume and mount the volume on client using fuse. 2. Create files using for (( i=1; i <= 50; i++ )) do dd if=/dev/zero of=file$i count=1000 bs=5M status=progress done 3. After the creation is done. reboot the second brick. 4. start the renaming process of the files to test$i..n 5. When the second brick comes up it fails with below errors. [2016-05-05 14:37:45.826772] E [MSGID: 113096] [posix.c:6443:posix_create_unlink_dir] 0-arbiter-posix: Creating directory /rhs/brick1/arbiter/.glusterfs/unlink failed [No space left on device] [2016-05-05 14:37:45.826856] E [MSGID: 113096] [posix.c:6866:init] 0-arbiter-posix: Creation of unlink directory failed [2016-05-05 14:37:45.826880] E [MSGID: 101019] [xlator.c:433:xlator_init] 0-arbiter-posix: Initialization of volume 'arbiter-posix' failed, review your volfile again [2016-05-05 14:37:45.826925] E [graph.c:322:glusterfs_graph_init] 0-arbiter-posix: initializing translator failed [2016-05-05 14:37:45.826943] E [gr
REVIEW: http://review.gluster.org/15092 (posix: Do not move and recreate .glusterfs/unlink directory) posted (#1) for review on release-3.7 by Ashish Pandey (aspandey)
COMMIT: http://review.gluster.org/15092 committed in release-3.7 by Pranith Kumar Karampuri (pkarampu) ------ commit 699fcc27d109584d08824caf54aafe5fd76f9e7b Author: Ashish Pandey <aspandey> Date: Wed Jul 27 15:49:25 2016 +0530 posix: Do not move and recreate .glusterfs/unlink directory Problem: At the time of start of a volume, it is checked if .glusterfs/unlink exist or not. If it does, move it to landfill and recreate unlink directory. If a volume is mounted and we write data on it till we face ENOSPC, restart of that volume fails as it will not be able to create unlink dir. mkdir will fail with ENOSPC. This will not allow volume to restart. Solution: If .glusterfs/unlink directory exist, don't move it to landfill. Delete all the entries inside it. master - http://review.gluster.org/#/c/15030/ Change-Id: Icde3fb36012f2f01aeb119a2da042f761203c11f BUG: 1364354 Signed-off-by: Ashish Pandey <aspandey> Reviewed-on: http://review.gluster.org/15092 NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Smoke: Gluster Build System <jenkins.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.15, please open a new bug report. glusterfs-3.7.15 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] https://www.gluster.org/pipermail/gluster-devel/2016-September/050714.html [2] https://www.gluster.org/pipermail/gluster-users/