Hide Forgot
Created attachment 452
I just noticed that 3.1.3 came out yesterday and that a bunch of the changes in it deal with rebalance bug 764101. That sounds exactly like what I'm seeing so I'll give that a try.
I'm having an issue with gluster. To highlight the problem I've attached a window showing every command executed to setup glsuter. The basic over is, I have one computer which has 3 data partitions (EXT4). \The machine is running RHEL6 with gluster 3.1.2 installed. Here is the problem - I create a volume with 1 brick - I put a couple hundred files on it - I add another brick - When I upload files, they all still fall on brick1 - I issue a rebalance and files are spread over both bricks - THE PROBLEM: When I upload files they still only land on 1 brick One thing I have noticed though is if I write the files as the root user then the properly spread over each brick. Please take a look at the attachment as that shows the whole process, including setup, creation of files, rebalance and then the failures.
Just saw the latest comment, which is same as what I wanted to convey. Please try 3.1.3 release and see if this issue is fixed.
Awesome!! Gluster 3.1.3 fixes the rebalance problem, files are now owned by the proper user after the rebalance. However there two minor things I noticed that should possibly be addressed: 1.) from the documentation it makes it sound like fix-layout is an option to rebalance but the command doesn't like to accept them as options [myuser@myserver ~]$ sudo gluster volume rebalance my_vol fix-layout start Usage: volume rebalance <VOLNAME> [fix-layout|migrate-data] {start|stop|status} If however I just issue a rebalance without the fix-layout option that works and does both the fix-layout and migrae-data. So thats really a non issue since not specifying them works but just thought that may cause some confusion. 2.) After doing the rebalance it doesn't seem to put files in the right place yet for directories that existed prior to adding the bricks. In order to get those older directories to accept new files I have to remount the gluster fileystem via the fuse client. Is this the way its suppose to work? I'm fine if that is the case that it has to be remounted to get the new layout information to the fuse clients. If that is indeed the case you may consider putting that in the 3.1.3 release notes just so others don't encounter that. Regardless of those two minor issues everything seems to work great, thanks for getting this fixed!! P.S. I'll be upgrading my operational gluster shortly and then I'll see if that fixes bug 764263.
Hi Steve, Thanks for trying out 3.1.3, and letting us know the comment about the issues. Issue 1 : fix-layout command itself not working is fixed by this patch : http://patches.gluster.com/patch/6496/ should be committed soon. Issue 2 : needing a re-mount. -> This one is not very clear to us, and glusterfs logs during this error/time will help us corner the issue. By all means, we want to prevent a case of remount, and thats the whole purpose of saying 'dynamic volume management' of glusterfs. Will keep you updated with this issue. Thanks again for feebbacks. Regards, Amar
PATCH: http://patches.gluster.com/patch/6717 in master (gluster rebalance: don't depend on the sticky bit for validating link file.)
With the patch above, the sticky bit issue should be gone, and all known permission issues are addressed regarding rebalance. DP -> It needs major fix update in Release notes.
the permission issue no more exists. hence marking it DNR
using the same test as done for bug764101, [root@centos-qa-client-1 nfs-dir]# for i in {1..20}; do touch file.$i; chown saurabh file.$i; chgrp wheel file.$i;chmod 655 file.$i; chmod +t file.$i; echo $i; done [root@centos-qa-client-1 nfs-dir]# ls -l total 80 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:48 file.1 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.10 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.11 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.12 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.13 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.14 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.15 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.16 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:50 file.17 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:50 file.18 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:50 file.19 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.2 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:50 file.20 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.3 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.4 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.5 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.6 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.7 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.8 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.9 [root@centos-qa-client-3 sbin]# ./gluster volume add-brick dist4 10.1.12.134:/mnt/ddsit4 10.1.12.135:/mnt/ddist4 Add Brick successful [root@centos-qa-client-3 sbin]# ./gluster volume rebalance dist4 start starting rebalance on volume dist4 has been successful [root@centos-qa-client-1 nfs-dir]# ls -l total 80 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.1 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.10 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.11 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.12 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.13 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.14 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.15 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.16 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.17 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.18 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.19 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.2 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.20 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.3 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.4 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.5 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.6 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.7 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:49 file.8 -rw-r-xr-t 1 saurabh wheel 0 Apr 15 03:52 file.9