This service will be undergoing maintenance at 00:00 UTC, 2016-08-01. It is expected to last about 1 hours

Bug 764923 (GLUSTER-3191)

Summary: rebalance follows symlinks outside of volume
Product: [Community] GlusterFS Reporter: Joe Julian <joe>
Component: glusterdAssignee: Amar Tumballi <amarts>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: medium    
Version: 3.1.5CC: amarts, gluster-bugs, vijaykumar, vraman
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: DP CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Attachments:
Description Flags
use lstat instead of stat when walking the directory tree none

Description Amar Tumballi 2011-07-19 06:56:40 EDT
This should be mentioned in major bug in next release notes.

Thanks for pointing this out Joe. IMO very critical bug.

Regards,
Amar
Comment 1 Joe Julian 2011-07-19 07:15:39 EDT
Created attachment 559


This will prevent S_ISDIR from returning true for symlinks, preventing it from walking out of the filesystem.
Comment 2 Amar Tumballi 2011-07-19 07:20:03 EDT
(In reply to comment #2)
> Created an attachment (id=559) [details]
> use lstat instead of stat when walking the directory tree
> 
> This will prevent S_ISDIR from returning true for symlinks, preventing it from
> walking out of the filesystem.

Yes, absolutely right fix. Found that as soon as I saw the bug report. Currently changes are made, and testing the fix :-)
Comment 3 Joe Julian 2011-07-19 07:49:56 EDT
Another point to consider is that glusterd segfaulted in the snprintf on line 223. Is some boundary checking necessary?
Comment 4 Anand Avati 2011-07-19 07:59:01 EDT
CHANGE: http://review.gluster.com/26 (so the symlinks pointing to directories are not considered) merged in release-3.2 by Anand Avati (avati@gluster.com)
Comment 5 Amar Tumballi 2011-07-19 08:01:07 EDT
(In reply to comment #4)
> Another point to consider is that glusterd segfaulted in the snprintf on line
> 223. Is some boundary checking necessary?

We consider PATH_MAX as the boundary. So, with this fix, we should not be hitting that error anymore.
Comment 6 Joe Julian 2011-07-19 08:09:29 EDT
Please also patch against release-3.1
Comment 7 Anand Avati 2011-07-19 08:18:07 EDT
CHANGE: http://review.gluster.com/28 (so the symlinks pointing to directories are not considered) merged in release-3.1 by Anand Avati (avati@gluster.com)
Comment 8 Anand Avati 2011-07-19 08:51:28 EDT
CHANGE: http://review.gluster.com/30 (so the symlinks pointing to directories are not considered) merged in master by Anand Avati (avati@gluster.com)
Comment 9 Amar Tumballi 2011-07-19 09:28:03 EDT
Fix committed to all the branches.
Comment 10 Joe Julian 2011-07-19 09:48:09 EDT
glusterd dumped core on me last night while performing a rebalance. The core dump shows an endless loop of:

#48 0x00f38889 in gf_glusterd_rebalance_fix_layout (volinfo=0xacb07508, 
    dir=0xab70e548 "/var/lib/glusterd/mount/home/jjulian/.wine64/dosdevices/z:/cluster/3/home/jjulian/.wine64/dosdevices/f:/lib/glusterd/mount/home/jjulian/.wine64/dosdevices/z:/cluster/3/home/jjulian/.wine64/dosdevices/"...)
    at glusterd-rebalance.c:237

where /var/lib/glusterd = /etc/glusterd under your default configuration.

ls -l home/jjulian/.wine64/dosdevices/
total 12
lrwxrwxrwx. 1  502  500 10 Jun  2 16:47 c: -> ../drive_c
lrwxrwxrwx. 1  502  500  8 Jun  2 16:47 d: -> /usr/src
lrwxrwxrwx. 1 root root  6 Jun  2 16:47 e: -> /home~
lrwxrwxrwx. 1  502  500  4 Jun  2 16:47 f: -> /var
lrwxrwxrwx. 1  502  500  8 Jun  2 16:47 g: -> /var/log
lrwxrwxrwx. 1 root root 19 Jun  2 16:47 h: -> /share/home/jjulian
lrwxrwxrwx. 1  502  500  1 Jun  2 16:47 z: -> /

So it's clearly following the symlinks and getting itself into an endless loop.
Comment 11 Vijaykumar 2011-07-27 03:45:49 EDT
I followed the following steps for verification

- created a distribute volume and also created some symlinks on its mount point

- later i added two brick and rebalanced

in version 3.1.5

- According to logs , it was following the symlink and was getting hung

In recent pull,

- It is not showing any such logs, which means its neglecting the symlinks.
Comment 12 Vijaykumar 2011-07-29 02:04:08 EDT
I have tested it again in 3.1.6qa2 release with same steps. Its working fine.
Comment 13 Joe Julian 2011-10-11 18:25:56 EDT
Sorry, I forgot bugzilla takes me to the next bug after I save changes...