Bug 764923 (GLUSTER-3191)

Summary: rebalance follows symlinks outside of volume
Product: [Community] GlusterFS Reporter: Joe Julian <joe>
Component: glusterdAssignee: Amar Tumballi <amarts>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: medium    
Version: 3.1.5CC: amarts, gluster-bugs, vijaykumar, vraman
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: DP CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
use lstat instead of stat when walking the directory tree none

Description Amar Tumballi 2011-07-19 10:56:40 UTC
This should be mentioned in major bug in next release notes.

Thanks for pointing this out Joe. IMO very critical bug.

Regards,
Amar

Comment 1 Joe Julian 2011-07-19 11:15:39 UTC
Created attachment 559


This will prevent S_ISDIR from returning true for symlinks, preventing it from walking out of the filesystem.

Comment 2 Amar Tumballi 2011-07-19 11:20:03 UTC
(In reply to comment #2)
> Created an attachment (id=559) [details]
> use lstat instead of stat when walking the directory tree
> 
> This will prevent S_ISDIR from returning true for symlinks, preventing it from
> walking out of the filesystem.

Yes, absolutely right fix. Found that as soon as I saw the bug report. Currently changes are made, and testing the fix :-)

Comment 3 Joe Julian 2011-07-19 11:49:56 UTC
Another point to consider is that glusterd segfaulted in the snprintf on line 223. Is some boundary checking necessary?

Comment 4 Anand Avati 2011-07-19 11:59:01 UTC
CHANGE: http://review.gluster.com/26 (so the symlinks pointing to directories are not considered) merged in release-3.2 by Anand Avati (avati)

Comment 5 Amar Tumballi 2011-07-19 12:01:07 UTC
(In reply to comment #4)
> Another point to consider is that glusterd segfaulted in the snprintf on line
> 223. Is some boundary checking necessary?

We consider PATH_MAX as the boundary. So, with this fix, we should not be hitting that error anymore.

Comment 6 Joe Julian 2011-07-19 12:09:29 UTC
Please also patch against release-3.1

Comment 7 Anand Avati 2011-07-19 12:18:07 UTC
CHANGE: http://review.gluster.com/28 (so the symlinks pointing to directories are not considered) merged in release-3.1 by Anand Avati (avati)

Comment 8 Anand Avati 2011-07-19 12:51:28 UTC
CHANGE: http://review.gluster.com/30 (so the symlinks pointing to directories are not considered) merged in master by Anand Avati (avati)

Comment 9 Amar Tumballi 2011-07-19 13:28:03 UTC
Fix committed to all the branches.

Comment 10 Joe Julian 2011-07-19 13:48:09 UTC
glusterd dumped core on me last night while performing a rebalance. The core dump shows an endless loop of:

#48 0x00f38889 in gf_glusterd_rebalance_fix_layout (volinfo=0xacb07508, 
    dir=0xab70e548 "/var/lib/glusterd/mount/home/jjulian/.wine64/dosdevices/z:/cluster/3/home/jjulian/.wine64/dosdevices/f:/lib/glusterd/mount/home/jjulian/.wine64/dosdevices/z:/cluster/3/home/jjulian/.wine64/dosdevices/"...)
    at glusterd-rebalance.c:237

where /var/lib/glusterd = /etc/glusterd under your default configuration.

ls -l home/jjulian/.wine64/dosdevices/
total 12
lrwxrwxrwx. 1  502  500 10 Jun  2 16:47 c: -> ../drive_c
lrwxrwxrwx. 1  502  500  8 Jun  2 16:47 d: -> /usr/src
lrwxrwxrwx. 1 root root  6 Jun  2 16:47 e: -> /home~
lrwxrwxrwx. 1  502  500  4 Jun  2 16:47 f: -> /var
lrwxrwxrwx. 1  502  500  8 Jun  2 16:47 g: -> /var/log
lrwxrwxrwx. 1 root root 19 Jun  2 16:47 h: -> /share/home/jjulian
lrwxrwxrwx. 1  502  500  1 Jun  2 16:47 z: -> /

So it's clearly following the symlinks and getting itself into an endless loop.

Comment 11 Vijaykumar 2011-07-27 07:45:49 UTC
I followed the following steps for verification

- created a distribute volume and also created some symlinks on its mount point

- later i added two brick and rebalanced

in version 3.1.5

- According to logs , it was following the symlink and was getting hung

In recent pull,

- It is not showing any such logs, which means its neglecting the symlinks.

Comment 12 Vijaykumar 2011-07-29 06:04:08 UTC
I have tested it again in 3.1.6qa2 release with same steps. Its working fine.

Comment 13 Joe Julian 2011-10-11 22:25:56 UTC
Sorry, I forgot bugzilla takes me to the next bug after I save changes...