Bug 555728

Summary: mouting of gluster filesystems fails with a crash
Product: [Fedora] Fedora Reporter: Chris Jones <rollercow>
Component: glusterfsAssignee: Jonathan Steffan <jonathansteffan>
Status: CLOSED NEXTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 13CC: dennis, jonathansteffan, matthias, peljasz
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-04-17 20:16:13 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Chris Jones 2010-01-15 10:52:09 UTC
Description of problem:
Using shipped config I get a crash when trying to mount the file system

Version-Release number of selected component (if applicable):
0:2.0.9-1.fc12

How reproducible:
Everytime.

Steps to Reproduce:
1. Install glusterfs-client and glusterfs-server
2. service start glusterfsd
3. mount -t glusterfs /etc/glusterfs/glusterfs.vol /mnt/
4. ls /mnt/
  
Actual results:
ls /mnt/
ls: cannot access /mnt: Transport endpoint is not connected

Expected results:
No error

Additional info from log:
[2010-01-15 10:14:33] N [glusterfsd.c:1306:main] glusterfs: Successfully started
[2010-01-15 10:14:33] N [client-protocol.c:5733:client_setvolume_cbk] client: Connected to 127.0.0.1:6996, attached to remote volume 'brick'.
[2010-01-15 10:14:33] N [client-protocol.c:5733:client_setvolume_cbk] client: Connected to 127.0.0.1:6996, attached to remote volume 'brick'.
pending frames:
frame : type(1) op(STAT)

patchset: v2.0.9
signal received: 6
time of crash: 2010-01-15 10:14:40
configuration details:
argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 2.0.9
[0x7d9400]
[0x7d9424]
/lib/libc.so.6(gsignal+0x51)[0x18da91]
/lib/libc.so.6(abort+0x17a)[0x18f35a]
/lib/libc.so.6(+0x1b334d)[0x1cb34d]
/lib/libc.so.6(__fortify_fail+0x4d)[0x2571ed]
/lib/libc.so.6(+0x23d30a)[0x25530a]
/lib/libc.so.6(__strcpy_chk+0x44)[0x2545e4]
/usr/lib/glusterfs/2.0.9/xlator/protocol/client.so(client_lookup+0x1c0)[0xbc1730]
/usr/lib/glusterfs/2.0.9/xlator/mount/fuse.so(fuse_root_lookup+0x169)[0x511f29]
/usr/lib/glusterfs/2.0.9/xlator/mount/fuse.so(+0x92c6)[0x5122c6]
/lib/libpthread.so.0(-0xffd0554b)[0x14eab5]
/lib/libc.so.6(clone+0x5e)[0x23ddce]

Comment 1 Dennis Schafroth 2010-02-15 08:31:37 UTC
I see a similar crash on FC12, but not on FC11, which seems to use the same version of glusterfs. The glibc version differs:

FC11:
[root@bluebox0 ~]# rpm -q fuse glusterfs-client glusterfs-server glibc
fuse-2.8.1-2.fc11.i586
glusterfs-client-2.0.9-1.fc11.i586
glusterfs-server-2.0.9-1.fc11.i586
glibc-2.10.2-1.i586

FC12:
[root@applebook ~]# rpm -q fuse glusterfs-client glusterfs-server glibc
fuse-2.8.1-4.fc12.i686
glusterfs-client-2.0.9-1.fc12.i686
glusterfs-server-2.0.9-1.fc12.i686
glibc-2.11.1-1.i686

Comment 2 Dennis Schafroth 2010-02-15 21:36:50 UTC
Could be related to this: 

http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=197

a combination of gcc and glibc changes together with non-compliant C99 code in glusterfs.

Comment 3 Bug Zapper 2010-03-15 14:00:53 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 13 development cycle.
Changing version to '13'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 4 Jonathan Steffan 2010-04-10 22:37:21 UTC
https://bugzilla.altlinux.org/show_bug.cgi?id=20977 has more (and it reads like it's the correct fix for this) information also. I've tried the no-stack-protector stated by upstream, and this still fails. See: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=197#c23

I will be trying with -D_FORTIFY_SOURCE=1 shortly.

Comment 5 Jonathan Steffan 2010-04-10 23:12:38 UTC
-D_FORTIFY_SOURCE=1 prevents the crash, but I'm seeing other issues where the transport endpoint times out randomly. Still debugging that issue.

Comment 6 Jonathan Steffan 2010-04-11 20:51:32 UTC
https://admin.fedoraproject.org/updates/glusterfs-2.0.9-2.fc13

Please confirm you also see this build as working. Thanks.

Comment 7 lejeczek 2010-04-16 18:55:55 UTC
yes, 2.0.9-2.fc12 from updates-testing seems to work just fine,
glusterfs-volgen was crashing too, now seems fine
but now glusterfs-volgen binary is moved from glusterfs-common though yum says it is there, it is in devel rpm now, is it intentional?

Comment 8 Jonathan Steffan 2010-04-17 20:16:13 UTC
Yes, the move of glusterfs-volgen to the devel subpackage was intentional. I'm going to go ahead and push this build of glusterfs to stable. Thanks for the feedback.