Bug 761929 (GLUSTER-197)

Summary: glusterfs didn't work correctly when compiled with -D_FORTIFY_SOURCE=2 and recent gcc
Product: [Community] GlusterFS Reporter: Konstantin A. Lepikhov <lakostis>
Component: coreAssignee: Anand Avati <aavati>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: low    
Version: 2.0.6CC: amarts, chrisw, dennis, fharshav, gluster-bugs, joej, jonathansteffan, kirill, shehjart
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
spec file I built 2.0.7 with that fails
none
fail config.log none

Description Konstantin A. Lepikhov 2009-08-09 12:56:44 UTC
OS: ALTLinux Sisyphus
Tested kernels: 2.6.27-2.6.30
Tested fuse versions: 2.7.3 and 2.8.0pre3
Tested glibc version: 2.10.1
Tested GlusterFS versions: from GIT 48cbf6 to GIT 57a6cb (this is 2.0.6rc4 actually)

Affected configurations - simple directory export and afr

Steps to reproduce (simple directory export):

Server is x86_64 system, client run on i586.

server side:

volume test-storage
  type storage/posix
  option directory /mnt/test
end-volume
volume test-server
  type protocol/server
  option transport-type tcp
  subvolumes test-storage
  option auth.addr.test-storage.allow 192.168.5.*
end-volume


client side:

volume test
  type protocol/client
  option transport-type tcp
  option remote-host 192.168.5.101
  option remote-subvolume test-storage
end-volume

client side messages:

# mount -t glusterfs /etc/glusterfs/simple-client.vol /mnt/disk
# df -h
Filesystem            Size  Used Avail Use% Mounted on
...
df: `/mnt/disk': Software caused connection abort

# cat /var/log/glusterfs/mnt-disk.log

================================================================================
Version      : glusterfs 2.1.0git built on Aug  9 2009 10:26:53
TLA Revision : git://git.sv.gnu.org/gluster.git
Starting Time: 2009-08-09 16:51:38
Command line : /usr/sbin/glusterfs --log-level=NORMAL --volfile=/etc/glusterfs/simple-client.vol /mnt/disk 
PID          : 17933
System name  : Linux
Nodename     : lks.home
Kernel Release : 2.6.30-lks-pae-alt0.5
Hardware Identifier: i686

Given volfile:
+------------------------------------------------------------------------------+
  1: volume test
  2:    type protocol/client
  3:    option transport-type tcp
  4:    option remote-host 192.168.5.101
  5:    option remote-subvolume test-storage
  6: end-volume
  7: 

+------------------------------------------------------------------------------+
[2009-08-09 16:51:38] N [glusterfsd.c:1281:main] glusterfs: Successfully started
[2009-08-09 16:51:38] N [client-protocol.c:5496:client_setvolume_cbk] test: Connected to 192.168.5.101:6996, attached to remote volume 'test-storage'.
[2009-08-09 16:51:38] N [client-protocol.c:5496:client_setvolume_cbk] test: Connected to 192.168.5.101:6996, attached to remote volume 'test-storage'.
pending frames:
frame : type(1) op(STAT)

patchset: git://git.sv.gnu.org/gluster.git
signal received: 6
time of crash: 2009-08-09 16:51:43
configuration details:
argp 1
backtrace 1
bdb->cursor->get 1
db.h 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 2.1.0git
[0xb7fee400]
/lib/libc.so.6(gsignal+0x47)[0xb7e50157]
/lib/libc.so.6(abort+0x182)[0xb7e51862]
/lib/libc.so.6[0xb7e8a995]
/lib/libc.so.6(__fortify_fail+0x40)[0xb7f04f30]
/lib/libc.so.6[0xb7f03180]
/lib/libc.so.6(__strcpy_chk+0x44)[0xb7f024f4]
/usr/lib/glusterfs/2.1.0git/xlator/protocol/client.so(client_lookup+0x1bd)[0xb75ceb1d]
/usr/lib/glusterfs/2.1.0git/xlator/mount/fuse.so(fuse_root_lookup+0x179)[0xb7fe7169]
/usr/lib/glusterfs/2.1.0git/xlator/mount/fuse.so[0xb7fe7486]
/lib/libpthread.so.0[0xb7f7a480]
/lib/libc.so.6(clone+0x5e)[0xb7eef37e]
---------

Comment 1 Amar Tumballi 2009-08-09 14:38:37 UTC
Hi Konstantin,

> Version      : glusterfs 2.1.0git built on Aug  9 2009 10:26:53
> TLA Revision : git://git.sv.gnu.org/gluster.git

Please don't use 2.1.x git repository, it is undergoing lot of changes and is not ready for testing yet. Use 2.0.0git for any testing, and if you want to use it for production, we strongly advise using 2.0 stable releases from ftp repository.

(How to get 2.0.0git: http://gluster.org/docs/index.php/Development_Work_Flow#Cloning )

Please report back if the error is seen on 2.0.x branch.

Comment 2 Konstantin A. Lepikhov 2009-08-09 15:28:32 UTC
(In reply to comment #1)
> Hi Konstantin,
> 
> > Version      : glusterfs 2.1.0git built on Aug  9 2009 10:26:53
> > TLA Revision : git://git.sv.gnu.org/gluster.git
> 
> Please don't use 2.1.x git repository, it is undergoing lot of changes and is
> not ready for testing yet. Use 2.0.0git for any testing, and if you want to use
> it for production, we strongly advise using 2.0 stable releases from ftp
> repository.
> 
> (How to get 2.0.0git:
> http://gluster.org/docs/index.php/Development_Work_Flow#Cloning )
> 
> Please report back if the error is seen on 2.0.x branch.
Thanks! I'll try with release-2.0 branch. Anyway, I got the same behavior with 2.0.6rc4 (it's from stable branch isn't it?).

Comment 3 Konstantin A. Lepikhov 2009-08-09 15:52:18 UTC
checked with GIT 8dfdde57b38576270dd54f610b80f1288a16f09b from release-2.0 branch:

server side:

================================================================================
Version      : glusterfs 2.0.0git built on Aug  9 2009 18:41:08
TLA Revision : git://git.sv.gnu.org/gluster.git
Starting Time: 2009-08-09 22:46:46
Command line : /usr/sbin/glusterfsd -p /var/run/glusterfsd.pid -f /etc/glusterfs/simple-server.vol 
PID          : 4262
System name  : Linux
Nodename     : localhost.localdomain
Kernel Release : 2.6.28-lks-wks-alt1
Hardware Identifier: x86_64

Given volfile:
+------------------------------------------------------------------------------+
  1: volume test-storage
  2: type storage/posix
  3: option directory /mnt/disk
  4: end-volume
  5: 
  6: volume test-server
  7: type protocol/server
  8: option transport-type tcp
  9: subvolumes test-storage
 10: option auth.addr.test-storage.allow 192.168.5.*
 11: end-volume

+------------------------------------------------------------------------------+
[2009-08-09 22:46:46] N [glusterfsd.c:1224:main] glusterfs: Successfully started
[2009-08-09 22:47:42] N [server-protocol.c:7056:mop_setvolume] test-server: accepted client from 192.168.5.1:1023
[2009-08-09 22:47:42] N [server-protocol.c:7056:mop_setvolume] test-server: accepted client from 192.168.5.1:1022
[2009-08-09 22:48:36] N [server-protocol.c:7816:notify] test-server: 192.168.5.1:1023 disconnected
[2009-08-09 22:48:36] N [server-protocol.c:7816:notify] test-server: 192.168.5.1:1022 disconnected

client side:

================================================================================
Version      : glusterfs 2.0.0git built on Aug  9 2009 18:39:07
TLA Revision : git://git.sv.gnu.org/gluster.git
Starting Time: 2009-08-09 22:47:51
Command line : /usr/sbin/glusterfs --log-level=NORMAL --volfile=/etc/glusterfs/simple-client.vol /mnt/disk 
PID          : 21417
System name  : Linux
Nodename     : lks.home
Kernel Release : 2.6.30-lks-pae-alt0.5
Hardware Identifier: i686

Given volfile:
+------------------------------------------------------------------------------+
  1: volume test
  2:    type protocol/client
  3:    option transport-type tcp
  4:    option remote-host 192.168.5.101
  5:    option remote-subvolume test-storage
  6: end-volume
  7: 

+------------------------------------------------------------------------------+
[2009-08-09 22:47:51] N [glusterfsd.c:1224:main] glusterfs: Successfully started
[2009-08-09 22:47:51] N [client-protocol.c:5559:client_setvolume_cbk] test: Connected to 192.168.5.101:6996, attached to remote volume 'test-storage'.
[2009-08-09 22:47:51] N [client-protocol.c:5559:client_setvolume_cbk] test: Connected to 192.168.5.101:6996, attached to remote volume 'test-storage'.
pending frames:
frame : type(1) op(STAT)

patchset: git://git.sv.gnu.org/gluster.git
signal received: 6
time of crash: 2009-08-09 22:48:45
configuration details:
argp 1
backtrace 1
bdb->cursor->get 1
db.h 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 2.0.0git
[0xb7f93400]
/lib/libc.so.6(gsignal+0x47)[0xb7df9157]
/lib/libc.so.6(abort+0x182)[0xb7dfa862]
/lib/libc.so.6[0xb7e33995]
/lib/libc.so.6(__fortify_fail+0x40)[0xb7eadf30]
/lib/libc.so.6[0xb7eac180]
/lib/libc.so.6(__strcpy_chk+0x44)[0xb7eab4f4]
/usr/lib/glusterfs/2.0.0git/xlator/protocol/client.so(client_lookup+0x1ba)[0xb757883a]
/usr/lib/glusterfs/2.0.0git/xlator/mount/fuse.so(fuse_root_lookup+0x168)[0xb7f8bc38]
/usr/lib/glusterfs/2.0.0git/xlator/mount/fuse.so[0xb7f8bf36]
/lib/libpthread.so.0[0xb7f23480]
/lib/libc.so.6(clone+0x5e)[0xb7e9837e]
---------

Comment 4 Anand Avati 2009-08-09 17:15:11 UTC
The crash you have faced seems to very closely resemble the one described here -

http://osdir.com/ml/wine-patches/2009-07/msg00793.html

Is it possible to try on a system with a different glibc version?

Avati

Comment 5 Konstantin A. Lepikhov 2009-08-12 20:19:47 UTC
(In reply to comment #4)
> The crash you have faced seems to very closely resemble the one described here
> -
> 
> http://osdir.com/ml/wine-patches/2009-07/msg00793.html
> 
> Is it possible to try on a system with a different glibc version?
> 
> Avati

See comment from ALTLinux glibc maintainer -  https://bugzilla.altlinux.org/show_bug.cgi?id=20977#c7

I try to find place where this bug can by triggered and made simple patch:

diff --git a/glusterfs/xlators/protocol/client/src/client-protocol.c b/glusterfs/xlators/protocol/client/src/client-protocol.c
index 505ded7..d35235c 100644
--- a/glusterfs/xlators/protocol/client/src/client-protocol.c
+++ b/glusterfs/xlators/protocol/client/src/client-protocol.c
@@ -3154,7 +3154,7 @@ client_lookup (call_frame_t *frame, xlator_t *this, loc_t *loc,
 
        req->ino   = hton64 (ino);
        req->par   = hton64 (par);
-       strcpy (req->path, loc->path);
+       strncpy (req->path, loc->path, sizeof(req->path));
        if (baselen)
                strcpy (req->path + pathlen, loc->name);
        

After this patch df works fine but when I tried mkdir on this share I got another crash (now in server side):
+------------------------------------------------------------------------------+
[2009-08-13 02:51:33] N [glusterfsd.c:1224:main] glusterfs: Successfully started
[2009-08-13 02:53:08] N [server-protocol.c:7056:mop_setvolume] test-server: accepted client from 192.168.5.1:1023
[2009-08-13 02:53:08] N [server-protocol.c:7056:mop_setvolume] test-server: accepted client from 192.168.5.1:1022
pending frames:
frame : type(1) op(LOOKUP)

patchset: git://git.sv.gnu.org/gluster.git
signal received: 11
time of crash: 2009-08-13 02:54:09
configuration details:
argp 1
backtrace 1
bdb->cursor->get 1
db.h 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 2.0.0git
/lib64/libc.so.6[0x7fed4a390530]
/lib64/libc.so.6(strlen+0x30)[0x7fed4a3d7e10]
/lib64/libc.so.6(__strdup+0x16)[0x7fed4a3d7b46]
/usr/lib64/libglusterfs.so.0[0x7fed4a8e3dfb]
/usr/lib64/libglusterfs.so.0[0x7fed4a8e466b]
/usr/lib64/libglusterfs.so.0(inode_link+0x45)[0x7fed4a8e5525]
/usr/lib64/glusterfs/2.0.0git/xlator/protocol/server.so(server_lookup_cbk+0x465)[0x7fed49537855]
/usr/lib64/glusterfs/2.0.0git/xlator/storage/posix.so(posix_lookup+0x1fb)[0x7fed49754a1b]
/usr/lib64/glusterfs/2.0.0git/xlator/protocol/server.so(server_lookup_resume+0xee)[0x7fed4952c62e]
/usr/lib64/libglusterfs.so.0(call_resume+0x287)[0x7fed4a8e62b7]
/usr/lib64/glusterfs/2.0.0git/xlator/protocol/server.so(server_lookup+0x1c3)[0x7fed49537b83]
/usr/lib64/glusterfs/2.0.0git/xlator/protocol/server.so(protocol_server_pollin+0x7a)[0x7fed495297ea]
/usr/lib64/glusterfs/2.0.0git/xlator/protocol/server.so(notify+0x83)[0x7fed49529873]
/usr/lib64/glusterfs/2.0.0git/transport/socket.so(socket_event_handler+0xc8)[0x7fed48b1bf98]
/usr/lib64/libglusterfs.so.0[0x7fed4a8f097d]
/usr/sbin/glusterfsd(main+0x6ab)[0x40394b]
/lib64/libc.so.6(__libc_start_main+0xe6)[0x7fed4a37d7a6]
/usr/sbin/glusterfsd[0x402449]
---------

Comment 6 Konstantin A. Lepikhov 2009-08-16 19:17:52 UTC
Some information for future use:

1) This bug can be reproducible only with recent gcc containing this changes:
http://gcc.gnu.org/ml/gcc-patches/2009-06/msg00419.html

2) There are 2 variants of handling this - use memcpy instead of strcpy or compile with -D_FORTIFY_SOURCE=1 as workaround.

See example discussion about similar problem in tftpd - https://bugzilla.redhat.com/show_bug.cgi?id=515361

Hope it helps to solve this problem.

Comment 7 Konstantin A. Lepikhov 2009-08-18 16:49:58 UTC
Here is backtrace from this crash (made on Fedora rawhide system):

Core was generated by `glusterfs -f /home/lakostis/simple-client.vol --debug /mnt/test'.
Program terminated with signal 6, Aborted.
#0  0x00a71416 in __kernel_vsyscall ()
(gdb) bt
#0  0x00a71416 in __kernel_vsyscall ()
#1  0x00aa9b11 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#2  0x00aab3da in abort () at abort.c:92
#3  0x00ae7add in __libc_message (do_abort=2, fmt=0xbba07c "*** %s ***: %s terminated\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:186
#4  0x00b7570d in __fortify_fail (msg=<value optimized out>) at fortify_fail.c:32
#5  0x00b7379a in __chk_fail () at chk_fail.c:29
#6  0x00b72a74 in __strcpy_chk (dest=0xb62005c0 "", src=0x670326 "/", destlen=6) at strcpy_chk.c:61
#7  0x0083c49a in strcpy (__src=<value optimized out>, __dest=<value optimized out>) at /usr/include/bits/string3.h:106
#8  client_lookup (__src=<value optimized out>, __dest=<value optimized out>) at client-protocol.c:3157
#9  0x0066cbd8 in fuse_root_lookup (this=0x917bfd0) at fuse-bridge.c:2423
#10 0x0066ced6 in fuse_thread_proc (data=0x917bfd0) at fuse-bridge.c:2470
#11 0x00c2a9a5 in start_thread (arg=0xb6dc8b70) at pthread_create.c:297
#12 0x00b5c02e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130

Comment 8 Kirill A. Shutemov 2009-11-16 10:05:32 UTC
gluster uses a lot of tricks in protocol.h around flexible arrays. Why tricks? Let's look to gf_fop_lookup_req_t:

typedef struct {
        uint64_t ino; /* NOTE: used only in case of 'root' lookup */
        uint64_t par;
        uint32_t flags;
        uint32_t dictlen;
        char     path[0];
        char     bname[0];
        char     dict[0];
} __attribute__((packed)) gf_fop_lookup_req_t;

1. ISO C forbids zero-size arrays. You can see warning about it, if you use option -pedantic for gcc. You should use ISO C flexible arrays instead.
2. A structure can have only one flexible array and flexible array must be the last field of the structure. If you will use ISO C flexible arrays like 'char path[];', you get errors on structures like gf_fop_lookup_req_t;
3. A structure must contain something except flexible array. So structures like gf_mop_lock_req_t are invalid. You'll get error if you use ISO C flexible arrays.

gcc with _FORTIFY_SOURCE handles flexible arrays correctly if they used correctly. On tricky code like this gcc has problems.

If you'll fix issues with flexible arrays, it fixes the bug. It much better than disable _FORTIFY_SOURCE.

Comment 9 Joe Julian 2009-12-11 16:20:55 UTC
This bug still exists in 2.0.8

I discovered this bug when I upgraded to fc12. The current glibc in fc12 is 2.11. 

The rpms from kojipkgs for fc12 have this problem.

Comment 10 Harshavardhana 2009-12-13 22:11:37 UTC
Is the system under use is 32bit?. Can anyone reproduce this with a 64bit system?.

Also crash seems perfect with the latest enhancements with glibc which is 
going more strict with _FORTIFY_SOURCE=2 will result in abort as strcpy
in case of client_lookup is crossing boundaries. A fix would be to use
memcpy where ever we see a boundary crossing condition.

Comment 11 Joe Julian 2009-12-13 22:14:10 UTC
Yes, I did reproduce in x86_64

Comment 12 Harshavardhana 2009-12-14 20:51:02 UTC
This seems to be a serious issue when people will upgrade to new glibc and gcc. 

As we have extensive usage of flexible arrays all over protocol.h, looks like a good amount of work is involved.  

I am trying to get another F-12 system ready in house, so that we should adhere to glibc and gcc updates.

Comment 13 Harshavardhana 2009-12-15 00:10:48 UTC

I tried with F-12 with 

gcc - 4.4.2 
glibc - 2.11-2 
kernel - 2.6.31.6-166.fc12.x86_64

CFLAGS="-Wall -O2 -Wp,-D_FORTIFY_SOURCE=2" used for compiling glusterfs. 

But i am not able to reproduce the above mentioned issues.  

I would need to know the CFLAGS used for compiling the glusterfs in case
of ALTLinux.

Thanks

Comment 14 Joe Julian 2009-12-15 01:54:39 UTC
(In reply to comment #13)
> 
> I tried with F-12 with 
> 
> gcc - 4.4.2 
> glibc - 2.11-2 
> kernel - 2.6.31.6-166.fc12.x86_64
> 
> CFLAGS="-Wall -O2 -Wp,-D_FORTIFY_SOURCE=2" used for compiling glusterfs. 
> 
> But i am not able to reproduce the above mentioned issues.  
> 
> I would need to know the CFLAGS used for compiling the glusterfs in case
> of ALTLinux.

F12 RPMs installed that I encountered this with:
gcc-4.4.2-7.fc12.x86_64
glibc-2.11-2.x86_64
glibc-2.11-2.i686
kernel-2.6.31.6-166.fc12.x86_64

One try was with
http://kojipkgs.fedoraproject.org/packages/glusterfs/2.0.7/1.fc12/x86_64/glusterfs-client-2.0.7-1.fc12.x86_64.rpm
http://kojipkgs.fedoraproject.org/packages/glusterfs/2.0.7/1.fc12/x86_64/glusterfs-common-2.0.7-1.fc12.x86_64.rpm

When I tried from the v2.0.7 git checkpoint, no manual CFLAGS were configured. 
Packages were built using rpmbuild with the attached spec file.

I'll also attach the config.log. Maybe there's a clue in there that could help.

Comment 15 Joe Julian 2009-12-15 01:57:10 UTC
Created attachment 118 [details]
non-interactive mode (--auto)

Comment 16 Joe Julian 2009-12-15 02:06:16 UTC
Created attachment 119 [details]
trace file

Comment 17 Harshavardhana 2009-12-15 03:10:54 UTC
Joe still not useful i have the exact rpm spec file i am not able to reproduce it. I will try the versions provided by fedora project. config.log never had the 
_FORTIFY_SOURCE=2 added for CFLAGS. Then how did you reproduce this behaviour?. 

Can ALT Linux guys shed some light on this atleast provide me the necessary
CFLAGS to reproduce the behaviour?.

Comment 18 Harshavardhana 2009-12-15 03:40:16 UTC
Great i was able to reproduce this behaviour its not with _FORTIFY_SOURCE but
with "-fstack-protector"

-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic

Well i think this has been made a wee bit stricter this time around with gcc 
updates. Will see how to fix this. 

Thanks Joe for all the help.

Comment 19 Kirill A. Shutemov 2009-12-15 04:44:43 UTC
(In reply to comment #18)
> Well i think this has been made a wee bit stricter this time around with gcc 
> updates. Will see how to fix this. 

I've described the problem and how it can be fixed.

Comment 20 Anand Avati 2010-01-25 04:28:14 UTC
*** Bug 552 has been marked as a duplicate of this bug. ***

Comment 21 Dennis Schafroth 2010-02-15 18:39:18 UTC
(In reply to comment #19)
> (In reply to comment #18)
> > Well i think this has been made a wee bit stricter this time around with gcc 
> > updates. Will see how to fix this. 
> 
> I've described the problem and how it can be fixed.

I am looking for a fix for Fedora 12. Can you point to the description? 

It happens for version 2.0.9 and 3.0.2 as well.

Comment 22 Kirill A. Shutemov 2010-02-15 18:59:33 UTC
(In reply to comment #21)
> (In reply to comment #19)
> > (In reply to comment #18)
> > > Well i think this has been made a wee bit stricter this time around with gcc 
> > > updates. Will see how to fix this. 
> > 
> > I've described the problem and how it can be fixed.
> 
> I am looking for a fix for Fedora 12. Can you point to the description? 
> 
> It happens for version 2.0.9 and 3.0.2 as well.

See comment #8.

Comment 23 Jonathan Steffan 2010-04-10 19:36:09 UTC
(In reply to comment #18)
> Great i was able to reproduce this behaviour its not with _FORTIFY_SOURCE but
> with "-fstack-protector"

I have confirmed this is not an issue with stack-protector.

CFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fno-stack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic'

Trying to mount results in:

2010-04-10 16:23:17] N [client-protocol.c:5733:client_setvolume_cbk] client: Connected to 127.0.0.1:6996, attached to remote volume 'brick'.
*** buffer overflow detected ***: glusterfs terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x3bd58f75e7]
/lib64/libc.so.6[0x3bd58f5600]
/usr/lib64/glusterfs/2.0.9/xlator/protocol/client.so(client_lookup+0x1a0)[0x7f23dfaa8500]
/usr/lib64/glusterfs/2.0.9/xlator/mount/fuse.so(fuse_root_lookup+0x142)[0x7f23df892362]
/usr/lib64/glusterfs/2.0.9/xlator/mount/fuse.so(+0x9698)[0x7f23df892698]
/lib64/libpthread.so.0[0x3bd6006a3a]
/lib64/libc.so.6(clone+0x6d)[0x3bd58de67d]
======= Memory map: ========
00400000-00407000 r-xp 00000000 fd:01 92271                              /usr/sbin/glusterfsd
00607000-00608000 rw-p 00007000 fd:01 92271                              /usr/sbin/glusterfsd
01a37000-01a58000 rw-p 00000000 00:00 0                                  [heap]
36d9c00000-36d9c16000 r-xp 00000000 fd:01 9686                           /lib64/libgcc_s-4.4.3-20100127.so.1
36d9c16000-36d9e15000 ---p 00016000 fd:01 9686                           /lib64/libgcc_s-4.4.3-20100127.so.1
36d9e15000-36d9e16000 rw-p 00015000 fd:01 9686                           /lib64/libgcc_s-4.4.3-20100127.so.1
3895600000-3895624000 r-xp 00000000 fd:01 17516                          /lib64/libfuse.so.2.8.1
3895624000-3895824000 ---p 00024000 fd:01 17516                          /lib64/libfuse.so.2.8.1
3895824000-3895836000 rw-p 00024000 fd:01 17516                          /lib64/libfuse.so.2.8.1
3bd5400000-3bd541e000 r-xp 00000000 fd:01 10523                          /lib64/ld-2.11.1.so
3bd561d000-3bd561e000 r--p 0001d000 fd:01 10523                          /lib64/ld-2.11.1.so
3bd561e000-3bd561f000 rw-p 0001e000 fd:01 10523                          /lib64/ld-2.11.1.so
3bd561f000-3bd5620000 rw-p 00000000 00:00 0 
3bd5800000-3bd596f000 r-xp 00000000 fd:01 11516                          /lib64/libc-2.11.1.so
3bd596f000-3bd5b6f000 ---p 0016f000 fd:01 11516                          /lib64/libc-2.11.1.so
3bd5b6f000-3bd5b73000 r--p 0016f000 fd:01 11516                          /lib64/libc-2.11.1.so
3bd5b73000-3bd5b74000 rw-p 00173000 fd:01 11516                          /lib64/libc-2.11.1.so
3bd5b74000-3bd5b79000 rw-p 00000000 00:00 0 
3bd6000000-3bd6017000 r-xp 00000000 fd:01 12047                          /lib64/libpthread-2.11.1.so
3bd6017000-3bd6216000 ---p 00017000 fd:01 12047                          /lib64/libpthread-2.11.1.so
3bd6216000-3bd6217000 r--p 00016000 fd:01 12047                          /lib64/libpthread-2.11.1.so
3bd6217000-3bd6218000 rw-p 00017000 fd:01 12047                          /lib64/libpthread-2.11.1.so
3bd6218000-3bd621c000 rw-p 00000000 00:00 0 
3bd6400000-3bd6402000 r-xp 00000000 fd:01 13068                          /lib64/libdl-2.11.1.so
3bd6402000-3bd6602000 ---p 00002000 fd:01 13068                          /lib64/libdl-2.11.1.so
3bd6602000-3bd6603000 r--p 00002000 fd:01 13068                          /lib64/libdl-2.11.1.so
3bd6603000-3bd6604000 rw-p 00003000 fd:01 13068                          /lib64/libdl-2.11.1.so
3bd6c00000-3bd6c07000 r-xp 00000000 fd:01 12083                          /lib64/librt-2.11.1.so
3bd6c07000-3bd6e06000 ---p 00007000 fd:01 12083                          /lib64/librt-2.11.1.so
3bd6e06000-3bd6e07000 r--p 00006000 fd:01 12083                          /lib64/librt-2.11.1.so
3bd6e07000-3bd6e08000 rw-p 00007000 fd:01 12083                          /lib64/librt-2.11.1.so
7f23d8000000-7f23d8021000 rw-p 00000000 00:00 0 
7f23d8021000-7f23dc000000 ---p 00000000 00:00 0 
7f23de002000-7f23de00e000 r-xp 00000000 fd:01 56963                      /lib64/libnss_files-2.11.1.so
7f23de00e000-7f23de20d000 ---p 0000c000 fd:01 56963                      /lib64/libnss_files-2.11.1.so
7f23de20d000-7f23de20e000 r--p 0000b000 fd:01 56963                      /lib64/libnss_files-2.11.1.so
7f23de20e000-7f23de20f000 rw-p 0000c000 fd:01 56963                      /lib64/libnss_files-2.11.1.so
7f23de20f000-7f23de210000 ---p 00000000 00:00 0 
7f23de210000-7f23dec7e000 rw-p 00000000 00:00 0 
7f23dec7e000-7f23dec86000 r-xp 00000000 fd:01 1441820                    /usr/lib64/glusterfs/2.0.9/transport/socket.so.0.0.0
7f23dec86000-7f23dee86000 ---p 00008000 fd:01 1441820                    /usr/lib64/glusterfs/2.0.9/transport/socket.so.0.0.0
7f23dee86000-7f23dee88000 rw-p 00008000 fd:01 1441820                    /usr/lib64/glusterfs/2.0.9/transport/socket.so.0.0.0
7f23dee88000-7f23dee89000 ---p 00000000 00:00 0 
7f23dee89000-7f23df889000 rw-p 00000000 00:00 0 
7f23df889000-7f23df898000 r-xp 00000000 fd:01 1441856                    /usr/lib64/glusterfs/2.0.9/xlator/mount/fuse.so.0.0.0
7f23df898000-7f23dfa97000 ---p 0000f000 fd:01 1441856                    /usr/lib64/glusterfs/2.0.9/xlator/mount/fuse.so.0.0.0
7f23dfa97000-7f23dfa99000 rw-p 0000e000 fd:01 1441856                    /usr/lib64/glusterfs/2.0.9/xlator/mount/fuse.so.0.0.0
7f23dfa99000-7f23dfab6000 r-xp 00000000 fd:01 1441877                    /usr/lib64/glusterfs/2.0.9/xlator/protocol/client.so.0.0.0
7f23dfab6000-7f23dfcb6000 ---p 0001d000 fd:01 1441877                    /usr/lib64/glusterfs/2.0.9/xlator/protocol/client.so.0.0.0
7f23dfcb6000-7f23dfcb8000 rw-p 0001d000 fd:01 1441877                    /usr/lib64/glusterfs/2.0.9/xlator/protocol/client.so.0.0.0
7f23dfcb9000-7f23e0520000 rw-p 00000000 00:00 0 
7f23e0520000-7f23e0559000 r-xp 00000000 fd:01 10575                      /usr/lib64/libglusterfs.so.0.0.0
7f23e0559000-7f23e0758000 ---p 00039000 fd:01 10575                      /usr/lib64/libglusterfs.so.0.0.0
7f23e0758000-7f23e075a000 rw-p 00038000 fd:01 10575                      /usr/lib64/libglusterfs.so.0.0.0
7f23e075a000-7f23e075c000 rw-p 00000000 00:00 0 
7f23e077d000-7f23e077e000 rw-p 00000000 00:00 0 
7fffe7a63000-7fffe7a78000 rw-p 00000000 00:00 0                          [stack]
7fffe7b03000-7fffe7b04000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
pending frames:
frame : type(1) op(STAT)

patchset: v2.0.9
signal received: 6
time of crash: 2010-04-10 16:23:18
configuration details:
argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 2.0.9
/lib64/libc.so.6[0x3bd5832740]
/lib64/libc.so.6(gsignal+0x35)[0x3bd58326c5]
/lib64/libc.so.6(abort+0x175)[0x3bd5833ea5]
/lib64/libc.so.6[0x3bd586f0f3]
/lib64/libc.so.6(__fortify_fail+0x37)[0x3bd58f75e7]
/lib64/libc.so.6[0x3bd58f5600]
/usr/lib64/glusterfs/2.0.9/xlator/protocol/client.so(client_lookup+0x1a0)[0x7f23dfaa8500]
/usr/lib64/glusterfs/2.0.9/xlator/mount/fuse.so(fuse_root_lookup+0x142)[0x7f23df892362]
/usr/lib64/glusterfs/2.0.9/xlator/mount/fuse.so(+0x9698)[0x7f23df892698]
/lib64/libpthread.so.0[0x3bd6006a3a]
/lib64/libc.so.6(clone+0x6d)[0x3bd58de67d]
---------
Aborted (core dumped)

Comment 24 Jonathan Steffan 2010-04-11 18:24:50 UTC
I've fired off builds to address this for F12+:

https://admin.fedoraproject.org/updates/glusterfs-2.0.9-2.fc12
https://admin.fedoraproject.org/updates/glusterfs-2.0.9-2.fc13

These builds *should* make it into the final F13 release.

Comment 25 Amar Tumballi 2010-10-05 07:16:30 UTC
With changes in protocol (RPC/XDR), the FORTIFY_SOURCE and stack-protection issues are not any more happening with GlusterFS (In 3.1 releases).