Bug 764584 - (GLUSTER-2852) Glusterfsd crashes on AMI
Glusterfsd crashes on AMI
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: core (Show other bugs)
mainline
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Raghavendra G
:
: GLUSTER-2853 GLUSTER-3014 764759 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2011-04-27 08:50 EDT by Saurabh
Modified: 2011-06-15 04:13 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Saurabh 2011-04-27 05:54:06 EDT
[root@ip-10-85-154-57 ~]# glusterfs --version
glusterfs 3.2.0 built on Apr 26 2011 06:28:48
Repository revision: v3.2.0
Copyright (c) 2006-2010 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU Affero General Public License.



[root@ip-10-85-154-57 ~]# gluster volume info

Volume Name: dist-rep
Type: Distributed-Replicate
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: ip-10-85-154-57.ec2.internal:/mnt/dist-rep
Brick2: ip-10-117-33-233.ec2.internal:/mnt/dist-rep
Brick3: ip-10-82-247-35.ec2.internal:/mnt/dist-rep
Brick4: ip-10-83-105-82.ec2.internal:/mnt/dist-rep



[2011-04-27 06:02:07.586280] E [access-control.c:1842:ac_setattr_stat_cbk] 0-dist-rep-access-control: setattr failed with error: Operation not permitted
pending frames:

patchset: v3.2.0
signal received: 11
time of crash: 2011-04-27 06:02:07
configuration details:
argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 3.2.0
/lib64/libc.so.6[0x2aaaab9b62d0]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/features/marker.so(marker_setattr_cbk+0x139)[0x2aaaad983e79]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/performance/io-threads.so(iot_setattr_cbk+0x88)[0x2aaaad773718]
/opt/glusterfs/3.2.0/lib64/libglusterfs.so.0(default_setattr_cbk+0x88)[0x2aaaaaef1f28]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/features/access-control.so(ac_setattr_stat_cbk+0x137)[0x2aaaad354ec7]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/storage/posix.so(posix_stat+0x14d)[0x2aaaad13cbfd]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/features/access-control.so(ac_setattr+0x14c)[0x2aaaad350a7c]
/opt/glusterfs/3.2.0/lib64/libglusterfs.so.0(default_setattr+0xe9)[0x2aaaaaeeb659]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/performance/io-threads.so(iot_setattr_wrapper+0xe9)[0x2aaaad776749]
/opt/glusterfs/3.2.0/lib64/libglusterfs.so.0(call_resume+0xd81)[0x2aaaaaefc191]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/performance/io-threads.so(iot_worker+0x119)[0x2aaaad77a229]
/lib64/libpthread.so.0[0x2aaaab770617]
/lib64/libc.so.6(clone+0x6d)[0x2aaaaba59c2d]
Comment 1 Saurabh 2011-04-27 08:50:39 EDT

    
Comment 2 Anand Avati 2011-04-27 23:16:57 EDT
*** Bug 2853 has been marked as a duplicate of this bug. ***
Comment 3 Anand Avati 2011-04-29 04:24:34 EDT
PATCH: http://patches.gluster.com/patch/7064 in master (marker: fix NULL deref in logging)
Comment 4 Saurabh 2011-05-03 09:10:02 EDT
it happened again on tokyo region,


[2011-05-03 11:02:48.686121] E [access-control.c:1842:ac_setattr_stat_cbk] 0-dist-rep-access-control: setattr failed with error: Operation not permitted
pending frames:

patchset: v3.2.0
signal received: 11
time of crash: 2011-05-03 11:02:48
configuration details:
argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 3.2.0
/lib64/libc.so.6[0x2aaaab9b62d0]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/features/marker.so(marker_setattr_cbk+0x139)[0x2aaaad983e79]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/performance/io-threads.so(iot_setattr_cbk+0x88)[0x2aaaad773718]
/opt/glusterfs/3.2.0/lib64/libglusterfs.so.0(default_setattr_cbk+0x88)[0x2aaaaaef1f28]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/features/access-control.so(ac_setattr_stat_cbk+0x137)[0x2aaaad354ec7]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/storage/posix.so(posix_stat+0x14d)[0x2aaaad13cbfd]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/features/access-control.so(ac_setattr+0x14c)[0x2aaaad350a7c]
/opt/glusterfs/3.2.0/lib64/libglusterfs.so.0(default_setattr+0xe9)[0x2aaaaaeeb659]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/performance/io-threads.so(iot_setattr_wrapper+0xe9)[0x2aaaad776749]
/opt/glusterfs/3.2.0/lib64/libglusterfs.so.0(call_resume+0xd81)[0x2aaaaaefc191]
/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/performance/io-threads.so(iot_worker+0x119)[0x2aaaad77a229]
/lib64/libpthread.so.0[0x2aaaab770617]
/lib64/libc.so.6(clone+0x6d)[0x2aaaaba59c2d]
---------



[root@ip-10-150-138-205 ~]# hostname -f
ip-10-150-138-205.ap-northeast-1.compute.internal
[root@ip-10-150-138-205 ~]#
Comment 5 Raghavendra G 2011-05-03 23:22:55 EDT
Saurabh,

are you using a release which contained the fix (http://patches.gluster.com/patch/7064)? Seems the version you are using does not contain the fix.

regards,
(In reply to comment #4)
> it happened again on tokyo region,
> 
> 
> [2011-05-03 11:02:48.686121] E [access-control.c:1842:ac_setattr_stat_cbk]
> 0-dist-rep-access-control: setattr failed with error: Operation not permitted
> pending frames:
> 
> patchset: v3.2.0
> signal received: 11
> time of crash: 2011-05-03 11:02:48
> configuration details:
> argp 1
> backtrace 1
> dlfcn 1
> fdatasync 1
> libpthread 1
> llistxattr 1
> setfsid 1
> spinlock 1
> epoll.h 1
> xattr.h 1
> st_atim.tv_nsec 1
> package-string: glusterfs 3.2.0
> /lib64/libc.so.6[0x2aaaab9b62d0]
> /opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/features/marker.so(marker_setattr_cbk+0x139)[0x2aaaad983e79]
> /opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/performance/io-threads.so(iot_setattr_cbk+0x88)[0x2aaaad773718]
> /opt/glusterfs/3.2.0/lib64/libglusterfs.so.0(default_setattr_cbk+0x88)[0x2aaaaaef1f28]
> /opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/features/access-control.so(ac_setattr_stat_cbk+0x137)[0x2aaaad354ec7]
> /opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/storage/posix.so(posix_stat+0x14d)[0x2aaaad13cbfd]
> /opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/features/access-control.so(ac_setattr+0x14c)[0x2aaaad350a7c]
> /opt/glusterfs/3.2.0/lib64/libglusterfs.so.0(default_setattr+0xe9)[0x2aaaaaeeb659]
> /opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/performance/io-threads.so(iot_setattr_wrapper+0xe9)[0x2aaaad776749]
> /opt/glusterfs/3.2.0/lib64/libglusterfs.so.0(call_resume+0xd81)[0x2aaaaaefc191]
> /opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/performance/io-threads.so(iot_worker+0x119)[0x2aaaad77a229]
> /lib64/libpthread.so.0[0x2aaaab770617]
> /lib64/libc.so.6(clone+0x6d)[0x2aaaaba59c2d]
> ---------
> 
> 
> 
> [root@ip-10-150-138-205 ~]# hostname -f
> ip-10-150-138-205.ap-northeast-1.compute.internal
> [root@ip-10-150-138-205 ~]#
Comment 6 Saurabh 2011-05-04 22:40:28 EDT
Hello Raghavendra,


   I was given the new AMI after the patch was fixed, so by default I thought that the patch is propagated to the new AMI's also. Hence, I re-opened the bug, whereas after discussion with Avati came to know that he is going to make knew AMIs having the patch included. 

   Requesting you to keep it open till the AMI's are tested, if this issue is not found again I will move it back to Resolved/Verified state.


Thanks
Saurabh
Comment 7 Saurabh 2011-06-02 00:03:13 EDT
Executed the posix compliance test over nfs/fuse mounts on the filesystem for 3.2.1qa2, and didn't find this issue happening.
Comment 8 Vijay Bellur 2011-06-13 23:52:34 EDT
*** Bug 3014 has been marked as a duplicate of this bug. ***
Comment 9 Vijay Bellur 2011-06-14 20:47:11 EDT
*** Bug 3027 has been marked as a duplicate of this bug. ***
Comment 10 Christopher 2011-06-15 01:13:14 EDT
I can confirm that the problem is solved in 3.2.1

thank you guys!

Note You need to log in before you can comment on or make changes to this bug.