+++ This bug was initially created as a clone of Bug #823154 +++

Created attachment 585576 [details]
brick and rebalance logs

Description of problem:

One of the brick process crashed while rebalancing a distributed-replicate volume

Version-Release number of selected component (if applicable):
3.3.0qa41

How reproducible:


Steps to Reproduce:
1.created a dist-rep volume of 4x2 
2. started kernel untar on the mount
3. add a pair of brick to the volume and initiated rebalance.
  
Actual results:
eventually one of the brick process crashed.

 

Additional info:

Loaded symbols for /lib64/libgcc_s.so.1
Core was generated by `/usr/local/sbin/glusterfsd -s localhost --volfile-id replica.10.16.157.66.home-'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007ff25ce2d03e in posix_lookup (frame=0x7ff2601f0032, this=0x3831332f31383133, loc=0x2f303831332f3937, 
    xdata=0x31332f383731332f) at posix.c:137
137                     MAKE_ENTRY_HANDLE (real_path, par_path, this, loc, &buf);
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.9.x86_64 libgcc-4.4.6-3.el6.x86_64 openssl-1.0.0-20.el6_2.4.x86_64 zlib-1.2.3-27.el6.x86_64
(gdb) bt
#0  0x00007ff25ce2d03e in posix_lookup (frame=0x7ff2601f0032, this=0x3831332f31383133, loc=0x2f303831332f3937, 
    xdata=0x31332f383731332f) at posix.c:137
#1  0x00007ff25cc19d96 in posix_acl_lookup (frame=0x7ff2601f55ac, this=0x10dc270, loc=0x7ff25feb7868, xattr=0x7ff25fe645a8)
    at posix-acl.c:798
#2  0x00007ff25ca05e31 in pl_lookup (frame=0x7ff260200b14, this=0x10dd320, loc=0x7ff25feb7868, xdata=0x7ff25fe645a8) at posix.c:1661
#3  0x00007ff25c7e0f01 in iot_lookup_wrapper (frame=0x7ff2602032b8, this=0x10de440, loc=0x7ff25feb7868, xdata=0x7ff25fe645a8)
    at io-threads.c:288
#4  0x00007ff2613e79fc in call_resume_wind (stub=0x7ff25feb7828) at call-stub.c:2689
#5  0x00007ff2613eefea in call_resume (stub=0x7ff25feb7828) at call-stub.c:4151
#6  0x00007ff25c7e08d6 in iot_worker (data=0x10f0550) at io-threads.c:131
#7  0x000000358f2077f1 in start_thread () from /lib64/libpthread.so.0
#8  0x000000358eae5ccd in clone () from /lib64/libc.so.6



(gdb) f 0
#0  0x00007ff25ce2d03e in posix_lookup (frame=0x7ff2601f0032, this=0x3831332f31383133, loc=0x2f303831332f3937, 
    xdata=0x31332f383731332f) at posix.c:137
137                     MAKE_ENTRY_HANDLE (real_path, par_path, this, loc, &buf);


(gdb) p loc
$1 = (loc_t *) 0x2f303831332f3937
(gdb) f 1
#1  0x00007ff25cc19d96 in posix_acl_lookup (frame=0x7ff2601f55ac, this=0x10dc270, loc=0x7ff25feb7868, xattr=0x7ff25fe645a8)
    at posix-acl.c:798
798             STACK_WIND (frame, posix_acl_lookup_cbk,
(gdb) p loc
$2 = (loc_t *) 0x7ff25feb7868



looks like possible stack corruption.

attached the logs

--- Additional comment from amarts on 2012-05-20 23:38:12 EDT ---

went through the logs, there is no crash dump detail in the logs. Should I be looking at some other logs?

--- Additional comment from shmohan on 2012-05-21 03:00:58 EDT ---

Created attachment 585746 [details]
attached the gzippedcore

attached the gzipped core

--- Additional comment from sgowda on 2012-07-06 01:38:07 EDT ---

Please try to reproduce the bug and save the information.