Hide Forgot
The stack seems to be corrupted. Can you please reproduce the issue?
last night's valgrind run crashed with stripe setup on branch "release-3.2" (83a8938fec225268a3afdbcad1011f0e06e54d16) Complete log and core file can be found at root.12.137:/mnt/nightly_valgrind/logs_stripe/2011-09-22 valgrind log- --------- ==9770== Invalid read of size 4 ==9770== at 0x3D92C08DC1: pthread_mutex_lock (in /lib64/libpthread-2.10.2.so) ==9770== by 0x4C41E7C: inode_link (inode.c:823) ==9770== by 0x62D310B: fuse_entry_cbk (fuse-bridge.c:148) ==9770== by 0x62D3645: fuse_lookup_cbk (fuse-bridge.c:230) ==9770== by 0x7DC9FB2: io_stats_lookup_cbk (io-stats.c:1391) ==9770== by 0x7BA261D: sp_lookup_cbk (stat-prefetch.c:919) ==9770== by 0x7986A0E: qr_lookup_cbk (quick-read.c:439) ==9770== by 0x6F03E19: client3_1_mkdir_cbk (client3_1-fops.c:291) ==9770== by 0x4E93FE9: rpc_clnt_handle_reply (rpc-clnt.c:741) ==9770== by 0x4E94348: rpc_clnt_notify (rpc-clnt.c:854) ==9770== by 0x4E90701: rpc_transport_notify (rpc-transport.c:919) ==9770== by 0x83E1B0A: socket_event_poll_in (socket.c:1647) ==9770== Address 0x363837396e75723f is not stack'd, malloc'd or (recently) free'd fuse - log --------- [2011-09-22 16:43:05.530472] W [client3_1-fops.c:2669:client3_1_stat] 0-stripe-client-3: failed to send the fop Transport endpoint is not connected [2011-09-22 16:43:05.550762] W [client3_1-fops.c:3790:client3_1_fstat] 0-stripe-client-3: (5136460): failed to get fd ctx. EBADFD [2011-09-22 16:43:05.551455] W [client3_1-fops.c:3807:client3_1_fstat] 0-stripe-client-3: failed to send the fop: File descriptor in bad state [2011-09-22 16:43:05.559279] W [client3_1-fops.c:3662:client3_1_flush] 0-stripe-client-3: (5136460): failed to get fd ctx. EBADFD [2011-09-22 16:43:05.559546] W [client3_1-fops.c:3692:client3_1_flush] 0-stripe-client-3: failed to send the fop: File descriptor in bad state [2011-09-22 16:43:05.567815] W [client3_1-fops.c:2669:client3_1_stat] 0-stripe-client-3: failed to send the fop Transport endpoint is not connected [2011-09-22 16:43:05.572282] W [client3_1-fops.c:2669:client3_1_stat] 0-stripe-client-3: failed to send the fop Transport endpoint is not connected [2011-09-22 16:43:05.581215] W [client3_1-fops.c:2606:client3_1_lookup] 0-stripe-client-3: failed to send the fop: Transport endpoint is not connected [2011-09-22 16:43:05.592059] W [client3_1-fops.c:2606:client3_1_lookup] 0-stripe-client-3: failed to send the fop: Transport endpoint is not connected [2011-09-22 16:43:05.749684] W [client3_1-fops.c:2606:client3_1_lookup] 0-stripe-client-3: failed to send the fop: Transport endpoint is not connected [2011-09-22 16:43:05.758565] W [client3_1-fops.c:2606:client3_1_lookup] 0-stripe-client-3: failed to send the fop: Transport endpoint is not connected [2011-09-22 16:43:05.764022] W [client3_1-fops.c:3301:client3_1_mkdir] 0-stripe-client-3: failed to send the fop: Transport endpoint is not connected [2011-09-22 16:43:33.630252] I [client-handshake.c:1090:select_server_supported_programs] 0-stripe-client-3: Using Program GlusterFS 3.2git, Num (1298437), Version (310) [2011-09-22 16:43:33.632832] I [client-handshake.c:913:client_setvolume_cbk] 0-stripe-client-3: Connected to 10.1.12.137:8003, attached to remote volume '/data/export4'. [2011-09-22 16:43:33.633088] I [client-handshake.c:779:client_post_handshake] 0-stripe-client-3: 4 fds open - Delaying child_up until they are re-opened [2011-09-22 16:43:33.633426] W [inode.c:1044:inode_path] 0-stripe/inode: no dentry for non-root inode 5136463: 6279c771-3c61-4afe-ab15-14daca3634a3 [2011-09-22 16:43:33.633707] W [client-handshake.c:683:protocol_client_reopen] 0-stripe-client-3: couldn't build path from inode 6279c771-3c61-4afe-ab15-14daca3634a3 [2011-09-22 16:43:33.633934] E [client-handshake.c:722:protocol_client_reopen] 0-: failed to send the re-open request [2011-09-22 16:43:33.634225] W [inode.c:1044:inode_path] 0-stripe/inode: no dentry for non-root inode 5136488: 35d8be3e-df9f-4315-bfc7-b454ce6c9b34 [2011-09-22 16:43:33.634488] W [client-handshake.c:683:protocol_client_reopen] 0-stripe-client-3: couldn't build path from inode 35d8be3e-df9f-4315-bfc7-b454ce6c9b34 [2011-09-22 16:43:33.634718] E [client-handshake.c:722:protocol_client_reopen] 0-: failed to send the re-open request [2011-09-22 16:43:33.634993] W [inode.c:1044:inode_path] 0-stripe/inode: no dentry for non-root inode 5136455: 1a23309b-053a-4a78-b47f-96796ef04561 [2011-09-22 16:43:33.635281] W [client-handshake.c:683:protocol_client_reopen] 0-stripe-client-3: couldn't build path from inode 1a23309b-053a-4a78-b47f-96796ef04561 [2011-09-22 16:43:33.635523] E [client-handshake.c:722:protocol_client_reopen] 0-: failed to send the re-open request [2011-09-22 16:43:33.637434] I [client-lk.c:617:decrement_reopen_fd_count] 0-stripe-client-3: last fd open'd/lock-self-heal'd - notifying CHILD-UP [2011-09-22 16:43:33.637726] I [client3_1-fops.c:2364:client_fdctx_destroy] 0-stripe-client-3: sending release on fd [2011-09-22 16:43:33.693397] W [fuse-bridge.c:184:fuse_entry_cbk] 0-glusterfs-fuse: 1476580: MKDIR() /run9786/openssl-1.0.0d => -1 (Transport endpoint is not connected) pending frames: frame : type(1) op(LOOKUP) frame : type(1) op(LOOKUP) frame : type(1) op(LOOKUP) frame : type(1) op(LOOKUP) patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2011-09-22 16:43:33 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.2git /lib64/libc.so.6[0x3d920332f0] /lib64/libpthread.so.0(pthread_mutex_lock+0x1)[0x3d92c08dc1] /opt/glusterfs/nightly_valgrind/lib/libglusterfs.so.0(inode_link+0x9b)[0x4c41e7d] /opt/glusterfs/nightly_valgrind/lib/glusterfs/3.2git/xlator/mount/fuse.so[0x62d310c] /opt/glusterfs/nightly_valgrind/lib/glusterfs/3.2git/xlator/mount/fuse.so[0x62d3646] /opt/glusterfs/nightly_valgrind/lib/glusterfs/3.2git/xlator/debug/io-stats.so(io_stats_lookup_cbk+0x23c)[0x7dc9fb3] /opt/glusterfs/nightly_valgrind/lib/glusterfs/3.2git/xlator/performance/stat-prefetch.so(sp_lookup_cbk+0x434)[0x7ba261e] /opt/glusterfs/nightly_valgrind/lib/glusterfs/3.2git/xlator/performance/quick-read.so(qr_lookup_cbk+0x6fc)[0x7986a0f] /opt/glusterfs/nightly_valgrind/lib/glusterfs/3.2git/xlator/protocol/client.so(client3_1_mkdir_cbk+0x3c9)[0x6f03e1a] /opt/glusterfs/nightly_valgrind/lib/libgfrpc.so.0(rpc_clnt_handle_reply+0x211)[0x4e93fea] /opt/glusterfs/nightly_valgrind/lib/libgfrpc.so.0(rpc_clnt_notify+0x285)[0x4e94349] /opt/glusterfs/nightly_valgrind/lib/libgfrpc.so.0(rpc_transport_notify+0x130)[0x4e90702] /opt/glusterfs/nightly_valgrind/lib/glusterfs/3.2git/rpc-transport/socket.so(socket_event_poll_in+0x54)[0x83e1b0b] /opt/glusterfs/nightly_valgrind/lib/glusterfs/3.2git/rpc-transport/socket.so(socket_event_handler+0x21d)[0x83e208f] /opt/glusterfs/nightly_valgrind/lib/libglusterfs.so.0[0x4c58b0c] /opt/glusterfs/nightly_valgrind/lib/libglusterfs.so.0[0x4c58d2f] /opt/glusterfs/nightly_valgrind/lib/libglusterfs.so.0(event_dispatch+0x88)[0x4c590ba] /opt/glusterfs/nightly_valgrind/sbin/glusterfs(main+0x1b7)[0x406fd6] /lib64/libc.so.6(__libc_start_main+0xfd)[0x3d9201ea4d] /opt/glusterfs/nightly_valgrind/sbin/glusterfs[0x4036d9] ---------
release is 3.2.4
*** This bug has been marked as a duplicate of bug 3505 ***