Bug 868269 - glusterd receiving responses from NULL uuid
glusterd receiving responses from NULL uuid
Status: CLOSED WORKSFORME
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: glusterd (Show other bugs)
2.0
Unspecified Unspecified
medium Severity unspecified
: ---
: ---
Assigned To: Avra Sengupta
spandura
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-19 07:34 EDT by spandura
Modified: 2013-06-05 03:05 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-05-07 06:13:44 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description spandura 2012-10-19 07:34:00 EDT
Description of problem:
========================
glusterd receiving responses from NULL uuid. The messages are seen in glusterd log file. 

[2011-12-04 00:12:05.096714] E [rpc-clnt.c:373:saved_frames_unwind] (-->/usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x78) [0x7f8d11ff3818] (-->/usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0xb0) [0x7f8d11ff34d0] (-->/usr/lib64/libgfrpc.so.0(saved_frames_destroy+0xe) [0x7f8d11ff2f3e]))) 0-management: forced unwinding frame type(glusterd mgmt) op(--(2)) called at 2011-12-04 00:06:46.119673 (xid=0x69x)
[2011-12-04 00:12:05.118427] I [glusterd-rpc-ops.c:607:glusterd3_1_cluster_unlock_cbk] 0-glusterd: Received RJT from uuid: 00000000-0000-0000-0000-000000000000
[2011-12-04 00:12:05.130639] I [glusterd-handler.c:407:glusterd_friend_find] 0-glusterd: Unable to find peer by uuid
[2011-12-04 00:12:05.130689] C [glusterd-rpc-ops.c:613:glusterd3_1_cluster_unlock_cbk] 0-: Unlock response received from unknown peer 00000000-0000-0000-0000-000000000000



Version-Release number of selected component (if applicable):
===============================================================
[root@darrel ~]# gluster --version
glusterfs 3.3.0.3rhs built on Oct 10 2012 09:16:20
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.

[root@darrel ~]# uname -a
Linux darrel.lab.eng.blr.redhat.com 2.6.32-220.28.1.el6.x86_64 #1 SMP Wed Oct 3 12:26:28 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux

Steps to Reproduce:
======================
1. Create a 2 node replicate volume

2. Create a fuse mount and execute the following command : "for i in `seq 1 10000`; do mkdir dir.$i ; for j in `seq 1 10` ; do dd if=/dev/input_file of=dir.$i/file.$j bs=1k count=1024 ; done ; done "

3. While dd is in progress , reboot one of the server. 

4. Run the command 'gluster volume status all' from the active node.

5. once the server reboots, poweroff the server.  
  
Actual results:
=============
saw glusterd receiving responses from NULL uuid.
Comment 5 Kaushal 2012-12-13 02:13:09 EST
The uuid that is being logged here is obtained from the response structure, no the one from the peerinfo.  
The response structure gotten by the unlock_cbk function can be an empty structure, when the cbk is called because of connection failure (which causes a call_bail or a disconnect event). This causes the messages in the logs about null uuid.  
This is not a problem caused by peer probe or any other peer state-machine issues.

Note You need to log in before you can comment on or make changes to this bug.