Bug 764024 (GLUSTER-2292)

Summary: Replace-brick at replicate volume fail
Product: [Community] GlusterFS Reporter: Lukasz Jagiello <l.jagiello>
Component: replicateAssignee: krishnan parthasarathi <kparthas>
Status: CLOSED WORKSFORME QA Contact:
Severity: high Docs Contact:
Priority: low    
Version: 3.1.1CC: amarts, gluster-bugs, nsathyan, vijay
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: fuse
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Lukasz Jagiello 2011-01-13 20:21:54 UTC
gluster> volume info

Volume Name: ftp
Type: Replicate
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: 10.0.2.10:/d0/ftp
Brick2: 10.0.2.11:/d0/ftp
Options Reconfigured:
performance.cache-size: 256MB
nfs.enable-ino32: off

gluster> volume replace-brick ftp 10.0.2.11:/d0/ftp 10.0.2.12:/d0/ftp start

At 10.0.2.12:

#v+
==> /var/log/glusterfs/etc-glusterfs-glusterd.vol.log <==
[2011-01-13 21:09:53.79377] I [glusterd-handler.c:514:glusterd_handle_commit_op] glusterd: Received commit op from uuid: 7a8e9302-a42e-4005-a371-30e1d732c255
[2011-01-13 21:09:53.79410] I [glusterd-utils.c:730:glusterd_volume_brickinfo_get_by_brick] : brick: 10.0.2.11:/d0/ftp
[2011-01-13 21:09:53.79426] I [glusterd-utils.c:2105:glusterd_friend_find_by_hostname] glusterd: Friend 10.0.2.11 found.. state: 3
[2011-01-13 21:09:53.79439] I [glusterd-utils.c:701:glusterd_volume_brickinfo_get] : Found brick
[2011-01-13 21:09:53.79843] I [glusterd-op-sm.c:3129:glusterd_op_replace_brick] : I AM THE DESTINATION HOST
[2011-01-13 21:09:53.106633] I [glusterd-op-sm.c:3156:glusterd_op_replace_brick] : adding dst-brick port no
[2011-01-13 21:09:53.106686] I [glusterd-op-sm.c:2991:rb_update_dstbrick_port] : adding dst-brick port no
[2011-01-13 21:09:53.106760] I [glusterd-handler.c:2104:glusterd_op_commit_send_resp] glusterd: Responded to commit, ret: 0
[...]
[2011-01-13 21:10:02.82202] W [server-resolve.c:565:server_resolve] src-server: pure path resolution for /ftp/some_dir (SETATTR)
[2011-01-13 21:10:02.222641] E [rpcsvc.c:1548:rpcsvc_submit_generic] rpc-service: failed to submit message (XID: 0x1080x, Program: GlusterFS-3.1.0, ProgVers: 310, Proc: 31) to rpc-transport (tcp.src-server)
[2011-01-13 21:10:02.222672] E [server.c:137:server_submit_reply] : Reply submission failed
[2011-01-13 21:10:02.222706] I [server.c:428:server_rpc_notify] src-server: disconnected connection from 10.0.2.11:1014
[2011-01-13 21:10:02.222763] I [server-helpers.c:670:server_connection_destroy] src-server: destroyed connection of storage-01-b.gr-24644-2011/01/13-21:09:53:22152-ftp-replace-brick
[2011-01-13 21:10:02.222687] I [client.c:1590:client_rpc_notify] ftp-client-1: disconnected
[2011-01-13 21:10:07.98994] E [socket.c:1661:socket_connect_finish] ftp-client-1: connection to 10.0.2.11:24009 failed (Connection refused)
#v-

After that, when i check status:
gluster> volume replace-brick ftp 10.0.2.11:/d0/ftp 10.0.2.12:/d0/ftp status

command just freeze.

Comment 1 krishnan parthasarathi 2011-06-27 09:52:12 UTC
Lukasz,
Have you been able to recreate the issue when running glusterfs v3.2.1?
The fix for bug 764221 which went into 3.2.1 release, should mostly address this bug as well. Let me know if you still face the issue.