Bug 763388 - (GLUSTER-1656) replace-brick from non-source machine fails.
replace-brick from non-source machine fails.
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: glusterd (Show other bugs)
3.1-alpha
All Linux
low Severity high
: ---
: ---
Assigned To: shishir gowda
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-09-21 04:19 EDT by Lakshmipathi G
Modified: 2015-12-01 11:45 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: RTP
Mount Type: All
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Lakshmipathi G 2010-09-21 04:19:12 EDT
Replace didn't work with qa26. Here is the log

172.24.21.135#gluster volume replace-brick dht-simple 172.24.21.136:/mnt/dht/export1 172.24.21.138:/mnt/dht/export1 start
replace-brick failed to start
Replacing brick from Volume dht-simple failed.

on source brick (136) - no new process is started.
on target brick (138) - one process is running with rb_dst_brick.vol.

---------

cat usr-local-etc-glusterfs-glusterd.vol.log
[2010-09-21 01:02:20.460201] I [glusterd-op-sm.c:3885:glusterd_op_set_cli_op] : Returning 0
[2010-09-21 01:02:20.460309] I [glusterd-handler.c:1565:glusterd_handle_replace_brick] glusterd: Received replace brick req
[2010-09-21 01:02:20.460358] I [glusterd-utils.c:221:glusterd_lock] glusterd: Cluster lock held by 6ce01949-12c7-4974-b363-375ba47fbabc
[2010-09-21 01:02:20.460371] I [glusterd-handler.c:2776:glusterd_op_txn_begin] glusterd: Acquired local lock
[2010-09-21 01:02:20.460385] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 1
[2010-09-21 01:02:20.460397] I [glusterd-handler.c:2780:glusterd_op_txn_begin] glusterd: Returning 0
[2010-09-21 01:02:20.460475] I [glusterd3_1-mops.c:979:glusterd3_1_cluster_lock] glusterd: Sent lock req to 2 peers
[2010-09-21 01:02:20.460490] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 0 to 1
[2010-09-21 01:02:20.460503] W [glusterd3_1-mops.c:1358:glusterd_handle_rpc_msg] : Returning 0
[2010-09-21 01:02:20.460747] I [glusterd3_1-mops.c:402:glusterd3_1_cluster_lock_cbk] glusterd: Received ACC from uuid: f880e25b-cb5e-4806-85db-db10ef448b31
[2010-09-21 01:02:20.460783] I [glusterd-handler.c:114:glusterd_friend_find_by_uuid] glusterd: Friend found.. state: 3
[2010-09-21 01:02:20.460799] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 3
[2010-09-21 01:02:20.460818] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 1 to 1
[2010-09-21 01:02:20.460855] I [glusterd3_1-mops.c:402:glusterd3_1_cluster_lock_cbk] glusterd: Received ACC from uuid: 1f3d9cbd-4412-4af7-9be0-58777cfe14c2
[2010-09-21 01:02:20.460869] I [glusterd-handler.c:114:glusterd_friend_find_by_uuid] glusterd: Friend found.. state: 3
[2010-09-21 01:02:20.460881] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 3
[2010-09-21 01:02:20.460894] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 4
[2010-09-21 01:02:20.460905] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 1 to 1
[2010-09-21 01:02:20.460938] I [glusterd-utils.c:684:glusterd_brickinfo_get] : brick: 172.24.21.136:/mnt/dht/export1/
[2010-09-21 01:02:20.460953] I [glusterd-utils.c:711:glusterd_brickinfo_get] : Found brick
[2010-09-21 01:02:20.461046] I [glusterd3_1-mops.c:1114:glusterd3_1_stage_op] glusterd: Sent op req to 2 peers
[2010-09-21 01:02:20.461066] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 1 to 3
[2010-09-21 01:02:20.461387] I [glusterd3_1-mops.c:521:glusterd3_1_stage_op_cbk] glusterd: Received ACC from uuid: f880e25b-cb5e-4806-85db-db10ef448b31
[2010-09-21 01:02:20.461404] I [glusterd-handler.c:114:glusterd_friend_find_by_uuid] glusterd: Friend found.. state: 3
[2010-09-21 01:02:20.461419] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 3
[2010-09-21 01:02:20.461434] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 3 to 3
[2010-09-21 01:02:20.461467] I [glusterd3_1-mops.c:521:glusterd3_1_stage_op_cbk] glusterd: Received ACC from uuid: 1f3d9cbd-4412-4af7-9be0-58777cfe14c2
[2010-09-21 01:02:20.461497] I [glusterd-handler.c:114:glusterd_friend_find_by_uuid] glusterd: Friend found.. state: 3
[2010-09-21 01:02:20.461510] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 3
[2010-09-21 01:02:20.461523] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 5
[2010-09-21 01:02:20.461535] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 3 to 3
[2010-09-21 01:02:20.461559] I [glusterd-utils.c:684:glusterd_brickinfo_get] : brick: 172.24.21.136:/mnt/dht/export1/
[2010-09-21 01:02:20.461573] I [glusterd-utils.c:711:glusterd_brickinfo_get] : Found brick
[2010-09-21 01:02:20.461722] I [glusterd3_1-mops.c:1205:glusterd3_1_commit_op] glusterd: Sent op req to 2 peers
[2010-09-21 01:02:20.461738] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 3 to 5
[2010-09-21 01:02:20.462085] I [glusterd3_1-mops.c:648:glusterd3_1_commit_op_cbk] glusterd: Received RJT from uuid: f880e25b-cb5e-4806-85db-db10ef448b31
[2010-09-21 01:02:20.462115] I [glusterd-handler.c:114:glusterd_friend_find_by_uuid] glusterd: Friend found.. state: 3
[2010-09-21 01:02:20.462132] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 7
[2010-09-21 01:02:20.462147] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 11
[2010-09-21 01:02:20.462161] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 5 to 5
[2010-09-21 01:02:20.462206] I [glusterd3_1-mops.c:1030:glusterd3_1_cluster_unlock] glusterd: Sent unlock req to 2 peers
[2010-09-21 01:02:20.462219] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 5 to 7
[2010-09-21 01:02:20.462381] I [glusterd3_1-mops.c:461:glusterd3_1_cluster_unlock_cbk] glusterd: Received ACC from uuid: f880e25b-cb5e-4806-85db-db10ef448b31
[2010-09-21 01:02:20.462398] I [glusterd-handler.c:114:glusterd_friend_find_by_uuid] glusterd: Friend found.. state: 3
[2010-09-21 01:02:20.462418] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 3
[2010-09-21 01:02:20.462433] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 7 to 7
[2010-09-21 01:02:20.494629] I [glusterd3_1-mops.c:648:glusterd3_1_commit_op_cbk] glusterd: Received RJT from uuid: 1f3d9cbd-4412-4af7-9be0-58777cfe14c2
[2010-09-21 01:02:20.494645] I [glusterd-handler.c:114:glusterd_friend_find_by_uuid] glusterd: Friend found.. state: 3
[2010-09-21 01:02:20.494662] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 7
[2010-09-21 01:02:20.494674] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 7 to 7
[2010-09-21 01:02:20.494828] I [glusterd3_1-mops.c:461:glusterd3_1_cluster_unlock_cbk] glusterd: Received ACC from uuid: 1f3d9cbd-4412-4af7-9be0-58777cfe14c2
[2010-09-21 01:02:20.494844] I [glusterd-handler.c:114:glusterd_friend_find_by_uuid] glusterd: Friend found.. state: 3
[2010-09-21 01:02:20.494865] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 3
[2010-09-21 01:02:20.494880] I [glusterd-op-sm.c:3747:glusterd_op_sm_inject_event] glusterd: Enqueuing event: 4
[2010-09-21 01:02:20.494893] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 7 to 7
[2010-09-21 01:02:20.494910] I [glusterd-op-sm.c:3312:glusterd_op_txn_complete] glusterd: Cleared local lock
[2010-09-21 01:02:20.494965] I [glusterd-op-sm.c:3276:glusterd_op_send_cli_response] : Returning 0
[2010-09-21 01:02:20.494981] I [glusterd-op-sm.c:3347:glusterd_op_txn_complete] glusterd: Returning 0
[2010-09-21 01:02:20.494993] I [glusterd-op-sm.c:3452:glusterd_op_sm_transition_state] : Transitioning from 7 to 0
Comment 1 shishir gowda 2010-09-23 22:11:42 EDT
Fixed as part of bg 1235.

patch: http://patches.gluster.com/patch/4947
Comment 2 Lakshmipathi G 2010-09-23 23:36:34 EDT
twice it failed. It starts the process the target-brick but commit said-
"Replacing brick from Volume nfsdhtbeta failed".

third time,it passed !!!

#gluster volume replace-brick BETARB  10.192.134.144:/mnt/beta 10.214.231.112:/mnt/beta/ start
replace-brick started successfully
10.192.141.187#gluster volume replace-brick BETARB  10.192.134.144:/mnt/beta 10.214.231.112:/mnt/beta/ status

10.192.141.187#gluster volume replace-brick BETARB  10.192.134.144:/mnt/beta 10.214.231.112:/mnt/beta/ commit
replace-brick commit successful

creating new files on mount pt - distributed to added brick.

issues - 
-it didn't copy  existing files from old-brick 10.192.134.144 to  new-brick 10.214.231.112.
----

Note You need to log in before you can comment on or make changes to this bug.