Attachment 915393 Details for Bug 770478 – Comment

Comment

comment_4277362.txt (text/plain), 70.01 KB, created by Matt Harris on 2011-12-26 23:06:17 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Matt Harris

Created: 2011-12-26 23:06:17 UTC

Size: 70.01 KB

patch

obsolete

>Description of problem:
>
>volume replace-brick does not complete.  Stops anywhere from 1 gigabyte to a hundred gigabytes into the transfer.
>
>Version-Release number of selected component (if applicable):
>3.2.5
>
>How reproducible:
>Always reproducible on my production system, not reproducible on a smaller test cluster.
>
>Steps to Reproduce:
>1.  Join 4 servers to a trusted storage pool. (cloud0, cloud1, cloud2, cloud3)
>2.  Create a gluster volume (gluster volume create store replica 2 transport tcp cloud0:/mnt/live cloud1:/mnt/live) on two servers.
>3.  Verify that the volume is working properly.  Mount clients to the volume since this is a production cluster.
>4.  On cloud0, issue the command "gluster volume replace-brick store cloud1:/mnt/live cloud3:/mnt/live start"
>5.  Monitor the progress by running df on cloud2.  When the usage number stops increasing, the replace-brick has hung.
>6.  On cloud0, issue the command "gluster volume replace-brick store cloud1:/mnt/live cloud3:/mnt/live status" and receive a "status unknown" error.
>7.  To try again, glusterd must be stopped on all four servers and /etc/glusterd/vols/store/rb* must be removed on all four servers.  Otherwise, any volume operation will fail with a "replace-brick operation in progress" error.
>  
>Actual results:
>
>See above.
>
>Expected results:
>
>The replace-brick operation completes normally.
>
>Additional info:
>
>All servers are up-to-date installs of Ubuntu 10.04 LTS.  The bricks on cloud0 and cloud1 are ext4-formatted.  The bricks on cloud2 and cloud3 are formatted as xfs to work around the effective 16 TB ext4 limit.  
>
>The machines are all connected via certified CAT6e cables to a Cisco 2960 switch.  The switch is showing no networking errors on the relevant ports.
>
>Relevant section of cloud3:/var/log/glusterfs/etc-glusterd-vols-store-rb_dst_brick.vol.log:
>
>[2011-12-26 15:45:36.846948] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_74
>[2011-12-26 15:45:36.846993] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_21
>[2011-12-26 15:45:36.847026] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_99
>[2011-12-26 15:45:36.847055] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_41
>[2011-12-26 15:45:36.847085] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_10
>[2011-12-26 15:45:36.847115] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_12
>[2011-12-26 15:45:36.847152] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_78
>[2011-12-26 15:45:36.847192] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_37
>[2011-12-26 15:45:36.847224] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_98
>[2011-12-26 15:45:36.847263] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_92
>[2011-12-26 15:45:36.847296] I [server-helpers.c:485:do_fd_cleanup] 0-src-server: fd cleanup on /storage/voeis/cutthroat/sensitivity-2011-1-19-AFS/Simulation_12_2011-01-19-15-31-26/Repetition_92
>[2011-12-26 15:45:36.847382] I [server.c:438:server_rpc_notify] 0-src-server: disconnected connection from 153.90.203.41:1007
>[2011-12-26 15:45:36.847631] I [server-helpers.c:783:server_connection_destroy] 0-src-server: destroyed connection of cloud1-16677-2011/12/26-15:00:52:49439-store-replace-brick
>
>Relevant section of cloud1:/var/log/glusterfs/etc-glusterfs-glusterd.vol.log:
>
>[2011-12-26 14:59:02.173064] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 14:59:02.178316] I [glusterd-utils.c:909:glusterd_service_stop] 0-: Stopping gluster nfsd running in pid: 16629
>[2011-12-26 14:59:03.178851] I [glusterd-utils.c:2297:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV3 successfully
>[2011-12-26 14:59:03.179140] I [glusterd-utils.c:2302:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV1 successfully
>[2011-12-26 14:59:03.179427] I [glusterd-utils.c:2307:glusterd_nfs_pmap_deregister] 0-: De-registered NFSV3 successfully
>[2011-12-26 14:59:03.193257] I [glusterd-handler.c:2784:glusterd_op_commit_send_resp] 0-glusterd: Responded to commit, ret: 0
>[2011-12-26 14:59:03.193572] I [glusterd-handler.c:2693:glusterd_handle_cluster_unlock] 0-glusterd: Received UNLOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 14:59:03.193622] I [glusterd-handler.c:2671:glusterd_op_unlock_send_resp] 0-glusterd: Responded to unlock, ret: 0
>[2011-12-26 14:59:03.194045] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.40:1010)
>[2011-12-26 14:59:03.213705] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.11:1011)
>[2011-12-26 14:59:03.214005] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.41:1011)
>[2011-12-26 14:59:03.214454] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.10:1011)
>[2011-12-26 14:59:57.354734] I [glusterd-handler.c:448:glusterd_handle_cluster_lock] 0-glusterd: Received LOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 14:59:57.354775] I [glusterd-utils.c:243:glusterd_lock] 0-glusterd: Cluster lock held by ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 14:59:57.354807] I [glusterd-handler.c:2651:glusterd_op_lock_send_resp] 0-glusterd: Responded, ret: 0
>[2011-12-26 14:59:57.356674] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 14:59:57.356722] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 14:59:57.356925] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 14:59:57.357833] I [glusterd-handler.c:2743:glusterd_op_stage_send_resp] 0-glusterd: Responded to stage, ret: 0
>[2011-12-26 14:59:57.365894] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 14:59:57.365936] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 14:59:57.366125] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 14:59:57.366207] I [glusterd-op-sm.c:3408:rb_update_srcbrick_port] 0-: adding src-brick port no
>[2011-12-26 14:59:57.367111] I [glusterd-utils.c:909:glusterd_service_stop] 0-: Stopping gluster nfsd running in pid: 16636
>[2011-12-26 14:59:58.367632] I [glusterd-utils.c:2297:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV3 successfully
>[2011-12-26 14:59:58.367922] I [glusterd-utils.c:2302:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV1 successfully
>[2011-12-26 14:59:58.368174] I [glusterd-utils.c:2307:glusterd_nfs_pmap_deregister] 0-: De-registered NFSV3 successfully
>[2011-12-26 14:59:58.368504] I [glusterd-utils.c:909:glusterd_service_stop] 0-: Stopping gluster brick running in pid: 8118
>[2011-12-26 14:59:58.368562] E [glusterd-utils.c:1186:glusterd_brick_unlink_socket_file] 0-glusterd: Failed to remove /tmp/90776f37de281016353f4ec15769c6e0.socket error: Resource temporarily unavailable
>[2011-12-26 15:00:00.369314] I [glusterd-pmap.c:281:pmap_registry_remove] 0-pmap: removing brick /mnt/live on port 24012
>[2011-12-26 15:00:00.369348] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:00:00.403829] I [glusterd-handler.c:2784:glusterd_op_commit_send_resp] 0-glusterd: Responded to commit, ret: 0
>[2011-12-26 15:00:00.403905] I [glusterd-pmap.c:281:pmap_registry_remove] 0-pmap: removing brick (null) on port 24012
>[2011-12-26 15:00:00.404836] W [socket.c:204:__socket_rwv] 0-socket.management: writev failed (Connection reset by peer)
>[2011-12-26 15:00:00.404864] I [socket.c:1768:socket_event_handler] 0-transport: disconnecting now
>[2011-12-26 15:00:00.404975] W [socket.c:1494:__socket_proto_state_machine] 0-management: reading from socket failed. Error (Transport endpoint is not connected), peer (/tmp/90776f37de281016353f4ec15769c6e0.socket)
>[2011-12-26 15:00:00.407852] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /mnt/live on port 24012
>[2011-12-26 15:00:00.419554] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.41:1020)
>[2011-12-26 15:00:06.322664] I [glusterd-handler.c:2693:glusterd_handle_cluster_unlock] 0-glusterd: Received UNLOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:06.322720] I [glusterd-handler.c:2671:glusterd_op_unlock_send_resp] 0-glusterd: Responded to unlock, ret: 0
>[2011-12-26 15:00:11.764314] I [glusterd-handler.c:448:glusterd_handle_cluster_lock] 0-glusterd: Received LOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:11.764360] I [glusterd-utils.c:243:glusterd_lock] 0-glusterd: Cluster lock held by ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:11.764394] I [glusterd-handler.c:2651:glusterd_op_lock_send_resp] 0-glusterd: Responded, ret: 0
>[2011-12-26 15:00:11.767208] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:11.767248] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:00:11.767479] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 15:00:11.769935] I [glusterd-handler.c:2743:glusterd_op_stage_send_resp] 0-glusterd: Responded to stage, ret: 0
>[2011-12-26 15:00:12.56622] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:12.56681] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:00:12.56905] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 15:00:12.56985] I [glusterd-op-sm.c:3408:rb_update_srcbrick_port] 0-: adding src-brick port no
>[2011-12-26 15:00:12.60134] I [glusterd-utils.c:909:glusterd_service_stop] 0-: Stopping gluster nfsd running in pid: 16645
>[2011-12-26 15:00:13.60681] I [glusterd-utils.c:2297:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV3 successfully
>[2011-12-26 15:00:13.60968] I [glusterd-utils.c:2302:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV1 successfully
>[2011-12-26 15:00:13.61251] I [glusterd-utils.c:2307:glusterd_nfs_pmap_deregister] 0-: De-registered NFSV3 successfully
>[2011-12-26 15:00:13.61333] I [glusterd-utils.c:909:glusterd_service_stop] 0-: Stopping gluster brick running in pid: 16641
>[2011-12-26 15:00:13.61410] E [glusterd-utils.c:1186:glusterd_brick_unlink_socket_file] 0-glusterd: Failed to remove /tmp/90776f37de281016353f4ec15769c6e0.socket error: Resource temporarily unavailable
>[2011-12-26 15:00:15.62263] I [glusterd-pmap.c:281:pmap_registry_remove] 0-pmap: removing brick /mnt/live on port 24012
>[2011-12-26 15:00:15.62298] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:00:15.95318] I [glusterd-handler.c:2784:glusterd_op_commit_send_resp] 0-glusterd: Responded to commit, ret: 0
>[2011-12-26 15:00:15.95392] I [glusterd-pmap.c:281:pmap_registry_remove] 0-pmap: removing brick (null) on port 24012
>[2011-12-26 15:00:15.96407] W [socket.c:204:__socket_rwv] 0-socket.management: writev failed (Connection reset by peer)
>[2011-12-26 15:00:15.96436] I [socket.c:1768:socket_event_handler] 0-transport: disconnecting now
>[2011-12-26 15:00:15.96544] W [socket.c:1494:__socket_proto_state_machine] 0-management: reading from socket failed. Error (Transport endpoint is not connected), peer (/tmp/90776f37de281016353f4ec15769c6e0.socket)
>[2011-12-26 15:00:15.98904] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /mnt/live on port 24012
>[2011-12-26 15:00:15.112412] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.41:1013)
>[2011-12-26 15:00:20.324598] I [glusterd-handler.c:2693:glusterd_handle_cluster_unlock] 0-glusterd: Received UNLOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:20.324651] I [glusterd-handler.c:2671:glusterd_op_unlock_send_resp] 0-glusterd: Responded to unlock, ret: 0
>[2011-12-26 15:00:49.24300] I [glusterd-handler.c:448:glusterd_handle_cluster_lock] 0-glusterd: Received LOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:49.24338] I [glusterd-utils.c:243:glusterd_lock] 0-glusterd: Cluster lock held by ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:49.24370] I [glusterd-handler.c:2651:glusterd_op_lock_send_resp] 0-glusterd: Responded, ret: 0
>[2011-12-26 15:00:49.26315] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:49.26354] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:00:49.26554] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 15:00:49.27411] I [glusterd-handler.c:2743:glusterd_op_stage_send_resp] 0-glusterd: Responded to stage, ret: 0
>[2011-12-26 15:00:49.35323] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:49.35365] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:00:49.35552] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 15:00:49.35632] I [glusterd-op-sm.c:3408:rb_update_srcbrick_port] 0-: adding src-brick port no
>[2011-12-26 15:00:49.36650] I [glusterd-utils.c:909:glusterd_service_stop] 0-: Stopping gluster nfsd running in pid: 16667
>[2011-12-26 15:00:50.37192] I [glusterd-utils.c:2297:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV3 successfully
>[2011-12-26 15:00:50.37481] I [glusterd-utils.c:2302:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV1 successfully
>[2011-12-26 15:00:50.37690] I [glusterd-utils.c:2307:glusterd_nfs_pmap_deregister] 0-: De-registered NFSV3 successfully
>[2011-12-26 15:00:50.37772] I [glusterd-utils.c:909:glusterd_service_stop] 0-: Stopping gluster brick running in pid: 16663
>[2011-12-26 15:00:50.37842] E [glusterd-utils.c:1186:glusterd_brick_unlink_socket_file] 0-glusterd: Failed to remove /tmp/90776f37de281016353f4ec15769c6e0.socket error: Resource temporarily unavailable
>[2011-12-26 15:00:52.38751] I [glusterd-pmap.c:281:pmap_registry_remove] 0-pmap: removing brick /mnt/live on port 24012
>[2011-12-26 15:00:52.38794] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:00:52.72669] I [glusterd-handler.c:2784:glusterd_op_commit_send_resp] 0-glusterd: Responded to commit, ret: 0
>[2011-12-26 15:00:52.72747] I [glusterd-pmap.c:281:pmap_registry_remove] 0-pmap: removing brick (null) on port 24012
>[2011-12-26 15:00:52.73674] W [socket.c:204:__socket_rwv] 0-socket.management: writev failed (Connection reset by peer)
>[2011-12-26 15:00:52.73703] I [socket.c:1768:socket_event_handler] 0-transport: disconnecting now
>[2011-12-26 15:00:52.73811] W [socket.c:1494:__socket_proto_state_machine] 0-management: reading from socket failed. Error (Transport endpoint is not connected), peer (/tmp/90776f37de281016353f4ec15769c6e0.socket)
>[2011-12-26 15:00:52.76676] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /mnt/live on port 24012
>[2011-12-26 15:00:52.88636] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.41:1009)
>[2011-12-26 15:00:57.532443] I [glusterd-handler.c:2693:glusterd_handle_cluster_unlock] 0-glusterd: Received UNLOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:00:57.532498] I [glusterd-handler.c:2671:glusterd_op_unlock_send_resp] 0-glusterd: Responded to unlock, ret: 0
>[2011-12-26 15:07:55.244804] I [glusterd-handler.c:448:glusterd_handle_cluster_lock] 0-glusterd: Received LOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:07:55.244862] I [glusterd-utils.c:243:glusterd_lock] 0-glusterd: Cluster lock held by ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:07:55.244908] I [glusterd-handler.c:2651:glusterd_op_lock_send_resp] 0-glusterd: Responded, ret: 0
>[2011-12-26 15:07:55.248066] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:07:55.248104] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:07:55.248349] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 15:07:55.250394] I [glusterd-handler.c:2743:glusterd_op_stage_send_resp] 0-glusterd: Responded to stage, ret: 0
>[2011-12-26 15:07:55.492550] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:07:55.492614] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:07:55.492844] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 15:07:55.492925] I [glusterd-op-sm.c:3408:rb_update_srcbrick_port] 0-: adding src-brick port no
>[2011-12-26 15:07:55.495580] I [glusterd-handler.c:2784:glusterd_op_commit_send_resp] 0-glusterd: Responded to commit, ret: 0
>[2011-12-26 15:07:56.585634] I [glusterd-handler.c:2693:glusterd_handle_cluster_unlock] 0-glusterd: Received UNLOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:07:56.585705] I [glusterd-handler.c:2671:glusterd_op_unlock_send_resp] 0-glusterd: Responded to unlock, ret: 0
>[2011-12-26 15:10:56.602159] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.26:1021)
>[2011-12-26 15:19:34.906998] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.24:1021)
>[2011-12-26 15:27:40.842408] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.13:1021)
>[2011-12-26 15:31:08.165021] I [glusterd-handler.c:448:glusterd_handle_cluster_lock] 0-glusterd: Received LOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:31:08.165066] I [glusterd-utils.c:243:glusterd_lock] 0-glusterd: Cluster lock held by ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:31:08.165110] I [glusterd-handler.c:2651:glusterd_op_lock_send_resp] 0-glusterd: Responded, ret: 0
>[2011-12-26 15:31:08.168278] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:31:08.168318] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:31:08.171983] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 15:31:08.174211] I [glusterd-handler.c:2743:glusterd_op_stage_send_resp] 0-glusterd: Responded to stage, ret: 0
>[2011-12-26 15:31:08.392797] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:31:08.392864] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:31:08.393096] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 15:31:08.393176] I [glusterd-op-sm.c:3408:rb_update_srcbrick_port] 0-: adding src-brick port no
>[2011-12-26 15:31:08.395091] I [glusterd-handler.c:2784:glusterd_op_commit_send_resp] 0-glusterd: Responded to commit, ret: 0
>[2011-12-26 15:31:09.761100] I [glusterd-handler.c:2693:glusterd_handle_cluster_unlock] 0-glusterd: Received UNLOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:31:09.761172] I [glusterd-handler.c:2671:glusterd_op_unlock_send_resp] 0-glusterd: Responded to unlock, ret: 0
>[2011-12-26 15:45:36.628000] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1011)
>[2011-12-26 15:45:36.628118] W [socket.c:1494:__socket_proto_state_machine] 0-management: reading from socket failed. Error (Transport endpoint is not connected), peer (/tmp/90776f37de281016353f4ec15769c6e0.socket)
>[2011-12-26 15:45:37.589574] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:45:40.590099] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:45:43.590553] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:45:46.591054] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:45:49.591526] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:45:52.592030] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:45:55.592493] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:45:58.592994] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:01.593461] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:04.593958] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:07.594432] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:10.594939] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:13.595408] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:16.595911] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:19.596379] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:22.596884] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:25.597354] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:28.597873] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:31.598375] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:34.598851] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:37.599351] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:40.599820] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:43.600320] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:46.600789] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:49.601293] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:52.601761] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:55.602262] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:46:58.602725] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:01.603218] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:04.603690] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:07.604204] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:10.604659] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:13.605163] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:16.605625] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:19.606132] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:22.606594] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:25.607103] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:28.607571] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:30.165686] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.8:1020)
>[2011-12-26 15:47:31.608086] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:34.608582] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:37.609077] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:40.609584] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:43.610044] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:46.610531] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:49.611013] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:52.611542] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:55.612053] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:47:58.612526] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:01.613034] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:04.613494] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:07.613995] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:10.614462] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:13.614964] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:16.615429] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:19.615949] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:22.616418] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:25.616920] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:28.617407] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:31.617908] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:34.618427] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:37.618897] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:40.619385] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:43.619855] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:46.620369] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:49.620886] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:52.621361] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:55.621855] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:48:58.622329] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:01.622843] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:04.623306] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:07.623811] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:10.624293] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:13.624790] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:16.625293] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:19.625780] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:22.626286] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:25.626747] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:28.627251] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:31.627714] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:34.628218] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:37.628701] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:40.629206] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:43.629739] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:46.630204] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:49.630732] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:52.631242] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:55.631711] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:49:58.632226] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:01.632701] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:04.633196] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:07.633665] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:10.634181] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:13.634696] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:16.635183] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:19.635694] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:22.636201] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:25.636684] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:28.637187] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:31.637660] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:34.638167] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:37.638671] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:40.639168] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:43.639670] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:46.640133] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:49.640648] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:52.641146] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:55.641648] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:50:58.642153] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:01.642628] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:04.643142] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:07.643646] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:10.644125] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:13.644659] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:16.645174] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:19.645645] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:22.646149] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:25.646610] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:28.647112] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:31.647571] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:34.648075] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:37.648539] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:40.649059] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:43.649567] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:46.650053] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:49.650570] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:52.651029] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:55.651543] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:51:58.652032] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:01.652554] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:04.653069] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:07.653584] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:10.654051] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:13.654553] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:16.655077] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:19.655575] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:22.656085] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:25.656600] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:28.657086] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:31.657608] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:34.658105] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:37.658603] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:40.659107] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:43.659573] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:46.660080] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:49.660610] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:52.661090] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:55.661609] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:52:58.662146] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:01.662661] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:04.663143] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:07.663657] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:10.664174] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:13.664664] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:16.665169] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:19.665678] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:22.666172] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:25.666707] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:28.667231] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:31.667759] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:34.668233] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:37.668754] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:40.669272] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:43.669738] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:46.670259] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:49.670785] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:52.671269] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:55.671789] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:53:58.672303] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:01.672774] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:04.673271] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:07.673736] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:10.674254] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:13.674726] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:16.675245] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:19.675757] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:22.676237] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:25.676738] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:28.677257] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:31.677731] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:34.678257] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:37.678762] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:40.679247] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:43.679783] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:46.680302] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:49.680761] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:52.681282] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:55.681805] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:54:58.682277] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:01.682786] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:04.683250] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:07.683754] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:10.684214] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:13.684723] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:16.685185] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:19.685691] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:22.686153] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:25.686650] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:28.687113] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:31.687631] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:34.688142] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:37.688638] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:40.689149] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:43.689604] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:46.690125] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:49.690628] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:52.691119] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:55.691626] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:55:58.692138] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:01.692623] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:04.693149] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:07.693662] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:10.694148] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:13.694661] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:16.695168] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:19.695663] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:22.696178] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:25.696689] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:28.697175] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:31.697681] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:34.698142] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:37.698671] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:40.699201] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:43.699715] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:46.700204] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:49.700709] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:52.701222] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:55.701719] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:56:58.702216] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:01.702695] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:04.703192] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:07.703646] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:10.704168] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:13.704677] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:16.705158] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:19.705690] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:22.706211] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:25.706696] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:28.707218] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:31.707725] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:34.708196] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:37.708701] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:40.709174] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:43.709695] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:46.710232] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:49.710754] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:52.711223] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:55.711732] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:57:58.712231] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:01.712738] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:04.713253] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:07.713779] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:10.714277] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:13.714791] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:16.715296] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:19.715768] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:22.716286] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:25.716804] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:28.717273] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:31.426051] I [glusterd-handler.c:448:glusterd_handle_cluster_lock] 0-glusterd: Received LOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:58:31.426091] I [glusterd-utils.c:243:glusterd_lock] 0-glusterd: Cluster lock held by ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:58:31.426125] I [glusterd-handler.c:2651:glusterd_op_lock_send_resp] 0-glusterd: Responded, ret: 0
>[2011-12-26 15:58:31.428096] I [glusterd-handler.c:488:glusterd_req_ctx_create] 0-glusterd: Received op from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:58:31.428137] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: cloud1.rcg.montana.edu:/mnt/live
>[2011-12-26 15:58:31.428331] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick
>[2011-12-26 15:58:31.429135] I [glusterd-handler.c:2743:glusterd_op_stage_send_resp] 0-glusterd: Responded to stage, ret: 0
>[2011-12-26 15:58:31.594261] I [glusterd-handler.c:2693:glusterd_handle_cluster_unlock] 0-glusterd: Received UNLOCK from uuid: ac089196-dfe1-4743-96d9-fe349dae8387
>[2011-12-26 15:58:31.594311] I [glusterd-handler.c:2671:glusterd_op_unlock_send_resp] 0-glusterd: Responded to unlock, ret: 0
>[2011-12-26 15:58:31.717788] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:34.718264] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:37.718775] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:40.719288] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:43.719764] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:46.720293] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:49.720807] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:52.721279] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:55.721816] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:58:58.722355] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:01.722875] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:04.723393] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:07.723878] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:10.724374] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:13.724844] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:16.725355] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:19.725825] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:22.726325] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:25.726802] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:28.727313] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:31.727835] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:34.728312] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:37.728836] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:40.729343] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:43.729826] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:46.730351] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:49.730877] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:52.731392] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:55.731867] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 15:59:58.732379] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:01.732837] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:04.733368] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:07.733860] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:10.734360] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:13.734889] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:16.735403] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:19.735876] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:22.736388] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:25.736926] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:28.737414] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:31.737936] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:34.738441] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:37.738968] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:40.739445] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:43.739967] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:46.740485] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:49.741014] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:52.741510] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:55.742022] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:00:58.742552] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:01.743040] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:04.743563] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:07.744088] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:10.744610] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:13.745099] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:16.745615] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:19.746137] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:22.746598] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:25.747112] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:28.747597] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:31.748105] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:34.748635] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:37.749136] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:40.749642] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:43.750166] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:46.750698] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:49.751214] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:52.751693] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:55.752223] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:01:58.752730] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:01.753211] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:04.753750] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:07.754270] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:10.754774] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:13.755266] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:16.755775] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:19.756261] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:22.756770] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:25.757291] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:28.757801] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:31.758304] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:34.758830] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:37.759336] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:40.759817] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:43.760342] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:46.760857] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:49.761331] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:52.761851] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:55.762382] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:02:56.607592] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (153.90.203.15:1021)
>[2011-12-26 16:02:58.762912] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:01.763397] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:04.763920] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:07.764428] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:10.764920] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:13.765423] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:16.765935] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:19.766413] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:22.766939] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:25.767473] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:28.767968] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:31.768480] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:34.768991] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:37.769425] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:40.769955] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:43.770466] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:46.770944] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:49.771450] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:52.771921] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:55.772428] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:03:58.772902] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:01.773433] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:04.773972] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:07.774483] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:10.774958] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:13.775471] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:16.775993] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:19.776464] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:22.776977] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:25.777495] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:28.777979] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:31.778496] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:34.778998] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:37.779495] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:40.780017] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:43.780538] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:46.780998] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:49.781508] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:52.781976] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:55.782480] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:04:58.782944] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)
>[2011-12-26 16:05:01.783453] E [socket.c:2080:socket_connect] 0-management: connection attempt failed (Connection refused)

Actions: View

Attachments on bug 770478: 550577 | 915393