Attachment 415522 Details for Bug 594511 – fence_tool dump

fence_tool dump

fenced.txt (text/plain), 7.18 KB, created by David Teigland on 2010-05-20 21:33:44 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: David Teigland

Created: 2010-05-20 21:33:44 UTC

Size: 7.18 KB

patch

obsolete

>1274304341 logging mode 3 syslog f 160 p 6 logfile p 6 /var/log/cluster/fenced.log
>1274304341 fenced 3.0.12 started
>1274304341 cluster node 1 added
>1274304341 cluster node 3 added
>1274304341 cluster node 4 added
>1274304341 our_nodeid 1 our_name morph-01
>1274304342 logging mode 3 syslog f 160 p 6 logfile p 6 /var/log/cluster/fenced.log
>1274304342 cpg_join fenced:daemon ...
>1274304342 setup_cpg_daemon 12
>1274304342 group_mode 3 compat 0
>1274304342 fenced:daemon conf 3 1 0 memb 1 3 4 join 1 left
>1274304342 receive_protocol from 1 max 1.1.1.0 run 0.0.0.0
>1274304342 daemon node 1 max 0.0.0.0 run 0.0.0.0
>1274304342 daemon node 1 join 1274304342 left 0 local quorum 1274304341
>1274304342 receive_protocol from 3 max 1.1.1.0 run 1.1.1.1
>1274304342 daemon node 3 max 0.0.0.0 run 0.0.0.0
>1274304342 daemon node 3 join 1274304342 left 0 local quorum 1274304341
>1274304342 run protocol from nodeid 3
>1274304342 receive_protocol from 4 max 1.1.1.0 run 1.1.1.1
>1274304342 daemon node 4 max 0.0.0.0 run 0.0.0.0
>1274304342 daemon node 4 join 1274304342 left 0 local quorum 1274304341
>1274304342 daemon run 1.1.1 max 1.1.1
>1274304342 receive_protocol from 1 max 1.1.1.0 run 1.1.1.0
>1274304342 daemon node 1 max 1.1.1.0 run 0.0.0.0
>1274304342 daemon node 1 join 1274304342 left 0 local quorum 1274304341
>1274304342 client connection 3 fd 13
>1274304342 /cluster/fence_daemon/@clean_start is 0
>1274304342 /cluster/fence_daemon/@post_join_delay is 20
>1274304343 added 3 nodes from ccs
>1274304343 fenced:default conf 3 1 0 memb 1 3 4 join 1 left
>1274304343 add_change cg 1 joined nodeid 1
>1274304343 add_change cg 1 counts member 3 joined 1 remove 0 failed 0
>1274304343 check_quorum done
>1274304343 send_start cg 1 flags 1 counts 0 3 1 0 0
>1274304343 receive_start 4:2 len 192
>1274304343 match_change 4:2 matches cg 1
>1274304343 save_history 1 master 3 time 1274304264 how 1
>1274304343 save_history 3 master 1 time 1274302911 how 1
>1274304343 save_history 4 master 3 time 1274304257 how 1
>1274304343 wait_messages cg 1 need 2 of 3
>1274304343 receive_start 3:9 len 192
>1274304343 match_change 3:9 matches cg 1
>1274304343 wait_messages cg 1 need 1 of 3
>1274304343 receive_start 1:1 len 192
>1274304343 match_change 1:1 matches cg 1
>1274304343 wait_messages cg 1 got all 3
>1274304343 set_master from 0 to complete node 3
>1274304343 receive_complete 3:9 len 192
>1274304702 cluster node 3 removed
>1274304702 cluster node 4 removed
>1274304702 fenced:daemon conf 2 0 1 memb 1 4 join left 3
>1274304702 fenced:daemon conf 1 0 1 memb 1 join left 4
>1274304702 fenced:default conf 2 0 1 memb 1 4 join left 3
>1274304702 add_change cg 2 remove nodeid 3 reason 3
>1274304702 add_change cg 2 counts member 2 joined 0 remove 1 failed 1
>1274304702 add_victims node 3
>1274304702 check_quorum not quorate
>1274304702 fenced:default conf 1 0 1 memb 1 join left 4
>1274304702 add_change cg 3 remove nodeid 4 reason 3
>1274304702 add_change cg 3 counts member 1 joined 0 remove 1 failed 1
>1274304702 add_victims node 4
>1274304702 check_quorum not quorate
>1274304741 cluster node 3 added
>1274304741 check_quorum done
>1274304741 send_start cg 3 flags 2 counts 1 1 0 1 1
>1274304741 cpg_mcast_joined retried 7 start
>1274304741 receive_start 1:3 len 192
>1274304741 match_change 1:3 skip cg 2 expect counts 2 0 1 1
>1274304741 match_change 1:3 matches cg 3
>1274304741 wait_messages cg 3 got all 1
>1274304741 set_master from 3 to complete node 1
>1274304741 morph-04 not a cluster member after 0 sec post_fail_delay
>1274304741 morph-03 not a cluster member after 0 sec post_fail_delay
>1274304741 fencing node morph-04
>1274304746 fence morph-04 dev 0.0 agent fence_apc result: error from agent
>1274304746 fence morph-04 failed
>1274304749 fenced:daemon conf 2 1 0 memb 1 3 join 3 left
>1274304749 receive_protocol from 3 max 1.1.1.0 run 0.0.0.0
>1274304749 daemon node 3 max 0.0.0.0 run 0.0.0.0
>1274304749 daemon node 3 join 1274304749 left 1274304702 local quorum 1274304741
>1274304749 fencing node morph-04
>1274304753 fence morph-04 dev 0.0 agent fence_apc result: success
>1274304753 fence morph-04 success
>1274304753 send_victim_done cg 3 flags 2 victim nodeid 4
>1274304753 receive_protocol from 1 max 1.1.1.0 run 1.1.1.1
>1274304753 daemon node 1 max 1.1.1.0 run 1.1.1.0
>1274304753 daemon node 1 join 1274304342 left 0 local quorum 1274304741
>1274304753 receive_protocol from 3 max 1.1.1.0 run 1.1.1.0
>1274304753 daemon node 3 max 1.1.1.0 run 0.0.0.0
>1274304753 daemon node 3 join 1274304749 left 1274304702 local quorum 1274304741
>1274304753 daemon_member 3 is clean
>1274304753 averting fence of node morph-03 cluster member 1 cpg member 1 external 0
>1274304753 send_victim_done cg 3 flags 2 victim nodeid 3
>1274304753 send_complete cg 3 flags 2 counts 1 1 0 1 1
>1274304753 fenced:default conf 2 1 0 memb 1 3 join 3 left
>1274304753 add_change cg 4 joined nodeid 3
>1274304753 add_change cg 4 counts member 2 joined 1 remove 0 failed 0
>1274304753 check_quorum done
>1274304753 send_start cg 4 flags 2 counts 2 2 1 0 0
>1274304753 receive_start 3:1 len 192
>1274304753 match_change 3:1 matches cg 4
>1274304753 wait_messages cg 4 need 1 of 2
>1274304753 receive_victim_done 1:3 flags 2 len 80
>1274304753 receive_victim_done 1:3 remove victim 4 time 1274304753 how 1
>1274304753 wait_messages cg 4 need 1 of 2
>1274304753 receive_victim_done 1:3 flags 2 len 80
>1274304753 receive_victim_done 1:3 remove victim 3 time 1274304753 how 2
>1274304753 wait_messages cg 4 need 1 of 2
>1274304753 receive_complete 1:3 len 192
>1274304753 wait_messages cg 4 need 1 of 2
>1274304753 receive_start 1:4 len 192
>1274304753 match_change 1:4 matches cg 4
>1274304753 wait_messages cg 4 got all 2
>1274304753 set_master from 1 to complete node 1
>1274304753 send_complete cg 4 flags 2 counts 2 2 1 0 0
>1274304753 receive_complete 1:4 len 192
>1274304824 cluster node 4 added
>1274304826 fenced:daemon conf 3 1 0 memb 1 3 4 join 4 left
>1274304826 receive_protocol from 3 max 1.1.1.0 run 1.1.1.1
>1274304826 daemon node 3 max 1.1.1.0 run 1.1.1.0
>1274304826 daemon node 3 join 1274304749 left 1274304702 local quorum 1274304741
>1274304826 receive_protocol from 4 max 1.1.1.0 run 0.0.0.0
>1274304826 daemon node 4 max 0.0.0.0 run 0.0.0.0
>1274304826 daemon node 4 join 1274304826 left 1274304702 local quorum 1274304741
>1274304826 receive_protocol from 1 max 1.1.1.0 run 1.1.1.1
>1274304826 daemon node 1 max 1.1.1.0 run 1.1.1.1
>1274304826 daemon node 1 join 1274304342 left 0 local quorum 1274304741
>1274304826 receive_protocol from 4 max 1.1.1.0 run 1.1.1.0
>1274304826 daemon node 4 max 1.1.1.0 run 0.0.0.0
>1274304826 daemon node 4 join 1274304826 left 1274304702 local quorum 1274304741
>1274304828 fenced:default conf 3 1 0 memb 1 3 4 join 4 left
>1274304828 add_change cg 5 joined nodeid 4
>1274304828 add_change cg 5 counts member 3 joined 1 remove 0 failed 0
>1274304828 check_quorum done
>1274304828 send_start cg 5 flags 2 counts 3 3 1 0 0
>1274304828 receive_start 3:2 len 192
>1274304828 match_change 3:2 matches cg 5
>1274304828 wait_messages cg 5 need 2 of 3
>1274304828 receive_start 1:5 len 192
>1274304828 match_change 1:5 matches cg 5
>1274304828 wait_messages cg 5 need 1 of 3
>1274304828 receive_start 4:1 len 192
>1274304828 match_change 4:1 matches cg 5
>1274304828 wait_messages cg 5 got all 3
>1274304828 set_master from 1 to complete node 1
>1274304828 send_complete cg 5 flags 2 counts 3 3 1 0 0
>1274304828 receive_complete 1:5 len 192

Actions: View

Attachments on bug 594511: 415521 | 415522 | 415523 | 415524 | 415525