Attachment 310272 Details for Bug 452121 – log (with extra debugging) from taft-03 when this happened

log (with extra debugging) from taft-03 when this happened

taft-03.log (text/plain), 13.78 KB, created by Corey Marthaler on 2008-06-25 14:47:25 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Corey Marthaler

Created: 2008-06-25 14:47:25 UTC

Size: 13.78 KB

patch

obsolete

>Jun 25 05:45:38 taft-03 kernel: dm-cmirror: Node joining
>Jun 25 05:45:43 taft-03 kernel: dm-cmirror: cluster_resume: Setting recovery_halted = 0
>Jun 25 05:45:43 taft-03 kernel: dm-cmirror: cluster_resume[lOsOYeAK]:: in_sync = 0
>Jun 25 05:45:43 taft-03 kernel: dm-cmirror: server_id=dead, server_valid=0, lOsOYeAK
>Jun 25 05:45:43 taft-03 kernel: dm-cmirror: trigger = LRT_GET_RESYNC_WORK
>Jun 25 05:45:43 taft-03 kernel: dm-cmirror: LRT_ELECTION(10): (lOsOYeAK)
>Jun 25 05:45:43 taft-03 kernel: dm-cmirror:   starter     : 4
>Jun 25 05:45:43 taft-03 kernel: dm-cmirror:   co-ordinator: 4
>Jun 25 05:45:43 taft-03 kernel: dm-cmirror:   node_count  : 0
>Jun 25 05:45:43 taft-03 [540]: Monitoring mirror device helter_skelter-syncd_primary_core_2legs_1 for events 
>Jun 25 05:45:43 taft-03 lvm[540]: Loading config file: /etc/lvm/lvm.conf
>Jun 25 05:45:43 taft-03 lvm[540]: Opened /etc/lvm/lvm.conf RO
>Jun 25 05:45:43 taft-03 lvm[540]: Closed /etc/lvm/lvm.conf
>Jun 25 05:45:43 taft-03 lvm[540]: Setting log/syslog to 1
>Jun 25 05:45:43 taft-03 lvm[540]: Setting log/level to 0
>Jun 25 05:45:43 taft-03 lvm[540]: Setting log/verbose to 0
>Jun 25 05:45:43 taft-03 lvm[540]: Setting log/indent to 1
>Jun 25 05:45:43 taft-03 lvm[540]: Setting log/prefix to   
>Jun 25 05:45:43 taft-03 lvm[540]: Setting log/command_names to 0
>Jun 25 05:45:43 taft-03 lvm[540]: Setting global/test to 0
>Jun 25 05:45:43 taft-03 lvm[540]: Setting log/overwrite to 0
>Jun 25 05:45:43 taft-03 lvm[540]: log/activation not found in config: defaulting to 0
>Jun 25 05:45:43 taft-03 lvm[540]: Logging initialised at Wed Jun 25 05:45:43 2008
>Jun 25 05:45:43 taft-03 lvm[540]: Setting global/umask to 63
>Jun 25 05:45:43 taft-03 lvm[540]: Setting devices/dir to /dev
>Jun 25 05:45:43 taft-03 lvm[540]: Setting global/proc to /proc
>Jun 25 05:45:43 taft-03 lvm[540]: Setting global/activation to 1
>Jun 25 05:45:43 taft-03 lvm[540]: global/suffix not found in config: defaulting to 1
>Jun 25 05:45:43 taft-03 lvm[540]: Setting global/units to h
>Jun 25 05:45:43 taft-03 lvm[540]: Setting activation/readahead to auto
>Jun 25 05:45:43 taft-03 lvm[540]: devices/preferred_names not found in config file: using built-in preferences
>Jun 25 05:45:43 taft-03 lvm[540]: Matcher built with 3 dfa states
>Jun 25 05:45:43 taft-03 lvm[540]: Setting devices/ignore_suspended_devices to 0
>Jun 25 05:45:43 taft-03 lvm[540]: Setting devices/cache_dir to /etc/lvm/cache
>Jun 25 05:45:43 taft-03 lvm[540]: Setting devices/write_cache_state to 1
>Jun 25 05:45:43 taft-03 lvm[540]: Opened /etc/lvm/cache/.cache RO
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0003000000000000-part1: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram11: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/VolGroup00/LogVol01: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram10: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0004000000000000-part1: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/VolGroup00/LogVol00: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram12: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram6: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0005000000000000-part1: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram13: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram14: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0007000000000000-part1: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram5: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/mapper/VolGroup00-LogVol00: Aliased to /dev/VolGroup00/LogVol00 in device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/disk/by-path/pci-0000:02:0e.0-scsi-0:2:0:0-part1: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram1: Aliased to /dev/ram in device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/sda1: Aliased to /dev/disk/by-path/pci-0000:02:0e.0-scsi-0:2:0:0-part1 in device cache (preferred name)
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/sdb1: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/sdc1: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/sdg1: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/sdd1: Aliased to /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0003000000000000-part1 in device cache (preferred name)
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/sde1: Aliased to /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0004000000000000-part1 in device cache (preferred name)
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/sdf1: Aliased to /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0005000000000000-part1 in device cache (preferred name)
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/sdh1: Aliased to /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0007000000000000-part1 in device cache (preferred name)
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ramdisk: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/mapper/VolGroup00-LogVol01: Aliased to /dev/VolGroup00/LogVol01 in device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram0: Aliased to /dev/ramdisk in device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/root: Aliased to /dev/VolGroup00/LogVol00 in device cache (preferred name)
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/dm-1: Aliased to /dev/VolGroup00/LogVol01 in device cache (preferred name)
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/disk/by-path/pci-0000:02:0e.0-scsi-0:2:0:0-part2: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram2: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/sda2: Aliased to /dev/disk/by-path/pci-0000:02:0e.0-scsi-0:2:0:0-part2 in device cache (preferred name)
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram8: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram9: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/dm-0: Aliased to /dev/root in device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram15: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0001000000000000-part1: Aliased to /dev/sdb1 in device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0006000000000000-part1: Aliased to /dev/sdg1 in device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram3: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram4: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/ram7: Added to device cache
>Jun 25 05:45:43 taft-03 lvm[540]: /dev/disk/by-path/pci-0000:0b:02.0-fc-0x500805f3000a05b1:0x0002000000000000-part1: Aliased to /dev/sdc1 in device cache
>Jun 25 05:45:43 taft-03 lvm[540]: Loaded persistent filter cache from /etc/lvm/cache/.cache
>Jun 25 05:45:43 taft-03 lvm[540]: Closed /etc/lvm/cache/.cache
>Jun 25 05:45:43 taft-03 lvm[540]: Setting activation/reserved_stack to 256
>Jun 25 05:45:43 taft-03 lvm[540]: Setting activation/reserved_memory to 8192
>Jun 25 05:45:43 taft-03 lvm[540]: Setting activation/process_priority to -18
>Jun 25 05:45:43 taft-03 lvm[540]: Initialised format: lvm1
>Jun 25 05:45:43 taft-03 lvm[540]: Initialised format: pool
>Jun 25 05:45:43 taft-03 lvm[540]: Initialised format: lvm2
>Jun 25 05:45:43 taft-03 lvm[540]: global/format not found in config: defaulting to lvm2
>Jun 25 05:45:43 taft-03 lvm[540]: lvmcache: initialised VG #orphans_lvm1
>Jun 25 05:45:43 taft-03 lvm[540]: lvmcache: initialised VG #orphans_pool
>Jun 25 05:45:43 taft-03 lvm[540]: lvmcache: initialised VG #orphans_lvm2
>Jun 25 05:45:43 taft-03 lvm[540]: Initialised segtype: striped
>Jun 25 05:45:43 taft-03 lvm[540]: Initialised segtype: zero
>Jun 25 05:45:43 taft-03 lvm[540]: Initialised segtype: error
>Jun 25 05:45:43 taft-03 lvm[540]: Setting dmeventd/snapshot_library to libdevmapper-event-lvm2snapshot.so
>Jun 25 05:45:43 taft-03 lvm[540]: Setting global/library_dir to /usr/lib64
>Jun 25 05:45:43 taft-03 lvm[540]: Initialised segtype: snapshot
>Jun 25 05:45:43 taft-03 lvm[540]: Initialised segtype: mirror
>Jun 25 05:45:43 taft-03 lvm[540]: Setting backup/retain_days to 30
>Jun 25 05:45:43 taft-03 lvm[540]: Setting backup/retain_min to 10
>Jun 25 05:45:43 taft-03 lvm[540]: Setting backup/archive_dir to /etc/lvm/archive
>Jun 25 05:45:43 taft-03 lvm[540]: Setting backup/backup_dir to /etc/lvm/backup
>Jun 25 05:45:44 taft-03 lvm[540]: Locking memory
>Jun 25 05:45:44 taft-03 lvm[540]: memlock_count inc to 1
>Jun 25 05:45:44 taft-03 lvm[540]: Parsing: _memlock_inc
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: cluster_ctr[klbK4rk8]:: in_sync = -1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: Creating klbK4rk8 (1)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: cluster_resume: Setting recovery_halted = 0
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: cluster_resume[klbK4rk8]:: in_sync = 0
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: server_id=dead, server_valid=0, klbK4rk8
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: trigger = LRT_GET_SYNC_COUNT
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_ELECTION(10): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 4
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 4
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 0
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_ELECTION(10): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 2
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 2
>Jun 25 05:45:44 taft-03 lvm[540]: Monitoring mirror device helter_skelter-syncd_primary_core_2legs_2 for events 
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 2
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_ELECTION(10): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 3
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 3
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_ELECTION(10): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 4
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 2
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 4
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_SELECTION(11): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 2
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 2
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_SELECTION(11): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 3
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_SELECTION(11): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 4
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 2
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 4
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_ELECTION(10): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 3
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 2
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 3
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 1
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (klbK4rk8)
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   starter     : 4
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   co-ordinator: 2
>Jun 25 05:45:44 taft-03 kernel: dm-cmirror:   node_count  : 1
>Jun 25 07:51:50 taft-03 dhclient: DHCPREQUEST on eth0 to 10.15.89.100 port 67
>Jun 25 07:51:51 taft-03 dhclient: DHCPACK from 10.15.89.100
>Jun 25 07:51:51 taft-03 dhclient: bound to 10.15.89.69 -- renewal in 10465 seconds.
>Jun 25 08:26:04 taft-03 kernel: dm-cmirror: cluster_get_sync_count[lOsOYeAK]:: in_sync = 1
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror: LOG INFO:
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   uuid: LVM-ccA6T7OErxuQRuVb3RosU03BbB407YRAI5i3TccRQM5ZHWXdFi4hXfmClOsOYeAK
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   uuid_ref    : 1
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   log type    : core
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:  ?region_count: 1600
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:  ?sync_count  : 0
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:  ?sync_search : 0
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   in_sync     : YES
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   suspended   : NO
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   recovery_halted : NO
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   server_id   : 1
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   server_valid: YES
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror: LOG INFO:
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   uuid: LVM-ccA6T7OErxuQRuVb3RosU03BbB407YRAJ00pgD1Q4Dky2Ir16lIXCjIaklbK4rk8
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   uuid_ref    : 1
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   log type    : core
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:  ?region_count: 1600
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:  ?sync_count  : 0
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:  ?sync_search : 0
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   in_sync     : NO
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   suspended   : NO
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   recovery_halted : NO
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   server_id   : 2
>Jun 25 08:32:43 taft-03 kernel: dm-cmirror:   server_valid: YES

Actions: View

Attachments on bug 452121: 309850 | 310272 | 310273 | 310277 | 310279