Attachment 155370 Details for Bug 241248 – attaching full log from link-02

attaching full log from link-02

link-02 (text/plain), 211.97 KB, created by Corey Marthaler on 2007-05-24 17:43:12 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Corey Marthaler

Created: 2007-05-24 17:43:12 UTC

Size: 211.97 KB

patch

obsolete

>May 24 10:35:13 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:13 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:13 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3OxG0mO6)
>May 24 10:35:13 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:13 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:13 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:14 link-02 kernel: dm-cmirror: Removing 3OxG0mO6 (1)
>May 24 10:35:14 link-02 kernel: dm-cmirror: 0 region user structures freed
>May 24 10:35:14 link-02 qarshd[9530]: Nothing to do 
>May 24 10:35:17 link-02 qarshd[9530]: That's enough 
>May 24 10:35:17 link-02 qarshd[9609]: Talking to peer 10.15.80.47:39922
>May 24 10:35:17 link-02 qarshd[9609]: Running cmdline: lvs lock_stress/link-02.26406 --noheadings -o mirror_log | awk {'print $1'} 
>May 24 10:35:17 link-02 kernel: dm-cmirror: LOG INFO:
>May 24 10:35:17 link-02 kernel: dm-cmirror:   uuid: LVM-rRf2kwFdXA2tYF0WVAkkwbHFhTtzweLFU6swb0bAECh7zvYu3IRUt0zj3MYLUzK0
>May 24 10:35:17 link-02 kernel: dm-cmirror:   uuid_ref    : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror:  ?region_count: 1000
>May 24 10:35:17 link-02 kernel: dm-cmirror:  ?sync_count  : 0
>May 24 10:35:17 link-02 kernel: dm-cmirror:  ?sync_search : 0
>May 24 10:35:17 link-02 kernel: dm-cmirror:   in_sync     : NO
>May 24 10:35:17 link-02 kernel: dm-cmirror:   suspended   : NO
>May 24 10:35:17 link-02 kernel: dm-cmirror:   server_id   : 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   server_valid: YES
>May 24 10:35:17 link-02 kernel: dm-cmirror: Creating 3MYLUzK0 (2)
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:17 link-02 lvm[5604]: No longer monitoring mirror device lock_stress-link--07.26410 for events 
>May 24 10:35:17 link-02 kernel: dm-cmirror: server_id=dead, server_valid=0, 3MYLUzK0
>May 24 10:35:17 link-02 lvm[5604]: Unlocking memory
>May 24 10:35:17 link-02 kernel: dm-cmirror: trigger = LRT_GET_SYNC_COUNT
>May 24 10:35:17 link-02 lvm[5604]: memlock_count dec to 0
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:17 link-02 lvm[5604]: Dumping persistent device cache to /etc/lvm/.cache
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:17 link-02 lvm[5604]: Locking /etc/lvm/.cache (F_WRLCK, 1)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:35:17 link-02 lvm[5604]: Opened /etc/lvm/.cache RO
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:35:17 link-02 lvm[5604]: Loaded persistent filter cache from /etc/lvm/.cache
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (3MYLUzK0)
>May 24 10:35:17 link-02 lvm[5604]: Unlocking fd 8
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:17 link-02 lvm[5604]: Closed /etc/lvm/.cache
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:35:17 link-02 lvm[5604]: Wiping internal VG cache
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:17 link-02 kernel: dm-cmirror: I'm the cluster mirror log server for 3MYLUzK0
>May 24 10:35:17 link-02 kernel: dm-cmirror: Not reading disk log because I'm suspended (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror: Performing flush to work around bug 235040
>May 24 10:35:17 link-02 kernel: dm-cmirror: Log flush complete
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:17 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3MYLUzK0)
>May 24 10:35:17 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:17 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:17 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:18 link-02 kernel: dm-cmirror: Removing 3MYLUzK0 (1)
>May 24 10:35:18 link-02 kernel: dm-cmirror: 0 region user structures freed
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 0/3MYLUzK0
>May 24 10:35:18 link-02 [5604]: Monitoring mirror device lock_stress-link--07.26410 for events 
>May 24 10:35:18 link-02 lvm[5604]: Loading config file: /etc/lvm/lvm.conf
>May 24 10:35:18 link-02 lvm[5604]: Opened /etc/lvm/lvm.conf RO
>May 24 10:35:18 link-02 lvm[5604]: Closed /etc/lvm/lvm.conf
>May 24 10:35:18 link-02 lvm[5604]: Setting log/syslog to 1
>May 24 10:35:18 link-02 lvm[5604]: Setting log/level to 0
>May 24 10:35:18 link-02 lvm[5604]: Setting log/verbose to 0
>May 24 10:35:18 link-02 lvm[5604]: Setting log/indent to 1
>May 24 10:35:18 link-02 lvm[5604]: Setting log/prefix to   
>May 24 10:35:18 link-02 lvm[5604]: Setting log/command_names to 0
>May 24 10:35:18 link-02 lvm[5604]: Setting global/test to 0
>May 24 10:35:18 link-02 lvm[5604]: Setting log/overwrite to 0
>May 24 10:35:18 link-02 lvm[5604]: log/activation not found in config: defaulting to 0
>May 24 10:35:18 link-02 lvm[5604]: Logging initialised at Thu May 24 10:35:18 2007
>May 24 10:35:18 link-02 lvm[5604]: Setting global/umask to 63
>May 24 10:35:18 link-02 lvm[5604]: Setting devices/dir to /dev
>May 24 10:35:18 link-02 lvm[5604]: Setting global/proc to /proc
>May 24 10:35:18 link-02 lvm[5604]: Setting global/activation to 1
>May 24 10:35:18 link-02 lvm[5604]: global/suffix not found in config: defaulting to 1
>May 24 10:35:18 link-02 lvm[5604]: global/units not found in config: defaulting to h
>May 24 10:35:18 link-02 lvm[5604]: Matcher built with 11 dfa states
>May 24 10:35:18 link-02 lvm[5604]: Setting devices/ignore_suspended_devices to 0
>May 24 10:35:18 link-02 lvm[5604]: Setting devices/cache to /etc/lvm/.cache
>May 24 10:35:18 link-02 lvm[5604]: Setting devices/write_cache_state to 1
>May 24 10:35:18 link-02 lvm[5604]: Opened /etc/lvm/.cache RO
>May 24 10:35:18 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0001000000000000-part1: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x3000910891003200:0x0001000000000000-part1: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram11: Added to device cache
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 0
>May 24 10:35:18 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part1: Added to device cache
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 1
>May 24 10:35:18 link-02 lvm[5604]: /dev/VolGroup00/LogVol01: Added to device cache
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 1/3MYLUzK0
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram10: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/VolGroup00/LogVol00: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0004000000000000-part1: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0006000000000000-part1: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0005000000000000-part1: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram12: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part2: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram6: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0007000000000000-part1: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram13: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram14: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram5: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/mapper/VolGroup00-LogVol00: Aliased to /dev/VolGroup00/LogVol00 in device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/hda1: Aliased to /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part1 in device cache (preferred name)
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram1: Aliased to /dev/ram in device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/sda1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0001000000000000-part1 in device cache (preferred name)
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 1
>May 24 10:35:18 link-02 lvm[5604]: /dev/sdb1: Added to device cache
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 2
>May 24 10:35:18 link-02 lvm[5604]: /dev/sdc1: Added to device cache
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 2/3MYLUzK0
>May 24 10:35:18 link-02 lvm[5604]: /dev/sdg1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0007000000000000-part1 in device cache (preferred name)
>May 24 10:35:18 link-02 lvm[5604]: /dev/sdd1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0004000000000000-part1 in device cache (preferred name)
>May 24 10:35:18 link-02 lvm[5604]: /dev/sde1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0005000000000000-part1 in device cache (preferred name)
>May 24 10:35:18 link-02 lvm[5604]: /dev/sdf1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0006000000000000-part1 in device cache (preferred name)
>May 24 10:35:18 link-02 lvm[5604]: /dev/sdh1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x3000910891003200:0x0001000000000000-part1 in device cache (preferred name)
>May 24 10:35:18 link-02 lvm[5604]: /dev/ramdisk: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0002000000000000-part1: Aliased to /dev/sdb1 in device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/mapper/VolGroup00-LogVol01: Aliased to /dev/VolGroup00/LogVol01 in device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram0: Aliased to /dev/ramdisk in device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0003000000000000-part1: Aliased to /dev/sdc1 in device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/root: Aliased to /dev/VolGroup00/LogVol00 in device cache (preferred name)
>May 24 10:35:18 link-02 lvm[5604]: /dev/dm-1: Aliased to /dev/VolGroup00/LogVol01 in device cache (preferred name)
>May 24 10:35:18 link-02 lvm[5604]: /dev/hda2: Aliased to /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part2 in device cache (preferred name)
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram2: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram8: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram9: Added to device cache
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 2
>May 24 10:35:18 link-02 lvm[5604]: /dev/dm-0: Aliased to /dev/root in device cache
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 3
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram15: Added to device cache
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 3/3MYLUzK0
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram3: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram4: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: /dev/ram7: Added to device cache
>May 24 10:35:18 link-02 lvm[5604]: Loaded persistent filter cache from /etc/lvm/.cache
>May 24 10:35:18 link-02 lvm[5604]: Closed /etc/lvm/.cache
>May 24 10:35:18 link-02 lvm[5604]: Setting activation/reserved_stack to 256
>May 24 10:35:18 link-02 lvm[5604]: Setting activation/reserved_memory to 8192
>May 24 10:35:18 link-02 lvm[5604]: Setting activation/process_priority to -18
>May 24 10:35:18 link-02 lvm[5604]: Initialised format: lvm1
>May 24 10:35:18 link-02 lvm[5604]: Initialised format: pool
>May 24 10:35:18 link-02 lvm[5604]: Initialised format: lvm2
>May 24 10:35:18 link-02 lvm[5604]: global/format not found in config: defaulting to lvm2
>May 24 10:35:18 link-02 lvm[5604]: Initialised segtype: striped
>May 24 10:35:18 link-02 lvm[5604]: Initialised segtype: zero
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 3
>May 24 10:35:18 link-02 lvm[5604]: Initialised segtype: error
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 4
>May 24 10:35:18 link-02 lvm[5604]: Initialised segtype: snapshot
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 4/3MYLUzK0
>May 24 10:35:18 link-02 lvm[5604]: Initialised segtype: mirror
>May 24 10:35:18 link-02 lvm[5604]: Setting backup/retain_days to 30
>May 24 10:35:18 link-02 lvm[5604]: Setting backup/retain_min to 10
>May 24 10:35:18 link-02 lvm[5604]: Setting backup/archive_dir to /etc/lvm/archive
>May 24 10:35:18 link-02 lvm[5604]: Setting backup/backup_dir to /etc/lvm/backup
>May 24 10:35:18 link-02 lvm[5604]: Locking memory
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 4
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 5
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 5/3MYLUzK0
>May 24 10:35:18 link-02 lvm[5604]: memlock_count inc to 1
>May 24 10:35:18 link-02 lvm[5604]: Parsing: _memlock_inc
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 5
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 6
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 6/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 6
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 7
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 7/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 7
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 8
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 8/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 8
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 9
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 9/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 9
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 10
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 10/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 10
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 11
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 11/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 11
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 12
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 12/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 12
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 13
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 13/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 13
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 14
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 14/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 14
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 15
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 15/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 15
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 16
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 16/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 16
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 17
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 17/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 17
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 18
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 18/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 18
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 19
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 19/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 19
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 20
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 20/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 20
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 21
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 21/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 21
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 22
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 22/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 22
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 23
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 23/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 23
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 24
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 24/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 24
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 25
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 25/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 25
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 26
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 26/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 26
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 27
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 27/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 27
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 28
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 28/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 28
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 29
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 29/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 29
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 30
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 30/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 30
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 31
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 31/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 31
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 32
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 32/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 32
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 33
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 33/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 33
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 34
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 34/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 34
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 35
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 35/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 35
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 36
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 36/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 36
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 37
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 37/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 37
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 38
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 38/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 38
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 39
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 39/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 39
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 40
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 40/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 40
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 41
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 41/3MYLUzK0
>May 24 10:35:18 link-02 kernel: dm-cmirror: Resync work completed: 41
>May 24 10:35:18 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 42
>May 24 10:35:18 link-02 kernel: dm-cmirror: Client received resync work: 42/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 42
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 43
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 43/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 43
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 44
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 44/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 44
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 45
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 45/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 45
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 46
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 46/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 46
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 47
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 47/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 47
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 48
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 48/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 48
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 49
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 49/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 49
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 50
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 50/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 50
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 51
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 51/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 51
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 52
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 52/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 52
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 53
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 53/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 53
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 54
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 54/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 54
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 55
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 55/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 55
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 56
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 56/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 56
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 57
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 57/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 57
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 58
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 58/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 58
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 59
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 59/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 59
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 60
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 60/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 60
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 61
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 61/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 61
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 62
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 62/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 62
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 63
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 63/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 63
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 64
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 64/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 64
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 65
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 65/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 65
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 66
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 66/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 66
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 67
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 67/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 67
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 68
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 68/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 68
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 69
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 69/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 69
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 70
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 70/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 70
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 71
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 71/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 71
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 72
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 72/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 72
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 73
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 73/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 73
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 74
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 74/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 74
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 75
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 75/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 75
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 76
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 76/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 76
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 77
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 77/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 77
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 78
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 78/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 78
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 79
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 79/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 79
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 80
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 80/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 80
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 81
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 81/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 81
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 82
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 82/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 82
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 83
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 83/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 83
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 84
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 84/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 84
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 85
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 85/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 85
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 86
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 86/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 86
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 87
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 87/3MYLUzK0
>May 24 10:35:19 link-02 kernel: dm-cmirror: Resync work completed: 87
>May 24 10:35:19 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 88
>May 24 10:35:19 link-02 kernel: dm-cmirror: Client received resync work: 88/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 88
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 89
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 89/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 89
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 90
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 90/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 90
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 91
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 91/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 91
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 92
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 92/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 92
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 93
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 93/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 93
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 94
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 94/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 94
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 95
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 95/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 95
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 96
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 96/3MYLUzK0
>May 24 10:35:20 link-02 qarshd[9609]: Nothing to do 
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 96
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 97
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 97/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 97
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 98
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 98/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 98
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 99
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 99/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 99
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 100
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 100/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 100
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 101
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 101/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 101
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 102
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 102/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 102
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 103
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 103/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 103
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 104
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 104/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 104
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 105
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 105/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 105
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 106
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 106/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 106
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 107
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 107/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 107
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 108
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 108/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 108
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 109
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 109/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 109
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 110
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 110/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 110
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 111
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 111/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 111
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 112
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 112/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 112
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 113
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 113/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 113
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 114
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 114/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 114
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 115
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 115/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 115
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 116
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 116/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 116
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 117
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 117/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 117
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 118
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 118/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 118
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 119
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 119/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 119
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 120
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 120/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 120
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 121
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 121/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 121
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 122
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 122/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 122
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 123
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 123/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 123
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 124
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 124/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 124
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 125
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 125/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 125
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 126
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 126/3MYLUzK0
>May 24 10:35:20 link-02 kernel: dm-cmirror: Resync work completed: 126
>May 24 10:35:20 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 127
>May 24 10:35:20 link-02 kernel: dm-cmirror: Client received resync work: 127/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 127
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 128
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 128/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 128
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 129
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 129/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 129
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 130
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 130/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 130
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 131
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 131/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 131
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 132
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 132/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 132
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 133
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 133/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 133
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 134
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 134/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 134
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 135
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 135/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 135
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 136
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 136/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 136
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 137
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 137/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 137
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 138
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 138/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 138
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 139
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 139/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 139
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 140
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 140/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 140
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 141
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 141/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 141
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 142
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 142/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 142
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 143
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 143/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 143
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 144
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 144/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 144
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 145
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 145/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 145
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 146
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 146/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 146
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 147
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 147/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 147
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 148
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 148/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: LOG INFO:
>May 24 10:35:21 link-02 kernel: dm-cmirror:   uuid: LVM-rRf2kwFdXA2tYF0WVAkkwbHFhTtzweLFU6swb0bAECh7zvYu3IRUt0zj3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   uuid_ref    : 2
>May 24 10:35:21 link-02 kernel: dm-cmirror:  ?region_count: 1000
>May 24 10:35:21 link-02 kernel: dm-cmirror:  ?sync_count  : 148
>May 24 10:35:21 link-02 kernel: dm-cmirror:  ?sync_search : 149
>May 24 10:35:21 link-02 kernel: dm-cmirror:   in_sync     : NO
>May 24 10:35:21 link-02 kernel: dm-cmirror:   suspended   : NO
>May 24 10:35:21 link-02 kernel: dm-cmirror:   server_id   : 3
>May 24 10:35:21 link-02 kernel: dm-cmirror:   server_valid: YES
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 148
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 149
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 149/3MYLUzK0
>May 24 10:35:21 link-02 lvm[5604]: No longer monitoring mirror device lock_stress-link--07.26410 for events 
>May 24 10:35:21 link-02 lvm[5604]: Unlocking memory
>May 24 10:35:21 link-02 lvm[5604]: memlock_count dec to 0
>May 24 10:35:21 link-02 lvm[5604]: Dumping persistent device cache to /etc/lvm/.cache
>May 24 10:35:21 link-02 lvm[5604]: Locking /etc/lvm/.cache (F_WRLCK, 1)
>May 24 10:35:21 link-02 lvm[5604]: Unlocking fd 8
>May 24 10:35:21 link-02 lvm[5604]: Wiping internal VG cache
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 149
>May 24 10:35:21 link-02 kernel: dm-cmirror: Assigning recovery work to 3/3MYLUzK0: 150
>May 24 10:35:21 link-02 kernel: dm-cmirror: Client received resync work: 150/3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:21 link-02 kernel: dm-cmirror: Resync work completed: 150
>May 24 10:35:21 link-02 kernel: dm-cmirror: Performing flush to work around bug 235040
>May 24 10:35:21 link-02 kernel: dm-cmirror: Log flush complete
>May 24 10:35:21 link-02 kernel: dm-cmirror: Telling everyone I'm suspending (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:35:21 link-02 kernel: dm-cmirror: Suspending now (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3MYLUzK0)
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:21 link-02 kernel: dm-cmirror: LOG INFO:
>May 24 10:35:21 link-02 kernel: dm-cmirror:   uuid: LVM-rRf2kwFdXA2tYF0WVAkkwbHFhTtzweLFU6swb0bAECh7zvYu3IRUt0zj3MYLUzK0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   uuid_ref    : 2
>May 24 10:35:21 link-02 kernel: dm-cmirror:  ?region_count: 1000
>May 24 10:35:21 link-02 kernel: dm-cmirror:  ?sync_count  : 151
>May 24 10:35:21 link-02 kernel: dm-cmirror:  ?sync_search : 151
>May 24 10:35:21 link-02 kernel: dm-cmirror:   in_sync     : NO
>May 24 10:35:21 link-02 [5604]: Monitoring mirror device lock_stress-link--07.26410 for events 
>May 24 10:35:21 link-02 kernel: dm-cmirror:   suspended   : YES
>May 24 10:35:21 link-02 lvm[5604]: Loading config file: /etc/lvm/lvm.conf
>May 24 10:35:21 link-02 kernel: dm-cmirror:   server_id   : 57005
>May 24 10:35:21 link-02 lvm[5604]: Opened /etc/lvm/lvm.conf RO
>May 24 10:35:21 link-02 kernel: dm-cmirror:   server_valid: NO
>May 24 10:35:21 link-02 lvm[5604]: Closed /etc/lvm/lvm.conf
>May 24 10:35:21 link-02 kernel: dm-cmirror: server_id=dead, server_valid=1, 3MYLUzK0
>May 24 10:35:21 link-02 lvm[5604]: Setting log/syslog to 1
>May 24 10:35:21 link-02 kernel: dm-cmirror: trigger = LRT_GET_RESYNC_WORK
>May 24 10:35:21 link-02 lvm[5604]: Setting log/level to 0
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:21 link-02 lvm[5604]: Setting log/verbose to 0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:21 link-02 lvm[5604]: Setting log/indent to 1
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:35:21 link-02 lvm[5604]: Setting log/prefix to   
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:35:21 link-02 lvm[5604]: Setting log/command_names to 0
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:21 link-02 lvm[5604]: Setting global/test to 0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:21 link-02 lvm[5604]: Setting log/overwrite to 0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:35:21 link-02 lvm[5604]: log/activation not found in config: defaulting to 0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:21 link-02 lvm[5604]: Logging initialised at Thu May 24 10:35:21 2007
>May 24 10:35:21 link-02 lvm[5604]: Setting global/umask to 63
>May 24 10:35:21 link-02 lvm[5604]: Setting devices/dir to /dev
>May 24 10:35:21 link-02 lvm[5604]: Setting global/proc to /proc
>May 24 10:35:21 link-02 lvm[5604]: Setting global/activation to 1
>May 24 10:35:21 link-02 lvm[5604]: global/suffix not found in config: defaulting to 1
>May 24 10:35:21 link-02 lvm[5604]: global/units not found in config: defaulting to h
>May 24 10:35:21 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:21 link-02 lvm[5604]: Matcher built with 11 dfa states
>May 24 10:35:21 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:21 link-02 lvm[5604]: Setting devices/ignore_suspended_devices to 0
>May 24 10:35:21 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:21 link-02 lvm[5604]: Setting devices/cache to /etc/lvm/.cache
>May 24 10:35:21 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:21 link-02 lvm[5604]: Setting devices/write_cache_state to 1
>May 24 10:35:21 link-02 lvm[5604]: Opened /etc/lvm/.cache RO
>May 24 10:35:21 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0001000000000000-part1: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x3000910891003200:0x0001000000000000-part1: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/ram11: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part1: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/VolGroup00/LogVol01: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/ram10: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/ram: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/VolGroup00/LogVol00: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0004000000000000-part1: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0006000000000000-part1: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0005000000000000-part1: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/ram12: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part2: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/ram6: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0007000000000000-part1: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/ram13: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/ram14: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/ram5: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/mapper/VolGroup00-LogVol00: Aliased to /dev/VolGroup00/LogVol00 in device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/hda1: Aliased to /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part1 in device cache (preferred name)
>May 24 10:35:21 link-02 lvm[5604]: /dev/ram1: Aliased to /dev/ram in device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/sda1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0001000000000000-part1 in device cache (preferred name)
>May 24 10:35:21 link-02 lvm[5604]: /dev/sdb1: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/sdc1: Added to device cache
>May 24 10:35:21 link-02 lvm[5604]: /dev/sdg1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0007000000000000-part1 in device cache (preferred name)
>May 24 10:35:21 link-02 lvm[5604]: /dev/sdd1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0004000000000000-part1 in device cache (preferred name)
>May 24 10:35:21 link-02 lvm[5604]: /dev/sde1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0005000000000000-part1 in device cache (preferred name)
>May 24 10:35:22 link-02 lvm[5604]: /dev/sdf1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0006000000000000-part1 in device cache (preferred name)
>May 24 10:35:22 link-02 lvm[5604]: /dev/sdh1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x3000910891003200:0x0001000000000000-part1 in device cache (preferred name)
>May 24 10:35:22 link-02 lvm[5604]: /dev/ramdisk: Added to device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0002000000000000-part1: Aliased to /dev/sdb1 in device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/mapper/VolGroup00-LogVol01: Aliased to /dev/VolGroup00/LogVol01 in device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/ram0: Aliased to /dev/ramdisk in device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0003000000000000-part1: Aliased to /dev/sdc1 in device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/root: Aliased to /dev/VolGroup00/LogVol00 in device cache (preferred name)
>May 24 10:35:22 link-02 lvm[5604]: /dev/dm-1: Aliased to /dev/VolGroup00/LogVol01 in device cache (preferred name)
>May 24 10:35:22 link-02 lvm[5604]: /dev/hda2: Aliased to /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part2 in device cache (preferred name)
>May 24 10:35:22 link-02 lvm[5604]: /dev/ram2: Added to device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/ram8: Added to device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/ram9: Added to device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/dm-0: Aliased to /dev/root in device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/ram15: Added to device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/ram3: Added to device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/ram4: Added to device cache
>May 24 10:35:22 link-02 lvm[5604]: /dev/ram7: Added to device cache
>May 24 10:35:22 link-02 lvm[5604]: Loaded persistent filter cache from /etc/lvm/.cache
>May 24 10:35:22 link-02 lvm[5604]: Closed /etc/lvm/.cache
>May 24 10:35:22 link-02 lvm[5604]: Setting activation/reserved_stack to 256
>May 24 10:35:22 link-02 lvm[5604]: Setting activation/reserved_memory to 8192
>May 24 10:35:22 link-02 lvm[5604]: Setting activation/process_priority to -18
>May 24 10:35:22 link-02 lvm[5604]: Initialised format: lvm1
>May 24 10:35:22 link-02 lvm[5604]: Initialised format: pool
>May 24 10:35:22 link-02 lvm[5604]: Initialised format: lvm2
>May 24 10:35:22 link-02 lvm[5604]: global/format not found in config: defaulting to lvm2
>May 24 10:35:22 link-02 lvm[5604]: Initialised segtype: striped
>May 24 10:35:22 link-02 lvm[5604]: Initialised segtype: zero
>May 24 10:35:22 link-02 lvm[5604]: Initialised segtype: error
>May 24 10:35:22 link-02 lvm[5604]: Initialised segtype: snapshot
>May 24 10:35:22 link-02 lvm[5604]: Initialised segtype: mirror
>May 24 10:35:22 link-02 lvm[5604]: Setting backup/retain_days to 30
>May 24 10:35:22 link-02 lvm[5604]: Setting backup/retain_min to 10
>May 24 10:35:22 link-02 lvm[5604]: Setting backup/archive_dir to /etc/lvm/archive
>May 24 10:35:22 link-02 lvm[5604]: Setting backup/backup_dir to /etc/lvm/backup
>May 24 10:35:22 link-02 lvm[5604]: Locking memory
>May 24 10:35:22 link-02 lvm[5604]: memlock_count inc to 1
>May 24 10:35:22 link-02 lvm[5604]: Parsing: _memlock_inc
>May 24 10:35:22 link-02 lvm[5604]: lock_stress-link--07.26410 is now in-sync 
>May 24 10:35:22 link-02 qarshd[9609]: That's enough 
>May 24 10:35:22 link-02 kernel: dm-cmirror: LOG INFO:
>May 24 10:35:22 link-02 kernel: dm-cmirror:   uuid: LVM-rRf2kwFdXA2tYF0WVAkkwbHFhTtzweLFU6swb0bAECh7zvYu3IRUt0zj3MYLUzK0
>May 24 10:35:22 link-02 kernel: dm-cmirror:   uuid_ref    : 2
>May 24 10:35:22 link-02 kernel: dm-cmirror:  ?region_count: 1000
>May 24 10:35:22 link-02 kernel: dm-cmirror:  ?sync_count  : 151
>May 24 10:35:22 link-02 kernel: dm-cmirror:  ?sync_search : 0
>May 24 10:35:22 link-02 kernel: dm-cmirror:   in_sync     : YES
>May 24 10:35:22 link-02 kernel: dm-cmirror:   suspended   : NO
>May 24 10:35:22 link-02 kernel: dm-cmirror:   server_id   : 4
>May 24 10:35:22 link-02 kernel: dm-cmirror:   server_valid: YES
>May 24 10:35:22 link-02 lvm[5604]: No longer monitoring mirror device lock_stress-link--07.26410 for events 
>May 24 10:35:22 link-02 lvm[5604]: Unlocking memory
>May 24 10:35:22 link-02 lvm[5604]: memlock_count dec to 0
>May 24 10:35:22 link-02 lvm[5604]: Dumping persistent device cache to /etc/lvm/.cache
>May 24 10:35:22 link-02 lvm[5604]: Locking /etc/lvm/.cache (F_WRLCK, 1)
>May 24 10:35:22 link-02 lvm[5604]: Opened /etc/lvm/.cache RO
>May 24 10:35:22 link-02 lvm[5604]: Loaded persistent filter cache from /etc/lvm/.cache
>May 24 10:35:22 link-02 lvm[5604]: Unlocking fd 8
>May 24 10:35:22 link-02 lvm[5604]: Closed /etc/lvm/.cache
>May 24 10:35:22 link-02 lvm[5604]: Wiping internal VG cache
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:22 link-02 kernel: dm-cmirror: Performing flush to work around bug 235040
>May 24 10:35:22 link-02 kernel: dm-cmirror: Log flush complete
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:22 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (3MYLUzK0)
>May 24 10:35:22 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:22 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:35:22 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:22 link-02 kernel: dm-cmirror: Removing 3MYLUzK0 (2)
>May 24 10:35:22 link-02 kernel: dm-cmirror: 0 region user structures freed
>May 24 10:35:26 link-02 qarshd[9649]: Talking to peer 10.15.80.47:39924
>May 24 10:35:26 link-02 qarshd[9649]: Running cmdline: lvconvert -m 1 lock_stress/link-02.26406 
>May 24 10:35:29 link-02 qarshd[9649]: Nothing to do 
>May 24 10:35:38 link-02 last message repeated 3 times
>May 24 10:35:38 link-02 kernel: dm-cmirror: stop_server called
>May 24 10:35:41 link-02 qarshd[9649]: Nothing to do 
>May 24 10:35:42 link-02 kernel: dm-cmirror: Closing socket on server side
>May 24 10:35:44 link-02 qarshd[9649]: Nothing to do 
>May 24 10:35:46 link-02 kernel: dm-cmirror: Creating ymG90THI (1)
>May 24 10:35:46 link-02 kernel: dm-cmirror: start_server called
>May 24 10:35:46 link-02 kernel: dm-cmirror: cluster_log_serverd ready for work
>May 24 10:35:47 link-02 kernel: dm-cmirror: Node joining
>May 24 10:35:47 link-02 kernel: dm-cmirror: server_id=dead, server_valid=0, ymG90THI
>May 24 10:35:47 link-02 kernel: dm-cmirror: trigger = LRT_GET_SYNC_COUNT
>May 24 10:35:47 link-02 kernel: dm-cmirror: Node joining
>May 24 10:35:47 link-02 qarshd[9649]: Nothing to do 
>May 24 10:35:50 link-02 qarshd[9649]: Nothing to do 
>May 24 10:35:51 link-02 kernel: dm-cmirror: Node joining
>May 24 10:35:53 link-02 qarshd[9649]: Nothing to do 
>May 24 10:35:55 link-02 kernel: dm-cmirror: Node joining
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (ymG90THI)
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (ymG90THI)
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:55 link-02 [5604]: Monitoring mirror device lock_stress-link--02.26406 for events 
>May 24 10:35:55 link-02 lvm[5604]: Loading config file: /etc/lvm/lvm.conf
>May 24 10:35:55 link-02 lvm[5604]: Opened /etc/lvm/lvm.conf RO
>May 24 10:35:55 link-02 lvm[5604]: Closed /etc/lvm/lvm.conf
>May 24 10:35:55 link-02 lvm[5604]: Setting log/syslog to 1
>May 24 10:35:55 link-02 lvm[5604]: Setting log/level to 0
>May 24 10:35:55 link-02 lvm[5604]: Setting log/verbose to 0
>May 24 10:35:55 link-02 lvm[5604]: Setting log/indent to 1
>May 24 10:35:55 link-02 lvm[5604]: Setting log/prefix to   
>May 24 10:35:55 link-02 lvm[5604]: Setting log/command_names to 0
>May 24 10:35:55 link-02 lvm[5604]: Setting global/test to 0
>May 24 10:35:55 link-02 lvm[5604]: Setting log/overwrite to 0
>May 24 10:35:55 link-02 lvm[5604]: log/activation not found in config: defaulting to 0
>May 24 10:35:55 link-02 lvm[5604]: Logging initialised at Thu May 24 10:35:55 2007
>May 24 10:35:55 link-02 lvm[5604]: Setting global/umask to 63
>May 24 10:35:55 link-02 lvm[5604]: Setting devices/dir to /dev
>May 24 10:35:55 link-02 lvm[5604]: Setting global/proc to /proc
>May 24 10:35:55 link-02 lvm[5604]: Setting global/activation to 1
>May 24 10:35:55 link-02 lvm[5604]: global/suffix not found in config: defaulting to 1
>May 24 10:35:55 link-02 lvm[5604]: global/units not found in config: defaulting to h
>May 24 10:35:55 link-02 lvm[5604]: Matcher built with 11 dfa states
>May 24 10:35:55 link-02 lvm[5604]: Setting devices/ignore_suspended_devices to 0
>May 24 10:35:55 link-02 lvm[5604]: Setting devices/cache to /etc/lvm/.cache
>May 24 10:35:55 link-02 lvm[5604]: Setting devices/write_cache_state to 1
>May 24 10:35:55 link-02 lvm[5604]: Opened /etc/lvm/.cache RO
>May 24 10:35:55 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0001000000000000-part1: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x3000910891003200:0x0001000000000000-part1: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram11: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part1: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/VolGroup00/LogVol01: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram10: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/VolGroup00/LogVol00: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0004000000000000-part1: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0006000000000000-part1: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0005000000000000-part1: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram12: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part2: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram6: Added to device cache
>May 24 10:35:55 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0007000000000000-part1: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (ymG90THI)
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram13: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram14: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram5: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:55 link-02 lvm[5604]: /dev/mapper/VolGroup00-LogVol00: Aliased to /dev/VolGroup00/LogVol00 in device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (ymG90THI)
>May 24 10:35:55 link-02 lvm[5604]: /dev/hda1: Aliased to /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part1 in device cache (preferred name)
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram1: Aliased to /dev/ram in device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:55 link-02 lvm[5604]: /dev/sda1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0001000000000000-part1 in device cache (preferred name)
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:55 link-02 lvm[5604]: /dev/sdb1: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (ymG90THI)
>May 24 10:35:55 link-02 lvm[5604]: /dev/sdc1: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:55 link-02 lvm[5604]: /dev/sdg1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0007000000000000-part1 in device cache (preferred name)
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:55 link-02 lvm[5604]: /dev/sdd1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0004000000000000-part1 in device cache (preferred name)
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:35:55 link-02 lvm[5604]: /dev/sde1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0005000000000000-part1 in device cache (preferred name)
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (ymG90THI)
>May 24 10:35:55 link-02 lvm[5604]: /dev/sdf1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0006000000000000-part1 in device cache (preferred name)
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:55 link-02 lvm[5604]: /dev/sdh1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x3000910891003200:0x0001000000000000-part1 in device cache (preferred name)
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:55 link-02 lvm[5604]: /dev/ramdisk: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:55 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0002000000000000-part1: Aliased to /dev/sdb1 in device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (ymG90THI)
>May 24 10:35:55 link-02 lvm[5604]: /dev/mapper/VolGroup00-LogVol01: Aliased to /dev/VolGroup00/LogVol01 in device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram0: Aliased to /dev/ramdisk in device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:55 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0003000000000000-part1: Aliased to /dev/sdc1 in device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:55 link-02 lvm[5604]: /dev/root: Aliased to /dev/VolGroup00/LogVol00 in device cache (preferred name)
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (ymG90THI)
>May 24 10:35:55 link-02 lvm[5604]: /dev/dm-1: Aliased to /dev/VolGroup00/LogVol01 in device cache (preferred name)
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:55 link-02 lvm[5604]: /dev/hda2: Aliased to /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part2 in device cache (preferred name)
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram2: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram8: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (ymG90THI)
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram9: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:55 link-02 lvm[5604]: /dev/dm-0: Aliased to /dev/root in device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram15: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram3: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (ymG90THI)
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram4: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:55 link-02 lvm[5604]: /dev/ram7: Added to device cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:55 link-02 lvm[5604]: Loaded persistent filter cache from /etc/lvm/.cache
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:55 link-02 lvm[5604]: Closed /etc/lvm/.cache
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (ymG90THI)
>May 24 10:35:55 link-02 lvm[5604]: Setting activation/reserved_stack to 256
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:55 link-02 lvm[5604]: Setting activation/reserved_memory to 8192
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:55 link-02 lvm[5604]: Setting activation/process_priority to -18
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:55 link-02 lvm[5604]: Initialised format: lvm1
>May 24 10:35:55 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (ymG90THI)
>May 24 10:35:55 link-02 lvm[5604]: Initialised format: pool
>May 24 10:35:55 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:55 link-02 lvm[5604]: Initialised format: lvm2
>May 24 10:35:55 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:55 link-02 lvm[5604]: global/format not found in config: defaulting to lvm2
>May 24 10:35:55 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:55 link-02 lvm[5604]: Initialised segtype: striped
>May 24 10:35:55 link-02 lvm[5604]: Initialised segtype: zero
>May 24 10:35:55 link-02 lvm[5604]: Initialised segtype: error
>May 24 10:35:55 link-02 lvm[5604]: Initialised segtype: snapshot
>May 24 10:35:55 link-02 lvm[5604]: Initialised segtype: mirror
>May 24 10:35:55 link-02 lvm[5604]: Setting backup/retain_days to 30
>May 24 10:35:55 link-02 lvm[5604]: Setting backup/retain_min to 10
>May 24 10:35:55 link-02 lvm[5604]: Setting backup/archive_dir to /etc/lvm/archive
>May 24 10:35:55 link-02 lvm[5604]: Setting backup/backup_dir to /etc/lvm/backup
>May 24 10:35:55 link-02 lvm[5604]: Locking memory
>May 24 10:35:55 link-02 lvm[5604]: memlock_count inc to 1
>May 24 10:35:55 link-02 lvm[5604]: Parsing: _memlock_inc
>May 24 10:35:55 link-02 qarshd[9649]: That's enough 
>May 24 10:35:55 link-02 qarshd[9774]: Talking to peer 10.15.80.47:39927
>May 24 10:35:55 link-02 qarshd[9774]: Running cmdline: lvs lock_stress/link-02.26406 --noheadings -o mirror_log | awk {'print $1'} 
>May 24 10:35:57 link-02 kernel: dm-cmirror: Creating dZtyZsZC (1)
>May 24 10:35:57 link-02 kernel: dm-cmirror: server_id=dead, server_valid=0, dZtyZsZC
>May 24 10:35:57 link-02 kernel: dm-cmirror: trigger = LRT_GET_SYNC_COUNT
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:57 link-02 kernel: dm-cmirror: I'm the cluster mirror log server for dZtyZsZC
>May 24 10:35:57 link-02 kernel: dm-cmirror: Not reading disk log because I'm suspended (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:35:57 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (dZtyZsZC)
>May 24 10:35:57 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:35:57 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:35:57 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:35:57 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 0
>May 24 10:35:57 link-02 kernel: dm-cmirror: Client received resync work: 0/dZtyZsZC
>May 24 10:35:57 link-02 lvm[5604]: Monitoring mirror device lock_stress-link--08.26408 for events 
>May 24 10:35:57 link-02 kernel: dm-cmirror: Resync work completed: 0
>May 24 10:35:57 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 1
>May 24 10:35:57 link-02 kernel: dm-cmirror: Client received resync work: 1/dZtyZsZC
>May 24 10:35:57 link-02 kernel: dm-cmirror: Resync work completed: 1
>May 24 10:35:57 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 2
>May 24 10:35:57 link-02 kernel: dm-cmirror: Client received resync work: 2/dZtyZsZC
>May 24 10:35:57 link-02 kernel: dm-cmirror: Resync work completed: 2
>May 24 10:35:57 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 3
>May 24 10:35:57 link-02 kernel: dm-cmirror: Client received resync work: 3/dZtyZsZC
>May 24 10:35:57 link-02 kernel: dm-cmirror: Resync work completed: 3
>May 24 10:35:57 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 4
>May 24 10:35:57 link-02 kernel: dm-cmirror: Client received resync work: 4/dZtyZsZC
>May 24 10:35:57 link-02 kernel: dm-cmirror: Resync work completed: 4
>May 24 10:35:57 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 5
>May 24 10:35:57 link-02 kernel: dm-cmirror: Client received resync work: 5/dZtyZsZC
>May 24 10:35:57 link-02 kernel: dm-cmirror: Resync work completed: 5
>May 24 10:35:57 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 6
>May 24 10:35:57 link-02 kernel: dm-cmirror: Client received resync work: 6/dZtyZsZC
>May 24 10:35:57 link-02 kernel: dm-cmirror: Resync work completed: 6
>May 24 10:35:57 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 7
>May 24 10:35:57 link-02 kernel: dm-cmirror: Client received resync work: 7/dZtyZsZC
>May 24 10:35:57 link-02 kernel: dm-cmirror: Resync work completed: 7
>May 24 10:35:57 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 8
>May 24 10:35:57 link-02 kernel: dm-cmirror: Client received resync work: 8/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 8
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 9
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 9/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 9
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 10
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 10/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 10
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 11
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 11/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 11
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 12
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 12/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 12
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 13
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 13/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 13
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 14
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 14/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 14
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 15
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 15/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 15
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 16
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 16/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 16
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 17
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 17/dZtyZsZC
>May 24 10:35:58 link-02 qarshd[9774]: That's enough 
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 17
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 18
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 18/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 18
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 19
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 19/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 19
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 20
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 20/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 20
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 21
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 21/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 21
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 22
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 22/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 22
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 23
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 23/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 23
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 24
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 24/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 24
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 25
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 25/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 25
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 26
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 26/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 26
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 27
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 27/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 27
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 28
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 28/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 28
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 29
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 29/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 29
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 30
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 30/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 30
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 31
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 31/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 31
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 32
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 32/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 32
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 33
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 33/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 33
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 34
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 34/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 34
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 35
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 35/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 35
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 36
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 36/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 36
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 37
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 37/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 37
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 38
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 38/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 38
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 39
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 39/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 39
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 40
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 40/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 40
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 41
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 41/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 41
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 42
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 42/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 42
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 43
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 43/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 43
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 44
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 44/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 44
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 45
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 45/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 45
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 46
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 46/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 46
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 47
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 47/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 47
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 48
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 48/dZtyZsZC
>May 24 10:35:58 link-02 kernel: dm-cmirror: Resync work completed: 48
>May 24 10:35:58 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 49
>May 24 10:35:58 link-02 kernel: dm-cmirror: Client received resync work: 49/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 49
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 50
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 50/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 50
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 51
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 51/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 51
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 52
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 52/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 52
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 53
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 53/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 53
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 54
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 54/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 54
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 55
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 55/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 55
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 56
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 56/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 56
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 57
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 57/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 57
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 58
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 58/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 58
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 59
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 59/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 59
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 60
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 60/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 60
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 61
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 61/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 61
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 62
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 62/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 62
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 63
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 63/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 63
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 64
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 64/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 64
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 65
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 65/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 65
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 66
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 66/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 66
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 67
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 67/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 67
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 68
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 68/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 68
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 69
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 69/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 69
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 70
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 70/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 70
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 71
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 71/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 71
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 72
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 72/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 72
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 73
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 73/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 73
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 74
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 74/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 74
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 75
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 75/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 75
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 76
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 76/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 76
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 77
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 77/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 77
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 78
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 78/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 78
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 79
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 79/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 79
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 80
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 80/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 80
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 81
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 81/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 81
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 82
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 82/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 82
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 83
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 83/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 83
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 84
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 84/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 84
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 85
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 85/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 85
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 86
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 86/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 86
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 87
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 87/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 87
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 88
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 88/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 88
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 89
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 89/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 89
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 90
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 90/dZtyZsZC
>May 24 10:35:59 link-02 kernel: dm-cmirror: Resync work completed: 90
>May 24 10:35:59 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 91
>May 24 10:35:59 link-02 kernel: dm-cmirror: Client received resync work: 91/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 91
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 92
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 92/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 92
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 93
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 93/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 93
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 94
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 94/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 94
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 95
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 95/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 95
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 96
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 96/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 96
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 97
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 97/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 97
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 98
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 98/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 98
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 99
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 99/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 99
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 100
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 100/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 100
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 101
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 101/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 101
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 102
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 102/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 102
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 103
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 103/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 103
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 104
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 104/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 104
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 105
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 105/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 105
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 106
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 106/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 106
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 107
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 107/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 107
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 108
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 108/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 108
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 109
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 109/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 109
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 110
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 110/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 110
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 111
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 111/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 111
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 112
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 112/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 112
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 113
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 113/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 113
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 114
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 114/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 114
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 115
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 115/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 115
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 116
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 116/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 116
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 117
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 117/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 117
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 118
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 118/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 118
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 119
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 119/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 119
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 120
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 120/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 120
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 121
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 121/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 121
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 122
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 122/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 122
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 123
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 123/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 123
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 124
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 124/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 124
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 125
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 125/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 125
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 126
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 126/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 126
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 127
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 127/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 127
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 128
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 128/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 128
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 129
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 129/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 129
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 130
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 130/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 130
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 131
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 131/dZtyZsZC
>May 24 10:36:00 link-02 kernel: dm-cmirror: Resync work completed: 131
>May 24 10:36:00 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 132
>May 24 10:36:00 link-02 kernel: dm-cmirror: Client received resync work: 132/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 132
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 133
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 133/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 133
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 134
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 134/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 134
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 135
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 135/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 135
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 136
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 136/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 136
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 137
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 137/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 137
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 138
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 138/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 138
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 139
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 139/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 139
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 140
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 140/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 140
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 141
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 141/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 141
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 142
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 142/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 142
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 143
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 143/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 143
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 144
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 144/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 144
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 145
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 145/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 145
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 146
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 146/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 146
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 147
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 147/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 147
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 148
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 148/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 148
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 149
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 149/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 149
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 150
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 150/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 150
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 151
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 151/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 151
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 152
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 152/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 152
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 153
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 153/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 153
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 154
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 154/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 154
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 155
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 155/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 155
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 156
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 156/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 156
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 157
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 157/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 157
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 158
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 158/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 158
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 159
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 159/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 159
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 160
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 160/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 160
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 161
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 161/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 161
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 162
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 162/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 162
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 163
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 163/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 163
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 164
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 164/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 164
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 165
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 165/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 165
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 166
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 166/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 166
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 167
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 167/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 167
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 168
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 168/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 168
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 169
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 169/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 169
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 170
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 170/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 170
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 171
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 171/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 171
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 172
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 172/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 172
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 173
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 173/dZtyZsZC
>May 24 10:36:01 link-02 kernel: dm-cmirror: Resync work completed: 173
>May 24 10:36:01 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 174
>May 24 10:36:01 link-02 kernel: dm-cmirror: Client received resync work: 174/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 174
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 175
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 175/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 175
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 176
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 176/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 176
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 177
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 177/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 177
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 178
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 178/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 178
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 179
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 179/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 179
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 180
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 180/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 180
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 181
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 181/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 181
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 182
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 182/dZtyZsZC
>May 24 10:36:02 link-02 qarshd[9841]: Talking to peer 10.15.80.47:39929
>May 24 10:36:02 link-02 qarshd[9841]: Running cmdline: lvchange -an lock_stress/link-02.26406 
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 182
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 183
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 183/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 183
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 184
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 184/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 184
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 185
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 185/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 185
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 186
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 186/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 186
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 187
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 187/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 187
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 188
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 188/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 188
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 189
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 189/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 189
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 190
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 190/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 190
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 191
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 191/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 191
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 192
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 192/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 192
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 193
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 193/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 193
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 194
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 194/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 194
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 195
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 195/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 195
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 196
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 196/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 196
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 197
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 197/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 197
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 198
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 198/dZtyZsZC
>May 24 10:36:02 link-02 lvm[5604]: No longer monitoring mirror device lock_stress-link--02.26406 for events 
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 198
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 199
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 199/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Performing flush to work around bug 235040
>May 24 10:36:02 link-02 kernel: dm-cmirror: Log flush complete
>May 24 10:36:02 link-02 kernel: dm-cmirror: Removing ymG90THI (1)
>May 24 10:36:02 link-02 kernel: dm-cmirror: 0 region user structures freed
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 199
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 200
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 200/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 200
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 201
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 201/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 201
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 202
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 202/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 202
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 203
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 203/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 203
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 204
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 204/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (none)
>May 24 10:36:02 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:02 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:02 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:02 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (none)
>May 24 10:36:02 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:02 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:02 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:02 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (none)
>May 24 10:36:02 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:02 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:02 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:02 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (none)
>May 24 10:36:02 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:02 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:02 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 204
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 205
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 205/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 205
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 206
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 206/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 206
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 207
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 207/dZtyZsZC
>May 24 10:36:02 link-02 qarshd[9841]: That's enough 
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 207
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 208
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 208/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 208
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 209
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 209/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 209
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 210
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 210/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 210
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 211
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 211/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 211
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 212
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 212/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 212
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 213
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 213/dZtyZsZC
>May 24 10:36:02 link-02 kernel: dm-cmirror: Resync work completed: 213
>May 24 10:36:02 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 214
>May 24 10:36:02 link-02 kernel: dm-cmirror: Client received resync work: 214/dZtyZsZC
>May 24 10:36:03 link-02 kernel: dm-cmirror: Resync work completed: 214
>May 24 10:36:03 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 215
>May 24 10:36:03 link-02 kernel: dm-cmirror: Client received resync work: 215/dZtyZsZC
>May 24 10:36:03 link-02 kernel: dm-cmirror: Resync work completed: 215
>May 24 10:36:03 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 216
>May 24 10:36:03 link-02 kernel: dm-cmirror: Client received resync work: 216/dZtyZsZC
>May 24 10:36:03 link-02 kernel: dm-cmirror: Resync work completed: 216
>May 24 10:36:03 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 217
>May 24 10:36:03 link-02 kernel: dm-cmirror: Client received resync work: 217/dZtyZsZC
>May 24 10:36:03 link-02 kernel: dm-cmirror: Resync work completed: 217
>May 24 10:36:03 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 218
>May 24 10:36:03 link-02 kernel: dm-cmirror: Client received resync work: 218/dZtyZsZC
>May 24 10:36:03 link-02 kernel: dm-cmirror: Resync work completed: 218
>May 24 10:36:03 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 219
>May 24 10:36:03 link-02 kernel: dm-cmirror: Client received resync work: 219/dZtyZsZC
>May 24 10:36:03 link-02 kernel: dm-cmirror: Resync work completed: 219
>May 24 10:36:03 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 220
>May 24 10:36:03 link-02 kernel: dm-cmirror: Client received resync work: 220/dZtyZsZC
>May 24 10:36:03 link-02 kernel: dm-cmirror: Resync work completed: 220
>May 24 10:36:03 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 221
>May 24 10:36:03 link-02 kernel: dm-cmirror: Client received resync work: 221/dZtyZsZC
>May 24 10:36:03 link-02 lvm[5604]: No longer monitoring mirror device lock_stress-link--08.26408 for events 
>May 24 10:36:03 link-02 lvm[5604]: Unlocking memory
>May 24 10:36:03 link-02 lvm[5604]: memlock_count dec to 0
>May 24 10:36:03 link-02 lvm[5604]: Dumping persistent device cache to /etc/lvm/.cache
>May 24 10:36:03 link-02 lvm[5604]: Locking /etc/lvm/.cache (F_WRLCK, 1)
>May 24 10:36:03 link-02 lvm[5604]: Opened /etc/lvm/.cache RO
>May 24 10:36:03 link-02 lvm[5604]: Loaded persistent filter cache from /etc/lvm/.cache
>May 24 10:36:03 link-02 lvm[5604]: Unlocking fd 8
>May 24 10:36:03 link-02 lvm[5604]: Closed /etc/lvm/.cache
>May 24 10:36:03 link-02 lvm[5604]: Wiping internal VG cache
>May 24 10:36:03 link-02 kernel: dm-cmirror: Resync work completed: 221
>May 24 10:36:03 link-02 kernel: dm-cmirror: Assigning recovery work to 3/dZtyZsZC: 222
>May 24 10:36:03 link-02 kernel: dm-cmirror: Client received resync work: 222/dZtyZsZC
>May 24 10:36:03 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (dZtyZsZC)
>May 24 10:36:03 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:03 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:03 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:03 link-02 qarshd[9888]: Talking to peer 10.15.80.47:39932
>May 24 10:36:03 link-02 qarshd[9888]: Running cmdline: dmsetup ls | grep lock_stress | grep 26406 
>May 24 10:36:03 link-02 qarshd[9888]: That's enough 
>May 24 10:36:04 link-02 qarshd[9893]: Talking to peer 10.15.80.47:39933
>May 24 10:36:04 link-02 qarshd[9893]: Running cmdline: lvchange -ay lock_stress/link-02.26406 
>May 24 10:36:07 link-02 qarshd[9893]: Nothing to do 
>May 24 10:36:08 link-02 kernel: dm-cmirror: Disk Resume::  dZtyZsZC (active)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   Live nodes        :: 3
>May 24 10:36:08 link-02 kernel: dm-cmirror:   In-Use Regions    :: 1
>May 24 10:36:08 link-02 kernel: dm-cmirror:   Good IUR's        :: 1
>May 24 10:36:08 link-02 kernel: dm-cmirror:   Bad IUR's         :: 0
>May 24 10:36:08 link-02 kernel: dm-cmirror:   Sync count        :: 1000
>May 24 10:36:08 link-02 kernel: dm-cmirror:   Disk Region count :: 18446744073709551615
>May 24 10:36:08 link-02 kernel: dm-cmirror:   Region count      :: 1000
>May 24 10:36:08 link-02 kernel: dm-cmirror:   NOTE:  Mapping has changed.
>May 24 10:36:08 link-02 kernel: dm-cmirror: Marked regions::
>May 24 10:36:08 link-02 kernel: dm-cmirror:   Total = 0
>May 24 10:36:08 link-02 kernel: dm-cmirror: Out-of-sync regions::
>May 24 10:36:08 link-02 kernel: dm-cmirror:   Total = 0
>May 24 10:36:08 link-02 kernel: dm-cmirror: Resync work completed: 222
>May 24 10:36:08 link-02 kernel: dm-cmirror: Performing flush to work around bug 235040
>May 24 10:36:08 link-02 kernel: dm-cmirror: Log flush complete
>May 24 10:36:08 link-02 kernel: dm-cmirror: Telling everyone I'm suspending (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:08 link-02 kernel: dm-cmirror: Suspending now (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (dZtyZsZC)
>May 24 10:36:08 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:08 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:08 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:08 link-02 kernel: dm-cmirror: Removing dZtyZsZC (1)
>May 24 10:36:08 link-02 kernel: dm-cmirror: 0 region user structures freed
>May 24 10:36:10 link-02 qarshd[9893]: Nothing to do 
>May 24 10:36:16 link-02 last message repeated 2 times
>May 24 10:36:18 link-02 kernel: dm-cmirror: stop_server called
>May 24 10:36:19 link-02 qarshd[9893]: Nothing to do 
>May 24 10:36:22 link-02 qarshd[9893]: Nothing to do 
>May 24 10:36:23 link-02 kernel: dm-cmirror: Closing socket on server side
>May 24 10:36:24 link-02 kernel: dm-cmirror: Creating 2dHqGlmO (1)
>May 24 10:36:24 link-02 kernel: dm-cmirror: start_server called
>May 24 10:36:24 link-02 kernel: dm-cmirror: cluster_log_serverd ready for work
>May 24 10:36:24 link-02 kernel: dm-cmirror: Node joining
>May 24 10:36:24 link-02 kernel: dm-cmirror: Node joining
>May 24 10:36:24 link-02 kernel: dm-cmirror: server_id=dead, server_valid=0, 2dHqGlmO
>May 24 10:36:24 link-02 kernel: dm-cmirror: trigger = LRT_GET_SYNC_COUNT
>May 24 10:36:25 link-02 kernel: dm-cmirror: Node joining
>May 24 10:36:25 link-02 qarshd[9893]: Nothing to do 
>May 24 10:36:28 link-02 kernel: dm-cmirror: Node joining
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (2dHqGlmO)
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:36:28 link-02 qarshd[9893]: Nothing to do 
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (2dHqGlmO)
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:28 link-02 [5604]: Monitoring mirror device lock_stress-link--07.26410 for events 
>May 24 10:36:28 link-02 lvm[5604]: Loading config file: /etc/lvm/lvm.conf
>May 24 10:36:28 link-02 lvm[5604]: Opened /etc/lvm/lvm.conf RO
>May 24 10:36:28 link-02 lvm[5604]: Closed /etc/lvm/lvm.conf
>May 24 10:36:28 link-02 lvm[5604]: Setting log/syslog to 1
>May 24 10:36:28 link-02 lvm[5604]: Setting log/level to 0
>May 24 10:36:28 link-02 lvm[5604]: Setting log/verbose to 0
>May 24 10:36:28 link-02 lvm[5604]: Setting log/indent to 1
>May 24 10:36:28 link-02 lvm[5604]: Setting log/prefix to   
>May 24 10:36:28 link-02 lvm[5604]: Setting log/command_names to 0
>May 24 10:36:28 link-02 lvm[5604]: Setting global/test to 0
>May 24 10:36:28 link-02 lvm[5604]: Setting log/overwrite to 0
>May 24 10:36:28 link-02 lvm[5604]: log/activation not found in config: defaulting to 0
>May 24 10:36:28 link-02 lvm[5604]: Logging initialised at Thu May 24 10:36:28 2007
>May 24 10:36:28 link-02 lvm[5604]: Setting global/umask to 63
>May 24 10:36:28 link-02 lvm[5604]: Setting devices/dir to /dev
>May 24 10:36:28 link-02 lvm[5604]: Setting global/proc to /proc
>May 24 10:36:28 link-02 lvm[5604]: Setting global/activation to 1
>May 24 10:36:28 link-02 lvm[5604]: global/suffix not found in config: defaulting to 1
>May 24 10:36:28 link-02 lvm[5604]: global/units not found in config: defaulting to h
>May 24 10:36:28 link-02 lvm[5604]: Matcher built with 11 dfa states
>May 24 10:36:28 link-02 lvm[5604]: Setting devices/ignore_suspended_devices to 0
>May 24 10:36:28 link-02 lvm[5604]: Setting devices/cache to /etc/lvm/.cache
>May 24 10:36:28 link-02 lvm[5604]: Setting devices/write_cache_state to 1
>May 24 10:36:28 link-02 lvm[5604]: Opened /etc/lvm/.cache RO
>May 24 10:36:28 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0001000000000000-part1: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x3000910891003200:0x0001000000000000-part1: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram11: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part1: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/VolGroup00/LogVol01: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram10: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/VolGroup00/LogVol00: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0004000000000000-part1: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0006000000000000-part1: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0005000000000000-part1: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram12: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part2: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram6: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0007000000000000-part1: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram13: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram14: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram5: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/mapper/VolGroup00-LogVol00: Aliased to /dev/VolGroup00/LogVol00 in device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/hda1: Aliased to /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part1 in device cache (preferred name)
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram1: Aliased to /dev/ram in device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/sda1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0001000000000000-part1 in device cache (preferred name)
>May 24 10:36:28 link-02 lvm[5604]: /dev/sdb1: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/sdc1: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/sdg1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0007000000000000-part1 in device cache (preferred name)
>May 24 10:36:28 link-02 lvm[5604]: /dev/sdd1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0004000000000000-part1 in device cache (preferred name)
>May 24 10:36:28 link-02 lvm[5604]: /dev/sde1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0005000000000000-part1 in device cache (preferred name)
>May 24 10:36:28 link-02 lvm[5604]: /dev/sdf1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0006000000000000-part1 in device cache (preferred name)
>May 24 10:36:28 link-02 lvm[5604]: /dev/sdh1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x3000910891003200:0x0001000000000000-part1 in device cache (preferred name)
>May 24 10:36:28 link-02 lvm[5604]: /dev/ramdisk: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0002000000000000-part1: Aliased to /dev/sdb1 in device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/mapper/VolGroup00-LogVol01: Aliased to /dev/VolGroup00/LogVol01 in device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram0: Aliased to /dev/ramdisk in device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0003000000000000-part1: Aliased to /dev/sdc1 in device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/root: Aliased to /dev/VolGroup00/LogVol00 in device cache (preferred name)
>May 24 10:36:28 link-02 lvm[5604]: /dev/dm-1: Aliased to /dev/VolGroup00/LogVol01 in device cache (preferred name)
>May 24 10:36:28 link-02 lvm[5604]: /dev/hda2: Aliased to /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part2 in device cache (preferred name)
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram2: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram8: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram9: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/dm-0: Aliased to /dev/root in device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram15: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram3: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram4: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: /dev/ram7: Added to device cache
>May 24 10:36:28 link-02 lvm[5604]: Loaded persistent filter cache from /etc/lvm/.cache
>May 24 10:36:28 link-02 lvm[5604]: Closed /etc/lvm/.cache
>May 24 10:36:28 link-02 lvm[5604]: Setting activation/reserved_stack to 256
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (2dHqGlmO)
>May 24 10:36:28 link-02 lvm[5604]: Setting activation/reserved_memory to 8192
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:28 link-02 lvm[5604]: Setting activation/process_priority to -18
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:28 link-02 lvm[5604]: Initialised format: lvm1
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:28 link-02 lvm[5604]: Initialised format: pool
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (2dHqGlmO)
>May 24 10:36:28 link-02 lvm[5604]: Initialised format: lvm2
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:28 link-02 lvm[5604]: global/format not found in config: defaulting to lvm2
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:28 link-02 lvm[5604]: Initialised segtype: striped
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:28 link-02 lvm[5604]: Initialised segtype: zero
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (2dHqGlmO)
>May 24 10:36:28 link-02 lvm[5604]: Initialised segtype: error
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:28 link-02 lvm[5604]: Initialised segtype: snapshot
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:28 link-02 lvm[5604]: Initialised segtype: mirror
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:36:28 link-02 lvm[5604]: Setting backup/retain_days to 30
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (2dHqGlmO)
>May 24 10:36:28 link-02 lvm[5604]: Setting backup/retain_min to 10
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:28 link-02 lvm[5604]: Setting backup/archive_dir to /etc/lvm/archive
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:28 link-02 lvm[5604]: Setting backup/backup_dir to /etc/lvm/backup
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:28 link-02 lvm[5604]: Locking memory
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (2dHqGlmO)
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (2dHqGlmO)
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (2dHqGlmO)
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (2dHqGlmO)
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:28 link-02 lvm[5604]: memlock_count inc to 1
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:28 link-02 lvm[5604]: Parsing: _memlock_inc
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (2dHqGlmO)
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:28 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (2dHqGlmO)
>May 24 10:36:28 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:28 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:28 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:29 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (none)
>May 24 10:36:29 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:29 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:36:29 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:29 link-02 kernel: dm-cmirror: Creating ymG90THI (1)
>May 24 10:36:29 link-02 kernel: dm-cmirror: server_id=dead, server_valid=0, ymG90THI
>May 24 10:36:29 link-02 kernel: dm-cmirror: trigger = LRT_GET_RESYNC_WORK
>May 24 10:36:29 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (ymG90THI)
>May 24 10:36:29 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:29 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:36:29 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:36:29 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (ymG90THI)
>May 24 10:36:29 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:29 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:36:29 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:29 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (ymG90THI)
>May 24 10:36:29 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:29 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:36:29 link-02 lvm[5604]: Monitoring mirror device lock_stress-link--02.26406 for events 
>May 24 10:36:29 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:36:29 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (ymG90THI)
>May 24 10:36:29 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:29 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:29 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:29 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (ymG90THI)
>May 24 10:36:29 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:29 link-02 kernel: dm-cmirror:   co-ordinator: 4
>May 24 10:36:29 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:29 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (ymG90THI)
>May 24 10:36:29 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:29 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:36:29 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:36:29 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (ymG90THI)
>May 24 10:36:29 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:29 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:29 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:30 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (ymG90THI)
>May 24 10:36:30 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:30 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:30 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:30 link-02 qarshd[9893]: That's enough 
>May 24 10:36:31 link-02 kernel: dm-cmirror: Creating dZtyZsZC (1)
>May 24 10:36:31 link-02 kernel: dm-cmirror: server_id=dead, server_valid=0, dZtyZsZC
>May 24 10:36:31 link-02 kernel: dm-cmirror: trigger = LRT_GET_RESYNC_WORK
>May 24 10:36:31 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:36:31 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:31 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:36:31 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:36:31 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:36:31 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:31 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:36:31 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:31 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:36:31 link-02 lvm[5604]: Monitoring mirror device lock_stress-link--08.26408 for events 
>May 24 10:36:31 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:36:31 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:31 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:31 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:36:31 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:31 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:36:31 link-02 lvm[5604]: lock_stress-link--08.26408 is now in-sync 
>May 24 10:36:31 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:31 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   node_count  : 4
>May 24 10:36:31 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:36:31 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:31 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:31 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (dZtyZsZC)
>May 24 10:36:31 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:31 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (dZtyZsZC)
>May 24 10:36:31 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:36:31 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:36:31 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:31 link-02 qarshd[10146]: Talking to peer 10.15.80.47:36109
>May 24 10:36:31 link-02 qarshd[10146]: Running cmdline: dmsetup ls | grep lock_stress | grep 26406 2>&1 > /dev/null 
>May 24 10:36:31 link-02 qarshd[10146]: That's enough 
>May 24 10:36:31 link-02 qarshd[10151]: Talking to peer 10.15.80.47:36110
>May 24 10:36:31 link-02 qarshd[10151]: Running cmdline: lvchange -an lock_stress/link-02.26406 
>May 24 10:36:32 link-02 lvm[5604]: No longer monitoring mirror device lock_stress-link--02.26406 for events 
>May 24 10:36:32 link-02 kernel: dm-cmirror: Performing flush to work around bug 235040
>May 24 10:36:32 link-02 kernel: dm-cmirror: Log flush complete
>May 24 10:36:32 link-02 kernel: dm-cmirror: Removing ymG90THI (1)
>May 24 10:36:32 link-02 kernel: dm-cmirror: 0 region user structures freed
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 3
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 4
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:32 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (none)
>May 24 10:36:32 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:32 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:32 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:32 link-02 qarshd[10151]: That's enough 
>May 24 10:36:33 link-02 lvm[5604]: No longer monitoring mirror device lock_stress-link--08.26408 for events 
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (dZtyZsZC)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (dZtyZsZC)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror: Performing flush to work around bug 235040
>May 24 10:36:33 link-02 kernel: dm-cmirror: Log flush complete
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (dZtyZsZC)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror: Removing dZtyZsZC (1)
>May 24 10:36:33 link-02 kernel: dm-cmirror: 0 region user structures freed
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (none)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (none)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (none)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (none)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (none)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (none)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (none)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:33 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (none)
>May 24 10:36:33 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:33 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:33 link-02 kernel: dm-cmirror:   node_count  : 2
>May 24 10:36:33 link-02 qarshd[10242]: Talking to peer 10.15.80.47:36113
>May 24 10:36:33 link-02 qarshd[10242]: Running cmdline: dmsetup ls | grep lock_stress | grep 26406 
>May 24 10:36:33 link-02 qarshd[10242]: That's enough 
>May 24 10:36:33 link-02 qarshd[10247]: Talking to peer 10.15.80.47:36114
>May 24 10:36:33 link-02 qarshd[10247]: Running cmdline: lvremove lock_stress/link-02.26406 
>May 24 10:36:33 link-02 kernel: cdrom: open failed.
>May 24 10:36:34 link-02 lvm[5604]: Monitoring mirror device lock_stress-link--02.26406 for events 
>May 24 10:36:34 link-02 lvm[5604]: No longer monitoring mirror device lock_stress-link--02.26406 for events 
>May 24 10:36:34 link-02 qarshd[10247]: That's enough 
>May 24 10:36:36 link-02 lvm[5604]: No longer monitoring mirror device lock_stress-link--07.26410 for events 
>May 24 10:36:36 link-02 lvm[5604]: Unlocking memory
>May 24 10:36:36 link-02 lvm[5604]: memlock_count dec to 0
>May 24 10:36:36 link-02 lvm[5604]: Dumping persistent device cache to /etc/lvm/.cache
>May 24 10:36:36 link-02 lvm[5604]: Locking /etc/lvm/.cache (F_WRLCK, 1)
>May 24 10:36:36 link-02 lvm[5604]: Opened /etc/lvm/.cache RO
>May 24 10:36:36 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410_mimage_3: Added to device cache
>May 24 10:36:36 link-02 lvm[5604]: /dev/dm-6: Aliased to /dev/mapper/lock_stress-link--07.26410_mimage_3 in device cache (preferred name)
>May 24 10:36:36 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410_mimage_2: Added to device cache
>May 24 10:36:36 link-02 lvm[5604]: /dev/dm-5: Aliased to /dev/mapper/lock_stress-link--07.26410_mimage_2 in device cache (preferred name)
>May 24 10:36:36 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410_mimage_0: Added to device cache
>May 24 10:36:36 link-02 lvm[5604]: /dev/lock_stress/link-07.26410: Added to device cache
>May 24 10:36:36 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410_mimage_1: Added to device cache
>May 24 10:36:36 link-02 kernel: dm-cmirror: Performing flush to work around bug 235040
>May 24 10:36:36 link-02 lvm[5604]: /dev/dm-2: Added to device cache
>May 24 10:36:36 link-02 kernel: dm-cmirror: Log flush complete
>May 24 10:36:36 link-02 lvm[5604]: /dev/dm-3: Aliased to /dev/mapper/lock_stress-link--07.26410_mimage_0 in device cache (preferred name)
>May 24 10:36:36 link-02 kernel: dm-cmirror: Removing 2dHqGlmO (1)
>May 24 10:36:36 link-02 lvm[5604]: /dev/dm-4: Aliased to /dev/mapper/lock_stress-link--07.26410_mimage_1 in device cache (preferred name)
>May 24 10:36:36 link-02 kernel: dm-cmirror: 0 region user structures freed
>May 24 10:36:36 link-02 lvm[5604]: /dev/dm-7: Aliased to /dev/lock_stress/link-07.26410 in device cache (preferred name)
>May 24 10:36:36 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410_mlog: Aliased to /dev/dm-2 in device cache
>May 24 10:36:36 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410: Aliased to /dev/dm-7 in device cache
>May 24 10:36:36 link-02 lvm[5604]: Loaded persistent filter cache from /etc/lvm/.cache
>May 24 10:36:36 link-02 lvm[5604]: Unlocking fd 8
>May 24 10:36:36 link-02 lvm[5604]: Closed /etc/lvm/.cache
>May 24 10:36:36 link-02 lvm[5604]: Wiping internal VG cache
>May 24 10:36:41 link-02 kernel: dm-cmirror: LRT_MASTER_LEAVING(13): (none)
>May 24 10:36:41 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:41 link-02 kernel: dm-cmirror:   co-ordinator: 0
>May 24 10:36:41 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:41 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (none)
>May 24 10:36:41 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:41 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:41 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:41 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (none)
>May 24 10:36:41 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:41 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:41 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:41 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (none)
>May 24 10:36:41 link-02 kernel: dm-cmirror:   starter     : 1
>May 24 10:36:41 link-02 kernel: dm-cmirror:   co-ordinator: 57005
>May 24 10:36:41 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:36:51 link-02 kernel: dm-cmirror: stop_server called
>May 24 10:36:51 link-02 kernel: dm-cmirror: Closing socket on server side
>May 24 10:36:58 link-02 kernel: dm-cmirror: Creating 2dHqGlmO (1)
>May 24 10:36:58 link-02 kernel: dm-cmirror: start_server called
>May 24 10:36:58 link-02 kernel: dm-cmirror: cluster_log_serverd ready for work
>May 24 10:37:10 link-02 kernel: dm-cmirror: Node joining
>May 24 10:37:14 link-02 kernel: dm-cmirror: server_id=dead, server_valid=0, 2dHqGlmO
>May 24 10:37:14 link-02 kernel: dm-cmirror: trigger = LRT_GET_RESYNC_WORK
>May 24 10:37:14 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (2dHqGlmO)
>May 24 10:37:14 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:37:14 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:37:14 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:37:14 link-02 [5604]: Monitoring mirror device lock_stress-link--07.26410 for events 
>May 24 10:37:14 link-02 lvm[5604]: Loading config file: /etc/lvm/lvm.conf
>May 24 10:37:14 link-02 lvm[5604]: Opened /etc/lvm/lvm.conf RO
>May 24 10:37:14 link-02 lvm[5604]: Closed /etc/lvm/lvm.conf
>May 24 10:37:14 link-02 lvm[5604]: Setting log/syslog to 1
>May 24 10:37:14 link-02 lvm[5604]: Setting log/level to 0
>May 24 10:37:14 link-02 lvm[5604]: Setting log/verbose to 0
>May 24 10:37:14 link-02 lvm[5604]: Setting log/indent to 1
>May 24 10:37:14 link-02 lvm[5604]: Setting log/prefix to   
>May 24 10:37:14 link-02 lvm[5604]: Setting log/command_names to 0
>May 24 10:37:14 link-02 lvm[5604]: Setting global/test to 0
>May 24 10:37:14 link-02 lvm[5604]: Setting log/overwrite to 0
>May 24 10:37:14 link-02 lvm[5604]: log/activation not found in config: defaulting to 0
>May 24 10:37:14 link-02 lvm[5604]: Logging initialised at Thu May 24 10:37:14 2007
>May 24 10:37:14 link-02 lvm[5604]: Setting global/umask to 63
>May 24 10:37:14 link-02 lvm[5604]: Setting devices/dir to /dev
>May 24 10:37:14 link-02 lvm[5604]: Setting global/proc to /proc
>May 24 10:37:14 link-02 lvm[5604]: Setting global/activation to 1
>May 24 10:37:14 link-02 lvm[5604]: global/suffix not found in config: defaulting to 1
>May 24 10:37:14 link-02 lvm[5604]: global/units not found in config: defaulting to h
>May 24 10:37:14 link-02 lvm[5604]: Matcher built with 11 dfa states
>May 24 10:37:14 link-02 lvm[5604]: Setting devices/ignore_suspended_devices to 0
>May 24 10:37:14 link-02 lvm[5604]: Setting devices/cache to /etc/lvm/.cache
>May 24 10:37:14 link-02 lvm[5604]: Setting devices/write_cache_state to 1
>May 24 10:37:14 link-02 lvm[5604]: Opened /etc/lvm/.cache RO
>May 24 10:37:15 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0001000000000000-part1: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x3000910891003200:0x0001000000000000-part1: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410_mimage_3: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/dm-6: Aliased to /dev/mapper/lock_stress-link--07.26410_mimage_3 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram11: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part1: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/VolGroup00/LogVol01: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410_mimage_2: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/dm-5: Aliased to /dev/mapper/lock_stress-link--07.26410_mimage_2 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram10: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/VolGroup00/LogVol00: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0004000000000000-part1: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0006000000000000-part1: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410_mimage_0: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0005000000000000-part1: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram12: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/lock_stress/link-07.26410: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part2: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram6: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0007000000000000-part1: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410_mimage_1: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram13: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram14: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram5: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/mapper/VolGroup00-LogVol00: Aliased to /dev/VolGroup00/LogVol00 in device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/dm-2: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/hda1: Aliased to /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part1 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram1: Aliased to /dev/ram in device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/sda1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0001000000000000-part1 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/sdb1: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/sdc1: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/sdg1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0007000000000000-part1 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/sdd1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0004000000000000-part1 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/sde1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0005000000000000-part1 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/sdf1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0006000000000000-part1 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/sdh1: Aliased to /dev/disk/by-path/pci-0000:03:01.0-fc-0x3000910891003200:0x0001000000000000-part1 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/ramdisk: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0002000000000000-part1: Aliased to /dev/sdb1 in device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/mapper/VolGroup00-LogVol01: Aliased to /dev/VolGroup00/LogVol01 in device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/dm-3: Aliased to /dev/mapper/lock_stress-link--07.26410_mimage_0 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/dm-4: Aliased to /dev/mapper/lock_stress-link--07.26410_mimage_1 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/dm-7: Aliased to /dev/lock_stress/link-07.26410 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410_mlog: Aliased to /dev/dm-2 in device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram0: Aliased to /dev/ramdisk in device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/disk/by-path/pci-0000:03:01.0-fc-0x500805f3000a05b1:0x0003000000000000-part1: Aliased to /dev/sdc1 in device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/root: Aliased to /dev/VolGroup00/LogVol00 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/dm-1: Aliased to /dev/VolGroup00/LogVol01 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/hda2: Aliased to /dev/disk/by-path/pci-0000:00:07.1-ide-0:0-part2 in device cache (preferred name)
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram2: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram8: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram9: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/dm-0: Aliased to /dev/root in device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram15: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram3: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram4: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/ram7: Added to device cache
>May 24 10:37:15 link-02 lvm[5604]: /dev/mapper/lock_stress-link--07.26410: Aliased to /dev/dm-7 in device cache
>May 24 10:37:15 link-02 lvm[5604]: Loaded persistent filter cache from /etc/lvm/.cache
>May 24 10:37:15 link-02 lvm[5604]: Closed /etc/lvm/.cache
>May 24 10:37:15 link-02 lvm[5604]: Setting activation/reserved_stack to 256
>May 24 10:37:15 link-02 lvm[5604]: Setting activation/reserved_memory to 8192
>May 24 10:37:15 link-02 lvm[5604]: Setting activation/process_priority to -18
>May 24 10:37:15 link-02 lvm[5604]: Initialised format: lvm1
>May 24 10:37:15 link-02 lvm[5604]: Initialised format: pool
>May 24 10:37:15 link-02 lvm[5604]: Initialised format: lvm2
>May 24 10:37:15 link-02 lvm[5604]: global/format not found in config: defaulting to lvm2
>May 24 10:37:15 link-02 lvm[5604]: Initialised segtype: striped
>May 24 10:37:15 link-02 lvm[5604]: Initialised segtype: zero
>May 24 10:37:15 link-02 lvm[5604]: Initialised segtype: error
>May 24 10:37:15 link-02 lvm[5604]: Initialised segtype: snapshot
>May 24 10:37:15 link-02 lvm[5604]: Initialised segtype: mirror
>May 24 10:37:15 link-02 lvm[5604]: Setting backup/retain_days to 30
>May 24 10:37:15 link-02 lvm[5604]: Setting backup/retain_min to 10
>May 24 10:37:15 link-02 lvm[5604]: Setting backup/archive_dir to /etc/lvm/archive
>May 24 10:37:15 link-02 lvm[5604]: Setting backup/backup_dir to /etc/lvm/backup
>May 24 10:37:15 link-02 lvm[5604]: Locking memory
>May 24 10:37:15 link-02 lvm[5604]: memlock_count inc to 1
>May 24 10:37:15 link-02 lvm[5604]: Parsing: _memlock_inc
>May 24 10:37:59 link-02 kernel: CMAN: removing node link-08 from the cluster : Missed too many heartbeats
>May 24 10:38:00 link-02 fenced[5461]: fencing deferred to link-04
>May 24 10:38:39 link-02 kernel: dm-cmirror: A cluster mirror log member has failed.
>May 24 10:38:39 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (2dHqGlmO)
>May 24 10:38:39 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:38:39 link-02 kernel: dm-cmirror:   co-ordinator: 2
>May 24 10:38:39 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:38:39 link-02 kernel: dm-cmirror: LRT_SELECTION(11): (2dHqGlmO)
>May 24 10:38:39 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:38:39 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:38:39 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:38:39 link-02 kernel: dm-cmirror: LRT_MASTER_ASSIGN(12): (2dHqGlmO)
>May 24 10:38:39 link-02 kernel: dm-cmirror:   starter     : 2
>May 24 10:38:39 link-02 kernel: dm-cmirror:   co-ordinator: 1
>May 24 10:38:39 link-02 kernel: dm-cmirror:   node_count  : 1
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:38 link-02 kernel: dm-cmirror: server_id=dead, server_valid=1, 2dHqGlmO
>May 24 10:39:38 link-02 kernel: dm-cmirror: trigger = LRT_GET_RESYNC_WORK
>May 24 10:39:38 link-02 kernel: dm-cmirror: LRT_ELECTION(10): (2dHqGlmO)
>May 24 10:39:38 link-02 kernel: dm-cmirror:   starter     : 3
>May 24 10:39:38 link-02 kernel: dm-cmirror:   co-ordinator: 3
>May 24 10:39:38 link-02 kernel: dm-cmirror:   node_count  : 0
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:38 link-02 kernel: device-mapper: Primary mirror device has failed while mirror is out of sync.
>May 24 10:39:38 link-02 kernel: device-mapper: Unable to choose alternative primary device
>May 24 10:39:38 link-02 kernel: device-mapper: Read failure on mirror: Failing I/O.
>May 24 10:39:38 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:38 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 1040321
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:38 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:38 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:38 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:38 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:38 link-02 kernel: cdrom: open failed.
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 0
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 8
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 0
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 2533113728
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 2533112897
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 1
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 2
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 24
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 3
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 32
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 4
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 40
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 5
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 48
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 6
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 56
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 7
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 64
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 8
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 72
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 9
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 80
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 10
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 88
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 11
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 96
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 12
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 104
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 13
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 112
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 14
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 120
>May 24 10:39:38 link-02 kernel: Buffer I/O error on device sdh, logical block 15
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:38 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:38 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:38 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:38 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:38 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:38 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:38 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:38 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:38 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:38 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:38 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:38 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:39 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:39 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:39 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:39 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:39 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:39 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:39 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:39 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:39 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:39 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:39 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:39 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:39 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:39 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:39 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:39 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 10:39:39 link-02 kernel: device-mapper: A read failure occurred on a mirror device.
>May 24 10:39:39 link-02 kernel: device-mapper: Unable to retry read.
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 0
>May 24 10:39:39 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 10:39:39 link-02 kernel: end_request: I/O error, dev sdh, sector 16065
>May 24 10:40:42 link-02 kernel: CMAN: node link-04 has been removed from the cluster : Missed too many heartbeats
>May 24 10:40:43 link-02 kernel: CMAN: quorum lost, blocking activity
>May 24 10:40:46 link-02 ccsd[5316]: Cluster is not quorate.  Refusing connection. 
>May 24 10:40:46 link-02 ccsd[5316]: Error while processing connect: Connection refused 
>May 24 10:40:51 link-02 ccsd[5316]: Cluster is not quorate.  Refusing connection. 
>May 24 10:40:51 link-02 ccsd[5316]: Error while processing connect: Connection refused 
>May 24 10:40:56 link-02 ccsd[5316]: Cluster is not quorate.  Refusing connection. 
>May 24 10:40:56 link-02 ccsd[5316]: Error while processing connect: Connection refused 
>May 24 10:41:01 link-02 ccsd[5316]: Cluster is not quorate.  Refusing connection. 
>May 24 10:41:01 link-02 ccsd[5316]: Error while processing connect: Connection refused 
>May 24 10:41:06 link-02 ccsd[5316]: Cluster is not quorate.  Refusing connection. 
>May 24 10:41:06 link-02 ccsd[5316]: Error while processing connect: Connection refused 
>May 24 10:41:10 link-02 kernel: CMAN: node link-02 has been removed from the cluster : Missed too many heartbeats
>May 24 10:41:10 link-02 kernel: CMAN: killed by NODEDOWN message
>May 24 10:41:10 link-02 kernel: CMAN: we are leaving the cluster. Missed too many heartbeats
>May 24 10:41:10 link-02 kernel: WARNING: dlm_emergency_shutdown
>May 24 10:41:10 link-02 ccsd[5316]: Cluster manager shutdown.  Attemping to reconnect... 
>May 24 10:41:10 link-02 kernel: WARNING: dlm_emergency_shutdown
>May 24 10:41:10 link-02 kernel: SM: 00000005 sm_stop: SG still joined
>May 24 10:41:10 link-02 kernel: SM: 0100000a sm_stop: SG still joined
>May 24 10:41:25 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 30 seconds. 
>May 24 10:41:55 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 60 seconds. 
>May 24 10:42:25 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 90 seconds. 
>May 24 10:42:55 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 120 seconds. 
>May 24 10:43:25 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 150 seconds. 
>May 24 10:43:55 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 180 seconds. 
>May 24 10:44:26 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 210 seconds. 
>May 24 10:44:56 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 240 seconds. 
>May 24 10:45:26 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 270 seconds. 
>May 24 10:45:56 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 300 seconds. 
>May 24 10:46:26 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 330 seconds. 
>May 24 10:46:56 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 360 seconds. 
>May 24 10:47:26 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 390 seconds. 
>May 24 10:47:56 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 420 seconds. 
>May 24 10:48:26 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 450 seconds. 
>May 24 10:48:56 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 480 seconds. 
>May 24 10:49:26 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 510 seconds. 
>May 24 10:49:56 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 540 seconds. 
>May 24 10:50:26 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 570 seconds. 
>May 24 10:50:56 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 600 seconds. 
>May 24 10:51:26 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 630 seconds. 
>May 24 10:51:57 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 660 seconds. 
>May 24 10:52:27 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 690 seconds. 
>May 24 10:52:57 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 720 seconds. 
>May 24 10:53:27 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 750 seconds. 
>May 24 10:53:57 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 780 seconds. 
>May 24 10:54:27 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 810 seconds. 
>May 24 10:54:57 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 840 seconds. 
>May 24 10:55:27 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 870 seconds. 
>May 24 10:55:57 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 900 seconds. 
>May 24 10:56:27 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 930 seconds. 
>May 24 10:56:57 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 960 seconds. 
>May 24 10:57:27 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 990 seconds. 
>May 24 10:57:57 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1020 seconds. 
>May 24 10:58:27 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1050 seconds. 
>May 24 10:58:57 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1080 seconds. 
>May 24 10:59:28 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1110 seconds. 
>May 24 10:59:58 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1140 seconds. 
>May 24 11:00:28 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1170 seconds. 
>May 24 11:00:58 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1200 seconds. 
>May 24 11:01:28 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1230 seconds. 
>May 24 11:01:58 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1260 seconds. 
>May 24 11:02:28 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1290 seconds. 
>May 24 11:02:58 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1320 seconds. 
>May 24 11:03:28 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1350 seconds. 
>May 24 11:03:58 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1380 seconds. 
>May 24 11:04:28 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1410 seconds. 
>May 24 11:04:58 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1440 seconds. 
>May 24 11:05:28 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1470 seconds. 
>May 24 11:05:58 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1500 seconds. 
>May 24 11:06:28 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1530 seconds. 
>May 24 11:06:59 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1560 seconds. 
>May 24 11:07:29 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1590 seconds. 
>May 24 11:07:59 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1620 seconds. 
>May 24 11:08:29 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1650 seconds. 
>May 24 11:08:59 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1680 seconds. 
>May 24 11:09:29 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1710 seconds. 
>May 24 11:09:59 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1740 seconds. 
>May 24 11:10:29 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1770 seconds. 
>May 24 11:10:59 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1800 seconds. 
>May 24 11:11:29 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1830 seconds. 
>May 24 11:11:59 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1860 seconds. 
>May 24 11:12:29 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1890 seconds. 
>May 24 11:12:59 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1920 seconds. 
>May 24 11:13:29 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1950 seconds. 
>May 24 11:14:00 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 1980 seconds. 
>May 24 11:14:30 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2010 seconds. 
>May 24 11:15:00 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2040 seconds. 
>May 24 11:15:30 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2070 seconds. 
>May 24 11:16:00 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2100 seconds. 
>May 24 11:16:30 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2130 seconds. 
>May 24 11:17:00 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2160 seconds. 
>May 24 11:17:30 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2190 seconds. 
>May 24 11:18:00 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2220 seconds. 
>May 24 11:18:30 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2250 seconds. 
>May 24 11:19:00 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2280 seconds. 
>May 24 11:19:30 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2310 seconds. 
>May 24 11:20:00 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2340 seconds. 
>May 24 11:20:30 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2370 seconds. 
>May 24 11:21:00 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2400 seconds. 
>May 24 11:21:31 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2430 seconds. 
>May 24 11:22:01 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2460 seconds. 
>May 24 11:22:31 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2490 seconds. 
>May 24 11:23:01 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2520 seconds. 
>May 24 11:23:31 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2550 seconds. 
>May 24 11:24:01 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2580 seconds. 
>May 24 11:24:31 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2610 seconds. 
>May 24 11:25:01 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2640 seconds. 
>May 24 11:25:31 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2670 seconds. 
>May 24 11:26:01 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2700 seconds. 
>May 24 11:26:31 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2730 seconds. 
>May 24 11:26:44 link-02 kernel: SCSI error : <0 0 1 1> return code = 0x10000
>May 24 11:26:44 link-02 kernel: end_request: I/O error, dev sdh, sector 16449
>May 24 11:27:01 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2760 seconds. 
>May 24 11:27:31 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2790 seconds. 
>May 24 11:28:01 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2820 seconds. 
>May 24 11:28:31 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2850 seconds. 
>May 24 11:29:02 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2880 seconds. 
>May 24 11:29:32 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2910 seconds. 
>May 24 11:30:02 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2940 seconds. 
>May 24 11:30:32 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 2970 seconds. 
>May 24 11:31:02 link-02 ccsd[5316]: Unable to connect to cluster infrastructure after 3000 seconds.

Actions: View

Attachments on bug 241248: 155370 | 155371 | 155372 | 155373