Attachment 294121 Details for Bug 431696 – messages from taft-02 during the failure

messages from taft-02 during the failure

taft-02.log (text/plain), 81.28 KB, created by Corey Marthaler on 2008-02-06 16:10:25 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Corey Marthaler

Created: 2008-02-06 16:10:25 UTC

Size: 81.28 KB

patch

obsolete

>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram6) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram7) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram8) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram9) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram10) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram11) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram12) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram13) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram14) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram15) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:31:31 taft-02 kernel: sd 1:0:0:5: rejecting I/O to offline device
>Feb  6 09:31:31 taft-02 last message repeated 3 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:31:31 taft-02 kernel: sd 1:0:0:5: rejecting I/O to offline device
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 last message repeated 2 times
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:31 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:31 taft-02 kernel: sd 1:0:0:5: rejecting I/O to offline device
>Feb  6 09:31:32 taft-02 last message repeated 100 times
>Feb  6 09:31:32 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:32 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:32 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:32 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:32 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:32 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:32 taft-02 clogd[6717]: [vNwWKXtO] clog_postsuspend: leaving CPG 
>Feb  6 09:31:32 taft-02 clogd[6717]: [vNwWKXtO] clog_postsuspend: finalizing 
>Feb  6 09:31:51 taft-02 clogd[6717]: [vNwWKXtO] Cluster log removed 
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:51 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: Bad device removed from mirror volume, helter_skelter/syncd_primary_2legs_1
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: Mirror volume, helter_skelter/syncd_primary_2legs_1 converted to linear due to device failure.
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:31:52 taft-02 lvm[6805]: WARNING: dev_open(/etc/lvm/lvm.conf) called while suspended
>Feb  6 09:31:53 taft-02 lvm[6805]: No longer monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:32:06 taft-02 xinetd[6246]: START: qarsh pid=9422 from=10.15.80.47
>Feb  6 09:32:06 taft-02 qarshd[9422]: Talking to peer 10.15.80.47:53169
>Feb  6 09:32:06 taft-02 qarshd[9422]: Running cmdline: /usr/tests/sts-rhel5.2/bin/checkit -w /mnt/syncd_primary_2legs_1/checkit -f /tmp/checkit_syncd_primary_2legs_1 -v 
>Feb  6 09:32:09 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9422 duration=3(sec)
>Feb  6 09:32:16 taft-02 xinetd[6246]: START: qarsh pid=9424 from=10.15.80.47
>Feb  6 09:32:16 taft-02 qarshd[9424]: Talking to peer 10.15.80.47:53173
>Feb  6 09:32:16 taft-02 qarshd[9424]: Running cmdline: echo running > /sys/block/sdf/device/state 
>Feb  6 09:32:16 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9424 duration=0(sec)
>Feb  6 09:33:51 taft-02 clogd[6717]: [ESU1GepL] Cluster log created 
>Feb  6 09:33:51 taft-02 clogd[6717]: [ESU1GepL] Master resume: reading disk log 
>Feb  6 09:33:51 taft-02 [6805]: Monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:34:26 taft-02 lvm[6805]: helter_skelter-syncd_primary_2legs_1 is now in-sync 
>Feb  6 09:34:37 taft-02 clogd[6717]: [ESU1GepL] Inc reference count on cluster log 
>Feb  6 09:34:37 taft-02 clogd[6717]: [ESU1GepL] Cluster log created 
>Feb  6 09:34:37 taft-02 lvm[6805]: No longer monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:34:37 taft-02 clogd[6717]: [ESU1GepL] clog_postsuspend: leaving CPG 
>Feb  6 09:34:37 taft-02 clogd[6717]: [ESU1GepL] clog_postsuspend: finalizing 
>Feb  6 09:34:37 taft-02 clogd[6717]: [ESU1GepL] Dec reference count on cluster log 
>Feb  6 09:34:37 taft-02 clogd[6717]: [ESU1GepL] Non-master resume: bits pre-loaded 
>Feb  6 09:34:42 taft-02 kernel: device-mapper: dm-log-clustered: Timed out waiting for cluster log server [DM_CLOG_RESUME]
>Feb  6 09:34:42 taft-02 kernel: device-mapper: dm-log-clustered: Retrying request [DM_CLOG_RESUME]
>Feb  6 09:34:42 taft-02 kernel: device-mapper: dm-log-clustered: [69676][0][0][0][0][0] [3]
>Feb  6 09:34:42 taft-02 clogd[6717]: [ESU1GepL] ERROR:: Additional resume issued before suspend 
>Feb  6 09:34:42 taft-02 [6805]: Monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:34:42 taft-02 lvm[6805]: helter_skelter-syncd_primary_2legs_1 is now in-sync 
>Feb  6 09:36:57 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9339 duration=346(sec)
>Feb  6 09:37:11 taft-02 lvm[6805]: No longer monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:37:11 taft-02 clogd[6717]: [ESU1GepL] clog_postsuspend: leaving CPG 
>Feb  6 09:37:11 taft-02 clogd[6717]: [ESU1GepL] clog_postsuspend: finalizing 
>Feb  6 09:37:11 taft-02 clogd[6717]: [ESU1GepL] Cluster log removed 
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9660 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9660]: Talking to peer 10.15.80.47:50841
>Feb  6 09:37:54 taft-02 qarshd[9660]: Running cmdline: ls /dev/xvd* > /dev/null 2>&1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9660 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9663 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9663]: Talking to peer 10.15.80.47:50842
>Feb  6 09:37:54 taft-02 qarshd[9663]: Running cmdline: ls /dev/etherd/ > /dev/null 2>&1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9663 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9666 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9666]: Talking to peer 10.15.80.47:50843
>Feb  6 09:37:54 taft-02 qarshd[9666]: Running cmdline: cd /sys/block && ls -d sd* 2> /dev/null 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9666 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9669 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9669]: Talking to peer 10.15.80.47:50844
>Feb  6 09:37:54 taft-02 qarshd[9669]: Running cmdline: cd /sys/bus/pci/drivers/lpfc 2> /dev/null && ls -d [0-9]* 2> /dev/null 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9669 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9671 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9671]: Talking to peer 10.15.80.47:50845
>Feb  6 09:37:54 taft-02 qarshd[9671]: Running cmdline: cd /sys/bus/pci/drivers/qla2xxx 2> /dev/null && ls -d [0-9]* 2> /dev/null 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9671 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9674 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9674]: Talking to peer 10.15.80.47:50846
>Feb  6 09:37:54 taft-02 qarshd[9674]: Running cmdline: cd /sys/bus/pci/drivers/qla2300 2> /dev/null && ls -d [0-9]* 2> /dev/null 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9674 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9676 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9676]: Talking to peer 10.15.80.47:50847
>Feb  6 09:37:54 taft-02 qarshd[9676]: Running cmdline: cd /sys/bus/pci/drivers/qla2400 2> /dev/null && ls -d [0-9]* 2> /dev/null 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9676 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9678 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9678]: Talking to peer 10.15.80.47:50848
>Feb  6 09:37:54 taft-02 qarshd[9678]: Running cmdline: ls -l /sys/block/sda/device 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9678 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9680 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9680]: Talking to peer 10.15.80.47:50849
>Feb  6 09:37:54 taft-02 qarshd[9680]: Running cmdline: ls -l /sys/block/sdb/device 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9680 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9682 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9682]: Talking to peer 10.15.80.47:50850
>Feb  6 09:37:54 taft-02 qarshd[9682]: Running cmdline: ls -l /sys/block/sdc/device 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9682 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9684 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9684]: Talking to peer 10.15.80.47:50851
>Feb  6 09:37:54 taft-02 qarshd[9684]: Running cmdline: ls -l /sys/block/sdd/device 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9684 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9686 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9686]: Talking to peer 10.15.80.47:50852
>Feb  6 09:37:54 taft-02 qarshd[9686]: Running cmdline: ls -l /sys/block/sde/device 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9686 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9688 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9688]: Talking to peer 10.15.80.47:50853
>Feb  6 09:37:54 taft-02 qarshd[9688]: Running cmdline: ls -l /sys/block/sdf/device 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9688 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9690 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9690]: Talking to peer 10.15.80.47:50854
>Feb  6 09:37:54 taft-02 qarshd[9690]: Running cmdline: ls -l /sys/block/sdg/device 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9690 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9692 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9692]: Talking to peer 10.15.80.47:50855
>Feb  6 09:37:54 taft-02 qarshd[9692]: Running cmdline: ls -l /sys/block/sdh/device 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9692 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9694 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9694]: Talking to peer 10.15.80.47:50856
>Feb  6 09:37:54 taft-02 qarshd[9694]: Running cmdline: find /sys/block/sdb -name "sdb[1-9]*" -type d -printf "%f " 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9694 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9696 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9696]: Talking to peer 10.15.80.47:50857
>Feb  6 09:37:54 taft-02 qarshd[9696]: Running cmdline: df | grep /dev/sdb1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9696 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9700 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9700]: Talking to peer 10.15.80.47:50858
>Feb  6 09:37:54 taft-02 qarshd[9700]: Running cmdline: dd if=/dev/sdb of=/dev/null count=1 > /dev/null 2>&1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9700 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9703 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9703]: Talking to peer 10.15.80.47:50859
>Feb  6 09:37:54 taft-02 qarshd[9703]: Running cmdline: find /sys/block/sdc -name "sdc[1-9]*" -type d -printf "%f " 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9703 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9705 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9705]: Talking to peer 10.15.80.47:50860
>Feb  6 09:37:54 taft-02 qarshd[9705]: Running cmdline: df | grep /dev/sdc1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9705 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9709 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9709]: Talking to peer 10.15.80.47:50861
>Feb  6 09:37:54 taft-02 qarshd[9709]: Running cmdline: dd if=/dev/sdc of=/dev/null count=1 > /dev/null 2>&1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9709 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9712 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9712]: Talking to peer 10.15.80.47:50862
>Feb  6 09:37:54 taft-02 qarshd[9712]: Running cmdline: find /sys/block/sdd -name "sdd[1-9]*" -type d -printf "%f " 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9712 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9714 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9714]: Talking to peer 10.15.80.47:50863
>Feb  6 09:37:54 taft-02 qarshd[9714]: Running cmdline: df | grep /dev/sdd1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9714 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9718 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9718]: Talking to peer 10.15.80.47:50864
>Feb  6 09:37:54 taft-02 qarshd[9718]: Running cmdline: dd if=/dev/sdd of=/dev/null count=1 > /dev/null 2>&1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9718 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9721 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9721]: Talking to peer 10.15.80.47:50865
>Feb  6 09:37:54 taft-02 qarshd[9721]: Running cmdline: find /sys/block/sde -name "sde[1-9]*" -type d -printf "%f " 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9721 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9723 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9723]: Talking to peer 10.15.80.47:50866
>Feb  6 09:37:54 taft-02 qarshd[9723]: Running cmdline: df | grep /dev/sde1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9723 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9727 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9727]: Talking to peer 10.15.80.47:50867
>Feb  6 09:37:54 taft-02 qarshd[9727]: Running cmdline: dd if=/dev/sde of=/dev/null count=1 > /dev/null 2>&1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9727 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9730 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9730]: Talking to peer 10.15.80.47:50868
>Feb  6 09:37:54 taft-02 qarshd[9730]: Running cmdline: find /sys/block/sdf -name "sdf[1-9]*" -type d -printf "%f " 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9730 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9732 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9732]: Talking to peer 10.15.80.47:50869
>Feb  6 09:37:54 taft-02 qarshd[9732]: Running cmdline: df | grep /dev/sdf1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9732 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9736 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9736]: Talking to peer 10.15.80.47:50870
>Feb  6 09:37:54 taft-02 qarshd[9736]: Running cmdline: dd if=/dev/sdf of=/dev/null count=1 > /dev/null 2>&1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9736 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9739 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9739]: Talking to peer 10.15.80.47:50871
>Feb  6 09:37:54 taft-02 qarshd[9739]: Running cmdline: find /sys/block/sdg -name "sdg[1-9]*" -type d -printf "%f " 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9739 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9741 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9741]: Talking to peer 10.15.80.47:50872
>Feb  6 09:37:54 taft-02 qarshd[9741]: Running cmdline: df | grep /dev/sdg1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9741 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9745 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9745]: Talking to peer 10.15.80.47:50873
>Feb  6 09:37:54 taft-02 qarshd[9745]: Running cmdline: dd if=/dev/sdg of=/dev/null count=1 > /dev/null 2>&1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9745 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9748 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9748]: Talking to peer 10.15.80.47:50874
>Feb  6 09:37:54 taft-02 qarshd[9748]: Running cmdline: find /sys/block/sdh -name "sdh[1-9]*" -type d -printf "%f " 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9748 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9750 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9750]: Talking to peer 10.15.80.47:50875
>Feb  6 09:37:54 taft-02 qarshd[9750]: Running cmdline: df | grep /dev/sdh1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9750 duration=0(sec)
>Feb  6 09:37:54 taft-02 xinetd[6246]: START: qarsh pid=9754 from=10.15.80.47
>Feb  6 09:37:54 taft-02 qarshd[9754]: Talking to peer 10.15.80.47:50876
>Feb  6 09:37:54 taft-02 qarshd[9754]: Running cmdline: dd if=/dev/sdh of=/dev/null count=1 > /dev/null 2>&1 
>Feb  6 09:37:54 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9754 duration=0(sec)
>Feb  6 09:37:56 taft-02 xinetd[6246]: START: qarsh pid=9757 from=10.15.80.47
>Feb  6 09:37:56 taft-02 qarshd[9757]: Talking to peer 10.15.80.47:50949
>Feb  6 09:37:56 taft-02 qarshd[9757]: Running cmdline: /usr/tests/sts-rhel5.2/bin/dice -d /dev/sdb -p 1 2>&1 
>Feb  6 09:37:56 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9757 duration=0(sec)
>Feb  6 09:37:56 taft-02 xinetd[6246]: START: qarsh pid=9762 from=10.15.80.47
>Feb  6 09:37:56 taft-02 qarshd[9762]: Talking to peer 10.15.80.47:50950
>Feb  6 09:37:56 taft-02 qarshd[9762]: Running cmdline: /usr/tests/sts-rhel5.2/bin/dice -d /dev/sdc -p 1 2>&1 
>Feb  6 09:37:56 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9762 duration=0(sec)
>Feb  6 09:37:56 taft-02 xinetd[6246]: START: qarsh pid=9768 from=10.15.80.47
>Feb  6 09:37:56 taft-02 qarshd[9768]: Talking to peer 10.15.80.47:50951
>Feb  6 09:37:56 taft-02 qarshd[9768]: Running cmdline: /usr/tests/sts-rhel5.2/bin/dice -d /dev/sdd -p 1 2>&1 
>Feb  6 09:37:56 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9768 duration=0(sec)
>Feb  6 09:37:56 taft-02 xinetd[6246]: START: qarsh pid=9780 from=10.15.80.47
>Feb  6 09:37:56 taft-02 qarshd[9780]: Talking to peer 10.15.80.47:50952
>Feb  6 09:37:56 taft-02 qarshd[9780]: Running cmdline: /usr/tests/sts-rhel5.2/bin/dice -d /dev/sde -p 1 2>&1 
>Feb  6 09:37:56 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9780 duration=0(sec)
>Feb  6 09:37:56 taft-02 xinetd[6246]: START: qarsh pid=9793 from=10.15.80.47
>Feb  6 09:37:56 taft-02 qarshd[9793]: Talking to peer 10.15.80.47:50953
>Feb  6 09:37:56 taft-02 qarshd[9793]: Running cmdline: /usr/tests/sts-rhel5.2/bin/dice -d /dev/sdf -p 1 2>&1 
>Feb  6 09:37:56 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9793 duration=0(sec)
>Feb  6 09:37:56 taft-02 xinetd[6246]: START: qarsh pid=9809 from=10.15.80.47
>Feb  6 09:37:56 taft-02 qarshd[9809]: Talking to peer 10.15.80.47:50954
>Feb  6 09:37:57 taft-02 qarshd[9809]: Running cmdline: /usr/tests/sts-rhel5.2/bin/dice -d /dev/sdg -p 1 2>&1 
>Feb  6 09:37:57 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9809 duration=1(sec)
>Feb  6 09:37:57 taft-02 xinetd[6246]: START: qarsh pid=9824 from=10.15.80.47
>Feb  6 09:37:57 taft-02 qarshd[9824]: Talking to peer 10.15.80.47:50955
>Feb  6 09:37:57 taft-02 qarshd[9824]: Running cmdline: /usr/tests/sts-rhel5.2/bin/dice -d /dev/sdh -p 1 2>&1 
>Feb  6 09:37:57 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9824 duration=0(sec)
>Feb  6 09:37:59 taft-02 xinetd[6246]: START: qarsh pid=9869 from=10.15.80.47
>Feb  6 09:37:59 taft-02 qarshd[9869]: Talking to peer 10.15.80.47:50963
>Feb  6 09:37:59 taft-02 qarshd[9869]: Running cmdline: /usr/tests/sts-rhel5.2/bin/readpart /dev/sdb 
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdb: 284511150 512-byte hdwr sectors (145670 MB)
>Feb  6 09:37:59 taft-02 kernel: sdb: Write Protect is off
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdb: drive cache: write back
>Feb  6 09:37:59 taft-02 kernel:  sdb: sdb1
>Feb  6 09:37:59 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9869 duration=0(sec)
>Feb  6 09:37:59 taft-02 xinetd[6246]: START: qarsh pid=9873 from=10.15.80.47
>Feb  6 09:37:59 taft-02 qarshd[9873]: Talking to peer 10.15.80.47:50964
>Feb  6 09:37:59 taft-02 qarshd[9873]: Running cmdline: /usr/tests/sts-rhel5.2/bin/readpart /dev/sdc 
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdc: 284511150 512-byte hdwr sectors (145670 MB)
>Feb  6 09:37:59 taft-02 kernel: sdc: Write Protect is off
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdc: drive cache: write back
>Feb  6 09:37:59 taft-02 kernel:  sdc: sdc1
>Feb  6 09:37:59 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9873 duration=0(sec)
>Feb  6 09:37:59 taft-02 xinetd[6246]: START: qarsh pid=9877 from=10.15.80.47
>Feb  6 09:37:59 taft-02 qarshd[9877]: Talking to peer 10.15.80.47:50965
>Feb  6 09:37:59 taft-02 qarshd[9877]: Running cmdline: /usr/tests/sts-rhel5.2/bin/readpart /dev/sdd 
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdd: 284511150 512-byte hdwr sectors (145670 MB)
>Feb  6 09:37:59 taft-02 kernel: sdd: Write Protect is off
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdd: drive cache: write back
>Feb  6 09:37:59 taft-02 kernel:  sdd: sdd1
>Feb  6 09:37:59 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9877 duration=0(sec)
>Feb  6 09:37:59 taft-02 xinetd[6246]: START: qarsh pid=9881 from=10.15.80.47
>Feb  6 09:37:59 taft-02 qarshd[9881]: Talking to peer 10.15.80.47:50966
>Feb  6 09:37:59 taft-02 qarshd[9881]: Running cmdline: /usr/tests/sts-rhel5.2/bin/readpart /dev/sde 
>Feb  6 09:37:59 taft-02 kernel: SCSI device sde: 284511150 512-byte hdwr sectors (145670 MB)
>Feb  6 09:37:59 taft-02 kernel: sde: Write Protect is off
>Feb  6 09:37:59 taft-02 kernel: SCSI device sde: drive cache: write back
>Feb  6 09:37:59 taft-02 kernel:  sde: sde1
>Feb  6 09:37:59 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9881 duration=0(sec)
>Feb  6 09:37:59 taft-02 xinetd[6246]: START: qarsh pid=9886 from=10.15.80.47
>Feb  6 09:37:59 taft-02 qarshd[9886]: Talking to peer 10.15.80.47:50967
>Feb  6 09:37:59 taft-02 qarshd[9886]: Running cmdline: /usr/tests/sts-rhel5.2/bin/readpart /dev/sdf 
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdf: 284511150 512-byte hdwr sectors (145670 MB)
>Feb  6 09:37:59 taft-02 kernel: sdf: Write Protect is off
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdf: drive cache: write back
>Feb  6 09:37:59 taft-02 kernel:  sdf: sdf1
>Feb  6 09:37:59 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9886 duration=0(sec)
>Feb  6 09:37:59 taft-02 xinetd[6246]: START: qarsh pid=9891 from=10.15.80.47
>Feb  6 09:37:59 taft-02 qarshd[9891]: Talking to peer 10.15.80.47:50968
>Feb  6 09:37:59 taft-02 qarshd[9891]: Running cmdline: /usr/tests/sts-rhel5.2/bin/readpart /dev/sdg 
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdg: 284511150 512-byte hdwr sectors (145670 MB)
>Feb  6 09:37:59 taft-02 kernel: sdg: Write Protect is off
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdg: drive cache: write back
>Feb  6 09:37:59 taft-02 kernel:  sdg: sdg1
>Feb  6 09:37:59 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9891 duration=0(sec)
>Feb  6 09:37:59 taft-02 xinetd[6246]: START: qarsh pid=9896 from=10.15.80.47
>Feb  6 09:37:59 taft-02 qarshd[9896]: Talking to peer 10.15.80.47:50969
>Feb  6 09:37:59 taft-02 qarshd[9896]: Running cmdline: /usr/tests/sts-rhel5.2/bin/readpart /dev/sdh 
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdh: 284511150 512-byte hdwr sectors (145670 MB)
>Feb  6 09:37:59 taft-02 kernel: sdh: Write Protect is off
>Feb  6 09:37:59 taft-02 kernel: SCSI device sdh: drive cache: write back
>Feb  6 09:37:59 taft-02 kernel:  sdh: sdh1
>Feb  6 09:37:59 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9896 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9974 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9974]: Talking to peer 10.15.80.47:50984
>Feb  6 09:38:10 taft-02 qarshd[9974]: Running cmdline: ls /dev/xvd* > /dev/null 2>&1 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9974 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9977 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9977]: Talking to peer 10.15.80.47:50985
>Feb  6 09:38:10 taft-02 qarshd[9977]: Running cmdline: ls /dev/etherd/ > /dev/null 2>&1 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9977 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9980 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9980]: Talking to peer 10.15.80.47:50986
>Feb  6 09:38:10 taft-02 qarshd[9980]: Running cmdline: cd /sys/block && ls -d sd* 2> /dev/null 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9980 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9983 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9983]: Talking to peer 10.15.80.47:50987
>Feb  6 09:38:10 taft-02 qarshd[9983]: Running cmdline: cd /sys/bus/pci/drivers/lpfc 2> /dev/null && ls -d [0-9]* 2> /dev/null 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9983 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9985 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9985]: Talking to peer 10.15.80.47:50988
>Feb  6 09:38:10 taft-02 qarshd[9985]: Running cmdline: cd /sys/bus/pci/drivers/qla2xxx 2> /dev/null && ls -d [0-9]* 2> /dev/null 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9985 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9988 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9988]: Talking to peer 10.15.80.47:50989
>Feb  6 09:38:10 taft-02 qarshd[9988]: Running cmdline: cd /sys/bus/pci/drivers/qla2300 2> /dev/null && ls -d [0-9]* 2> /dev/null 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9988 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9990 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9990]: Talking to peer 10.15.80.47:50990
>Feb  6 09:38:10 taft-02 qarshd[9990]: Running cmdline: cd /sys/bus/pci/drivers/qla2400 2> /dev/null && ls -d [0-9]* 2> /dev/null 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9990 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9992 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9992]: Talking to peer 10.15.80.47:50991
>Feb  6 09:38:10 taft-02 qarshd[9992]: Running cmdline: ls -l /sys/block/sda/device 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9992 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9994 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9994]: Talking to peer 10.15.80.47:50992
>Feb  6 09:38:10 taft-02 qarshd[9994]: Running cmdline: ls -l /sys/block/sdb/device 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9994 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9996 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9996]: Talking to peer 10.15.80.47:50993
>Feb  6 09:38:10 taft-02 qarshd[9996]: Running cmdline: ls -l /sys/block/sdc/device 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9996 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=9998 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[9998]: Talking to peer 10.15.80.47:50994
>Feb  6 09:38:10 taft-02 qarshd[9998]: Running cmdline: ls -l /sys/block/sdd/device 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=9998 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10000 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10000]: Talking to peer 10.15.80.47:50995
>Feb  6 09:38:10 taft-02 qarshd[10000]: Running cmdline: ls -l /sys/block/sde/device 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10000 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10002 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10002]: Talking to peer 10.15.80.47:50996
>Feb  6 09:38:10 taft-02 qarshd[10002]: Running cmdline: ls -l /sys/block/sdf/device 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10002 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10004 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10004]: Talking to peer 10.15.80.47:50997
>Feb  6 09:38:10 taft-02 qarshd[10004]: Running cmdline: ls -l /sys/block/sdg/device 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10004 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10006 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10006]: Talking to peer 10.15.80.47:50998
>Feb  6 09:38:10 taft-02 qarshd[10006]: Running cmdline: ls -l /sys/block/sdh/device 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10006 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10008 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10008]: Talking to peer 10.15.80.47:50999
>Feb  6 09:38:10 taft-02 qarshd[10008]: Running cmdline: find /sys/block/sdb -name "sdb[1-9]*" -type d -printf "%f " 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10008 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10010 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10010]: Talking to peer 10.15.80.47:51000
>Feb  6 09:38:10 taft-02 qarshd[10010]: Running cmdline: df | grep /dev/sdb1 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10010 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10014 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10014]: Talking to peer 10.15.80.47:51001
>Feb  6 09:38:10 taft-02 qarshd[10014]: Running cmdline: find /sys/block/sdc -name "sdc[1-9]*" -type d -printf "%f " 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10014 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10016 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10016]: Talking to peer 10.15.80.47:51002
>Feb  6 09:38:10 taft-02 qarshd[10016]: Running cmdline: df | grep /dev/sdc1 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10016 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10020 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10020]: Talking to peer 10.15.80.47:51003
>Feb  6 09:38:10 taft-02 qarshd[10020]: Running cmdline: find /sys/block/sdd -name "sdd[1-9]*" -type d -printf "%f " 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10020 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10022 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10022]: Talking to peer 10.15.80.47:51004
>Feb  6 09:38:10 taft-02 qarshd[10022]: Running cmdline: df | grep /dev/sdd1 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10022 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10026 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10026]: Talking to peer 10.15.80.47:51005
>Feb  6 09:38:10 taft-02 qarshd[10026]: Running cmdline: find /sys/block/sde -name "sde[1-9]*" -type d -printf "%f " 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10026 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10028 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10028]: Talking to peer 10.15.80.47:51006
>Feb  6 09:38:10 taft-02 qarshd[10028]: Running cmdline: df | grep /dev/sde1 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10028 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10032 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10032]: Talking to peer 10.15.80.47:51007
>Feb  6 09:38:10 taft-02 qarshd[10032]: Running cmdline: find /sys/block/sdf -name "sdf[1-9]*" -type d -printf "%f " 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10032 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10034 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10034]: Talking to peer 10.15.80.47:51008
>Feb  6 09:38:10 taft-02 qarshd[10034]: Running cmdline: df | grep /dev/sdf1 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10034 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10038 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10038]: Talking to peer 10.15.80.47:51009
>Feb  6 09:38:10 taft-02 qarshd[10038]: Running cmdline: find /sys/block/sdg -name "sdg[1-9]*" -type d -printf "%f " 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10038 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10040 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10040]: Talking to peer 10.15.80.47:51010
>Feb  6 09:38:10 taft-02 qarshd[10040]: Running cmdline: df | grep /dev/sdg1 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10040 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10044 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10044]: Talking to peer 10.15.80.47:51011
>Feb  6 09:38:10 taft-02 qarshd[10044]: Running cmdline: find /sys/block/sdh -name "sdh[1-9]*" -type d -printf "%f " 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10044 duration=0(sec)
>Feb  6 09:38:10 taft-02 xinetd[6246]: START: qarsh pid=10046 from=10.15.80.47
>Feb  6 09:38:10 taft-02 qarshd[10046]: Talking to peer 10.15.80.47:51012
>Feb  6 09:38:10 taft-02 qarshd[10046]: Running cmdline: df | grep /dev/sdh1 
>Feb  6 09:38:10 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10046 duration=0(sec)
>Feb  6 09:38:11 taft-02 xinetd[6246]: START: qarsh pid=10050 from=10.15.80.47
>Feb  6 09:38:11 taft-02 qarshd[10050]: Talking to peer 10.15.80.47:51014
>Feb  6 09:38:11 taft-02 qarshd[10050]: Running cmdline: vgscan 
>Feb  6 09:38:11 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10050 duration=0(sec)
>Feb  6 09:38:12 taft-02 xinetd[6246]: START: qarsh pid=10053 from=10.15.80.47
>Feb  6 09:38:12 taft-02 qarshd[10053]: Talking to peer 10.15.80.47:51017
>Feb  6 09:38:12 taft-02 qarshd[10053]: Running cmdline: pvcreate /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 
>Feb  6 09:38:13 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10053 duration=1(sec)
>Feb  6 09:38:13 taft-02 xinetd[6246]: START: qarsh pid=10056 from=10.15.80.47
>Feb  6 09:38:13 taft-02 qarshd[10056]: Talking to peer 10.15.80.47:51018
>Feb  6 09:38:13 taft-02 qarshd[10056]: Running cmdline: vgcreate helter_skelter /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 
>Feb  6 09:38:13 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10056 duration=0(sec)
>Feb  6 09:38:17 taft-02 xinetd[6246]: START: qarsh pid=10059 from=10.15.80.47
>Feb  6 09:38:17 taft-02 qarshd[10059]: Talking to peer 10.15.80.47:51022
>Feb  6 09:38:17 taft-02 qarshd[10059]: Running cmdline: pvs --noheadings -o +pv_uuid 
>Feb  6 09:38:18 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10059 duration=1(sec)
>Feb  6 09:38:24 taft-02 clogd[6717]: [aCNfPqgK] Cluster log created 
>Feb  6 09:38:24 taft-02 clogd[6717]: [aCNfPqgK] Master resume: reading disk log 
>Feb  6 09:38:24 taft-02 [6805]: Monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:39:18 taft-02 xinetd[6246]: START: qarsh pid=10154 from=10.15.80.47
>Feb  6 09:39:18 taft-02 qarshd[10154]: Talking to peer 10.15.80.47:51035
>Feb  6 09:39:18 taft-02 qarshd[10154]: Running cmdline: mkdir -p /mnt/syncd_primary_2legs_1 
>Feb  6 09:39:18 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10154 duration=0(sec)
>Feb  6 09:39:18 taft-02 xinetd[6246]: START: qarsh pid=10156 from=10.15.80.47
>Feb  6 09:39:18 taft-02 qarshd[10156]: Talking to peer 10.15.80.47:51036
>Feb  6 09:39:18 taft-02 qarshd[10156]: Running cmdline: mount /dev/helter_skelter/syncd_primary_2legs_1 /mnt/syncd_primary_2legs_1 
>Feb  6 09:39:18 taft-02 lvm[6805]: helter_skelter-syncd_primary_2legs_1 is now in-sync 
>Feb  6 09:39:18 taft-02 kernel: Trying to join cluster "lock_dlm", "TAFT:gfs1"
>Feb  6 09:39:18 taft-02 kernel: Joined cluster. Now mounting FS...
>Feb  6 09:39:18 taft-02 kernel: GFS: fsid=TAFT:gfs1.1: jid=1: Trying to acquire journal lock...
>Feb  6 09:39:18 taft-02 kernel: GFS: fsid=TAFT:gfs1.1: jid=1: Looking at journal...
>Feb  6 09:39:18 taft-02 kernel: GFS: fsid=TAFT:gfs1.1: jid=1: Done
>Feb  6 09:39:18 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10156 duration=0(sec)
>Feb  6 09:39:20 taft-02 xinetd[6246]: START: qarsh pid=10190 from=10.15.80.47
>Feb  6 09:39:20 taft-02 qarshd[10190]: Talking to peer 10.15.80.47:51043
>Feb  6 09:39:20 taft-02 qarshd[10190]: Running cmdline: mkdir -p /mnt/syncd_primary_2legs_1/checkit 
>Feb  6 09:39:20 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10190 duration=0(sec)
>Feb  6 09:39:20 taft-02 xinetd[6246]: START: qarsh pid=10192 from=10.15.80.47
>Feb  6 09:39:20 taft-02 qarshd[10192]: Talking to peer 10.15.80.47:51044
>Feb  6 09:39:20 taft-02 qarshd[10192]: Running cmdline: /usr/tests/sts-rhel5.2/bin/checkit -w /mnt/syncd_primary_2legs_1/checkit -f /tmp/checkit_syncd_primary_2legs_1 -n 100 
>Feb  6 09:39:21 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10192 duration=1(sec)
>Feb  6 09:39:23 taft-02 xinetd[6246]: START: qarsh pid=10194 from=10.15.80.47
>Feb  6 09:39:23 taft-02 qarshd[10194]: Talking to peer 10.15.80.47:51052
>Feb  6 09:39:23 taft-02 qarshd[10194]: Running cmdline: /usr/tests/sts-rhel5.2/bin/xiogen -f buffered -m random -s write,writev -t 100 -T 1000000  -F 1000000:/mnt/syncd_primary_2legs_1/taft-02_load | /usr/tests/sts-rhel5.2/bin/xdoio -vD 
>Feb  6 09:39:39 taft-02 xinetd[6246]: START: qarsh pid=10199 from=10.15.80.47
>Feb  6 09:39:39 taft-02 qarshd[10199]: Talking to peer 10.15.80.47:51055
>Feb  6 09:39:39 taft-02 qarshd[10199]: Running cmdline: echo offline > /sys/block/sdg/device/state 
>Feb  6 09:39:39 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10199 duration=0(sec)
>Feb  6 09:39:43 taft-02 kernel: sd 1:0:0:6: rejecting I/O to offline device
>Feb  6 09:39:43 taft-02 lvm[6805]: Mirror device, 253:3, has failed. 
>Feb  6 09:39:43 taft-02 lvm[6805]: Device failure in helter_skelter-syncd_primary_2legs_1 
>Feb  6 09:39:43 taft-02 lvm[6805]: WARNING: dev_open(/etc/lvm/lvm.conf) called while suspended
>Feb  6 09:39:43 taft-02 kernel: sd 1:0:0:6: rejecting I/O to offline device
>Feb  6 09:39:43 taft-02 kernel: sd 1:0:0:6: rejecting I/O to offline device
>Feb  6 09:39:44 taft-02 clogd[6717]: [aCNfPqgK] clog_postsuspend: leaving CPG 
>Feb  6 09:39:44 taft-02 clogd[6717]: [aCNfPqgK] clog_postsuspend: finalizing 
>Feb  6 09:39:44 taft-02 clogd[6717]: [aCNfPqgK] Cluster log removed 
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ramdisk) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/cdrom) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sda) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sda) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/md0) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/root) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sda1) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/VolGroup00/LogVol01) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram2) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sda2) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram3) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram4) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram5) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/helter_skelter/syncd_primary_2legs_1) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram6) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram7) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram8) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram9) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram10) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram11) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram12) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram13) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram14) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram15) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:39:46 taft-02 last message repeated 2 times
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:39:46 taft-02 lvm[6805]: WARNING: dev_open(/etc/lvm/lvm.conf) called while suspended
>Feb  6 09:39:48 taft-02 lvm[6805]: No longer monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:40:00 taft-02 xinetd[6246]: START: qarsh pid=10270 from=10.15.80.47
>Feb  6 09:40:00 taft-02 qarshd[10270]: Talking to peer 10.15.80.47:51065
>Feb  6 09:40:00 taft-02 qarshd[10270]: Running cmdline: /usr/tests/sts-rhel5.2/bin/checkit -w /mnt/syncd_primary_2legs_1/checkit -f /tmp/checkit_syncd_primary_2legs_1 -v 
>Feb  6 09:40:03 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10270 duration=3(sec)
>Feb  6 09:40:09 taft-02 xinetd[6246]: START: qarsh pid=10273 from=10.15.80.47
>Feb  6 09:40:09 taft-02 qarshd[10273]: Talking to peer 10.15.80.47:51069
>Feb  6 09:40:09 taft-02 qarshd[10273]: Running cmdline: echo running > /sys/block/sdg/device/state 
>Feb  6 09:40:09 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10273 duration=0(sec)
>Feb  6 09:40:13 taft-02 clogd[6717]: [pZeLHWU8] Cluster log created 
>Feb  6 09:40:13 taft-02 clogd[6717]: [pZeLHWU8] Master resume: reading disk log 
>Feb  6 09:40:13 taft-02 [6805]: Monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:40:49 taft-02 lvm[6805]: helter_skelter-syncd_primary_2legs_1 is now in-sync 
>Feb  6 09:41:00 taft-02 clogd[6717]: [pZeLHWU8] Inc reference count on cluster log 
>Feb  6 09:41:00 taft-02 clogd[6717]: [pZeLHWU8] Cluster log created 
>Feb  6 09:41:00 taft-02 lvm[6805]: No longer monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:41:00 taft-02 clogd[6717]: [pZeLHWU8] clog_postsuspend: leaving CPG 
>Feb  6 09:41:00 taft-02 clogd[6717]: [pZeLHWU8] clog_postsuspend: finalizing 
>Feb  6 09:41:00 taft-02 clogd[6717]: [pZeLHWU8] Dec reference count on cluster log 
>Feb  6 09:41:00 taft-02 clogd[6717]: [pZeLHWU8] Non-master resume: bits pre-loaded 
>Feb  6 09:41:00 taft-02 [6805]: Monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:41:00 taft-02 lvm[6805]: helter_skelter-syncd_primary_2legs_1 is now in-sync 
>Feb  6 09:41:16 taft-02 xinetd[6246]: START: qarsh pid=10397 from=10.15.80.47
>Feb  6 09:41:16 taft-02 qarshd[10397]: Talking to peer 10.15.80.47:51082
>Feb  6 09:41:16 taft-02 qarshd[10397]: Running cmdline: /usr/tests/sts-rhel5.2/bin/checkit -w /mnt/syncd_primary_2legs_1/checkit -f /tmp/checkit_syncd_primary_2legs_1 -v 
>Feb  6 09:41:19 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10397 duration=3(sec)
>Feb  6 09:41:26 taft-02 qarshd[10194]: Sending child 10195 signal 2
>Feb  6 09:41:26 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10194 duration=123(sec)
>Feb  6 09:41:36 taft-02 xinetd[6246]: START: qarsh pid=10400 from=10.15.80.47
>Feb  6 09:41:36 taft-02 qarshd[10400]: Talking to peer 10.15.80.47:51087
>Feb  6 09:41:36 taft-02 qarshd[10400]: Running cmdline: rm -f /tmp/checkit_syncd_primary_2legs_1 
>Feb  6 09:41:36 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10400 duration=0(sec)
>Feb  6 09:41:37 taft-02 xinetd[6246]: START: qarsh pid=10402 from=10.15.80.47
>Feb  6 09:41:37 taft-02 qarshd[10402]: Talking to peer 10.15.80.47:51092
>Feb  6 09:41:37 taft-02 qarshd[10402]: Running cmdline: umount /mnt/syncd_primary_2legs_1 
>Feb  6 09:41:37 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10402 duration=0(sec)
>Feb  6 09:41:37 taft-02 xinetd[6246]: START: qarsh pid=10417 from=10.15.80.47
>Feb  6 09:41:37 taft-02 qarshd[10417]: Talking to peer 10.15.80.47:51093
>Feb  6 09:41:37 taft-02 qarshd[10417]: Running cmdline: rmdir /mnt/syncd_primary_2legs_1 
>Feb  6 09:41:37 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10417 duration=0(sec)
>Feb  6 09:41:38 taft-02 lvm[6805]: No longer monitoring mirror device helter_skelter-syncd_primary_2legs_1 for events 
>Feb  6 09:41:39 taft-02 clogd[6717]: [pZeLHWU8] clog_postsuspend: leaving CPG 
>Feb  6 09:41:39 taft-02 clogd[6717]: [pZeLHWU8] clog_postsuspend: finalizing 
>Feb  6 09:41:39 taft-02 clogd[6717]: [pZeLHWU8] Cluster log removed 
>Feb  6 09:41:40 taft-02 xinetd[6246]: START: qarsh pid=10456 from=10.15.80.47
>Feb  6 09:41:40 taft-02 qarshd[10456]: Talking to peer 10.15.80.47:51104
>Feb  6 09:41:40 taft-02 qarshd[10456]: Running cmdline: pvs -o +vg_name 
>Feb  6 09:41:42 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10456 duration=2(sec)
>Feb  6 09:41:42 taft-02 xinetd[6246]: START: qarsh pid=10479 from=10.15.80.47
>Feb  6 09:41:42 taft-02 qarshd[10479]: Talking to peer 10.15.80.47:51105
>Feb  6 09:41:42 taft-02 qarshd[10479]: Running cmdline: pvs --noheadings -o pv_uuid /dev/sdg1 
>Feb  6 09:41:42 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10479 duration=0(sec)
>Feb  6 09:41:44 taft-02 xinetd[6246]: START: qarsh pid=10482 from=10.15.80.47
>Feb  6 09:41:44 taft-02 qarshd[10482]: Talking to peer 10.15.80.47:51107
>Feb  6 09:41:44 taft-02 qarshd[10482]: Running cmdline: pvs --noheadings -o +pv_uuid 
>Feb  6 09:41:45 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10482 duration=1(sec)
>Feb  6 09:41:48 taft-02 xinetd[6246]: START: qarsh pid=10485 from=10.15.80.47
>Feb  6 09:41:48 taft-02 qarshd[10485]: Talking to peer 10.15.80.47:51110
>Feb  6 09:41:48 taft-02 qarshd[10485]: Running cmdline: lvcreate -m 2 -n syncd_primary_3legs_1 -L 800M helter_skelter /dev/sdg1:0-1000 /dev/sdd1:0-1000 /dev/sdb1:0-1000 /dev/sde1:0-150 
>Feb  6 09:41:51 taft-02 clogd[6717]: [ePNtLM47] Cluster log created 
>Feb  6 09:41:51 taft-02 clogd[6717]: [ePNtLM47] Non-master resume: bits pre-loaded 
>Feb  6 09:41:51 taft-02 [6805]: Monitoring mirror device helter_skelter-syncd_primary_3legs_1 for events 
>Feb  6 09:41:51 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10485 duration=3(sec)
>Feb  6 09:41:51 taft-02 xinetd[6246]: START: qarsh pid=10574 from=10.15.80.47
>Feb  6 09:41:51 taft-02 qarshd[10574]: Talking to peer 10.15.80.47:51111
>Feb  6 09:41:51 taft-02 qarshd[10574]: Running cmdline: lvs -o copy_percent --noheadings helter_skelter/syncd_primary_3legs_1 
>Feb  6 09:41:51 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10574 duration=0(sec)
>Feb  6 09:42:01 taft-02 xinetd[6246]: START: qarsh pid=10597 from=10.15.80.47
>Feb  6 09:42:01 taft-02 qarshd[10597]: Talking to peer 10.15.80.47:41778
>Feb  6 09:42:01 taft-02 qarshd[10597]: Running cmdline: lvs -o copy_percent --noheadings helter_skelter/syncd_primary_3legs_1 
>Feb  6 09:42:02 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10597 duration=1(sec)
>Feb  6 09:42:12 taft-02 xinetd[6246]: START: qarsh pid=10600 from=10.15.80.47
>Feb  6 09:42:12 taft-02 qarshd[10600]: Talking to peer 10.15.80.47:41779
>Feb  6 09:42:12 taft-02 qarshd[10600]: Running cmdline: lvs -o copy_percent --noheadings helter_skelter/syncd_primary_3legs_1 
>Feb  6 09:42:12 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10600 duration=0(sec)
>Feb  6 09:42:22 taft-02 xinetd[6246]: START: qarsh pid=10604 from=10.15.80.47
>Feb  6 09:42:22 taft-02 qarshd[10604]: Talking to peer 10.15.80.47:41781
>Feb  6 09:42:22 taft-02 qarshd[10604]: Running cmdline: lvs -o copy_percent --noheadings helter_skelter/syncd_primary_3legs_1 
>Feb  6 09:42:22 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10604 duration=0(sec)
>Feb  6 09:42:28 taft-02 lvm[6805]: helter_skelter-syncd_primary_3legs_1 is now in-sync 
>Feb  6 09:42:32 taft-02 xinetd[6246]: START: qarsh pid=10607 from=10.15.80.47
>Feb  6 09:42:32 taft-02 qarshd[10607]: Talking to peer 10.15.80.47:41782
>Feb  6 09:42:32 taft-02 qarshd[10607]: Running cmdline: lvs -o copy_percent --noheadings helter_skelter/syncd_primary_3legs_1 
>Feb  6 09:42:32 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10607 duration=0(sec)
>Feb  6 09:42:42 taft-02 xinetd[6246]: START: qarsh pid=10611 from=10.15.80.47
>Feb  6 09:42:42 taft-02 qarshd[10611]: Talking to peer 10.15.80.47:41783
>Feb  6 09:42:42 taft-02 qarshd[10611]: Running cmdline: gfs_mkfs -J 32M -j 4 -p lock_dlm -t TAFT:gfs1 /dev/helter_skelter/syncd_primary_3legs_1 -O 
>Feb  6 09:42:45 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10611 duration=3(sec)
>Feb  6 09:42:45 taft-02 xinetd[6246]: START: qarsh pid=10613 from=10.15.80.47
>Feb  6 09:42:45 taft-02 qarshd[10613]: Talking to peer 10.15.80.47:41786
>Feb  6 09:42:45 taft-02 qarshd[10613]: Running cmdline: mkdir -p /mnt/syncd_primary_3legs_1 
>Feb  6 09:42:45 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10613 duration=0(sec)
>Feb  6 09:42:45 taft-02 xinetd[6246]: START: qarsh pid=10615 from=10.15.80.47
>Feb  6 09:42:45 taft-02 qarshd[10615]: Talking to peer 10.15.80.47:41787
>Feb  6 09:42:45 taft-02 qarshd[10615]: Running cmdline: mount /dev/helter_skelter/syncd_primary_3legs_1 /mnt/syncd_primary_3legs_1 
>Feb  6 09:42:45 taft-02 kernel: Trying to join cluster "lock_dlm", "TAFT:gfs1"
>Feb  6 09:42:45 taft-02 kernel: Joined cluster. Now mounting FS...
>Feb  6 09:42:45 taft-02 kernel: GFS: fsid=TAFT:gfs1.1: jid=1: Trying to acquire journal lock...
>Feb  6 09:42:45 taft-02 kernel: GFS: fsid=TAFT:gfs1.1: jid=1: Looking at journal...
>Feb  6 09:42:45 taft-02 kernel: GFS: fsid=TAFT:gfs1.1: jid=1: Done
>Feb  6 09:42:45 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10615 duration=0(sec)
>Feb  6 09:42:47 taft-02 xinetd[6246]: START: qarsh pid=10649 from=10.15.80.47
>Feb  6 09:42:47 taft-02 qarshd[10649]: Talking to peer 10.15.80.47:41794
>Feb  6 09:42:47 taft-02 qarshd[10649]: Running cmdline: mkdir -p /mnt/syncd_primary_3legs_1/checkit 
>Feb  6 09:42:47 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10649 duration=0(sec)
>Feb  6 09:42:47 taft-02 xinetd[6246]: START: qarsh pid=10651 from=10.15.80.47
>Feb  6 09:42:47 taft-02 qarshd[10651]: Talking to peer 10.15.80.47:41795
>Feb  6 09:42:47 taft-02 qarshd[10651]: Running cmdline: /usr/tests/sts-rhel5.2/bin/checkit -w /mnt/syncd_primary_3legs_1/checkit -f /tmp/checkit_syncd_primary_3legs_1 -n 100 
>Feb  6 09:42:48 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10651 duration=1(sec)
>Feb  6 09:42:51 taft-02 xinetd[6246]: START: qarsh pid=10653 from=10.15.80.47
>Feb  6 09:42:51 taft-02 qarshd[10653]: Talking to peer 10.15.80.47:41802
>Feb  6 09:42:51 taft-02 qarshd[10653]: Running cmdline: /usr/tests/sts-rhel5.2/bin/xiogen -f buffered -m random -s write,writev -t 100 -T 1000000  -F 1000000:/mnt/syncd_primary_3legs_1/taft-02_load | /usr/tests/sts-rhel5.2/bin/xdoio -vD 
>Feb  6 09:43:07 taft-02 xinetd[6246]: START: qarsh pid=10658 from=10.15.80.47
>Feb  6 09:43:07 taft-02 qarshd[10658]: Talking to peer 10.15.80.47:41805
>Feb  6 09:43:07 taft-02 qarshd[10658]: Running cmdline: echo offline > /sys/block/sdg/device/state 
>Feb  6 09:43:07 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10658 duration=0(sec)
>Feb  6 09:43:09 taft-02 kernel: sd 1:0:0:6: rejecting I/O to offline device
>Feb  6 09:43:09 taft-02 lvm[6805]: Mirror device, 253:3, has failed. 
>Feb  6 09:43:09 taft-02 lvm[6805]: Device failure in helter_skelter-syncd_primary_3legs_1 
>Feb  6 09:43:09 taft-02 lvm[6805]: WARNING: dev_open(/etc/lvm/lvm.conf) called while suspended
>Feb  6 09:43:10 taft-02 kernel: sd 1:0:0:6: rejecting I/O to offline device
>Feb  6 09:43:10 taft-02 clogd[6717]: [ePNtLM47] Inc reference count on cluster log 
>Feb  6 09:43:10 taft-02 clogd[6717]: [ePNtLM47] Cluster log created 
>Feb  6 09:43:10 taft-02 clogd[6717]: [ePNtLM47] clog_postsuspend: leaving CPG 
>Feb  6 09:43:10 taft-02 clogd[6717]: [ePNtLM47] clog_postsuspend: finalizing 
>Feb  6 09:43:10 taft-02 clogd[6717]: [ePNtLM47] Dec reference count on cluster log 
>Feb  6 09:43:10 taft-02 clogd[6717]: [ePNtLM47] Non-master resume: bits pre-loaded 
>Feb  6 09:43:10 taft-02 lvm[6805]: Monitoring mirror device helter_skelter-syncd_primary_3legs_1 for events 
>Feb  6 09:43:10 taft-02 lvm[6805]: Another thread is handling an event.  Waiting...
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ramdisk) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/cdrom) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sda) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sda) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/md0) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/root) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sda1) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/VolGroup00/LogVol01) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram2) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sda2) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/mapper/helter_skelter-syncd_primary_3legs_1_mlog) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram3) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram4) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/mapper/helter_skelter-syncd_primary_3legs_1_mimage_1) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram5) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/mapper/helter_skelter-syncd_primary_3legs_1_mimage_2) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram6) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram7) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram8) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram9) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram10) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram11) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram12) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram13) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram14) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/ram15) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdg1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:43:11 taft-02 last message repeated 2 times
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdh1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdb1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdc1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdd1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sde1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/dev/sdf1) called while suspended
>Feb  6 09:43:11 taft-02 lvm[6805]: WARNING: dev_open(/etc/lvm/lvm.conf) called while suspended
>Feb  6 09:43:12 taft-02 xinetd[6246]: START: qarsh pid=10708 from=10.15.80.47
>Feb  6 09:43:12 taft-02 qarshd[10708]: Talking to peer 10.15.80.47:41808
>Feb  6 09:43:12 taft-02 qarshd[10708]: Running cmdline: dd if=/dev/zero of=/mnt/syncd_primary_3legs_1/ddfile count=10 bs=4M 
>Feb  6 09:43:12 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10708 duration=0(sec)
>Feb  6 09:43:12 taft-02 xinetd[6246]: START: qarsh pid=10710 from=10.15.80.47
>Feb  6 09:43:12 taft-02 qarshd[10710]: Talking to peer 10.15.80.47:41809
>Feb  6 09:43:12 taft-02 qarshd[10710]: Running cmdline: sync 
>Feb  6 09:43:13 taft-02 lvm[6805]: No longer monitoring mirror device helter_skelter-syncd_primary_3legs_1 for events 
>Feb  6 09:44:23 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10710 duration=71(sec)
>Feb  6 09:44:29 taft-02 xinetd[6246]: START: qarsh pid=10719 from=10.15.80.47
>Feb  6 09:44:29 taft-02 qarshd[10719]: Talking to peer 10.15.80.47:41812
>Feb  6 09:44:29 taft-02 qarshd[10719]: Running cmdline: lvs 
>Feb  6 09:44:30 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10719 duration=1(sec)
>Feb  6 09:44:30 taft-02 xinetd[6246]: START: qarsh pid=10722 from=10.15.80.47
>Feb  6 09:44:30 taft-02 qarshd[10722]: Talking to peer 10.15.80.47:41813
>Feb  6 09:44:30 taft-02 qarshd[10722]: Running cmdline: lvs -a -o +devices 
>Feb  6 09:44:30 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10722 duration=0(sec)
>Feb  6 09:44:30 taft-02 xinetd[6246]: START: qarsh pid=10725 from=10.15.80.47
>Feb  6 09:44:30 taft-02 qarshd[10725]: Talking to peer 10.15.80.47:41814
>Feb  6 09:44:30 taft-02 qarshd[10725]: Running cmdline: lvs -a -o +devices 
>Feb  6 09:44:30 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10725 duration=0(sec)
>Feb  6 09:44:30 taft-02 xinetd[6246]: START: qarsh pid=10728 from=10.15.80.47
>Feb  6 09:44:30 taft-02 qarshd[10728]: Talking to peer 10.15.80.47:41815
>Feb  6 09:44:30 taft-02 qarshd[10728]: Running cmdline: lvs -a -o +devices 
>Feb  6 09:44:31 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10728 duration=1(sec)
>Feb  6 09:44:31 taft-02 xinetd[6246]: START: qarsh pid=10731 from=10.15.80.47
>Feb  6 09:44:31 taft-02 qarshd[10731]: Talking to peer 10.15.80.47:41816
>Feb  6 09:44:31 taft-02 qarshd[10731]: Running cmdline: lvs -a -o +devices 
>Feb  6 09:44:31 taft-02 xinetd[6246]: EXIT: qarsh status=0 pid=10731 duration=0(sec)

Actions: View

Attachments on bug 431696: 294120 | 294121 | 294123 | 294124