Bug 192296 - cluster mirroring doesn't appear to work
cluster mirroring doesn't appear to work
Status: CLOSED CURRENTRELEASE
Product: Red Hat Cluster Suite
Classification: Red Hat
Component: cmirror (Show other bugs)
4
All Linux
high Severity high
: ---
: ---
Assigned To: Jonathan Earl Brassow
Cluster QE
:
Depends On:
Blocks: 180185
  Show dependency treegraph
 
Reported: 2006-05-18 16:19 EDT by Corey Marthaler
Modified: 2010-04-27 10:51 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-04-27 10:51:05 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Corey Marthaler 2006-05-18 16:19:52 EDT
Description of problem:
To the best of my knowledge I have everything in place on all the nodes in the
cluster, and this is supposed to be the new build with mirrors in it. I have the
cmirror-kernel rpm installed, the dm-mirror module loaded, and the dm-cmirror
module loaded. And if there is still something that I'm missing, it goes back to
bz 182432, where I should be getting a warning when the user is missing something.

[root@taft-02 ~]# lvcreate -m 1 -n taft-mirror -L 100G taft
  Error locking on node taft-04: Internal lvm error, check syslog
  Error locking on node taft-03: Internal lvm error, check syslog
  Error locking on node taft-01: Internal lvm error, check syslog
  Error locking on node taft-02: Internal lvm error, check syslog
  Failed to activate new LV.

dm_cmirror: no version for "kcl_register_service" found: kernel tainted.
device-mapper: Too few arguments to clustered_disk mirror log type.
device-mapper:   3 arguments supplied:
device-mapper:     253:2
device-mapper:     1024
device-mapper:     LVM-XIfZooktPY9Brrgf0pvK5A1IVRq2wn1pubKnhMlxLkwXDHPi3fDHqcBMt
device-mapper: dm-mirror: Error creating mirror dirty log
device-mapper: error adding target to table


This then leads to my disks becoming unusable until go though dm-setup to delete
the mirror logs by hand since they don't show up in lvm.

[root@taft-02 ~]# pvremove -ff /dev//sdb[1-2] /dev/sdc[1-2]
Really WIPE LABELS from physical volume "/dev//sdb1" of volume group "taft" [y/n]? y
  WARNING: Wiping physical volume label from /dev//sdb1 of volume group "taft"
  Can't open /dev/sdb1 exclusively.  Mounted filesystem?
Really WIPE LABELS from physical volume "/dev//sdb2" of volume group "taft" [y/n]? y
  WARNING: Wiping physical volume label from /dev//sdb2 of volume group "taft"
  Can't open /dev/sdb2 exclusively.  Mounted filesystem?
Really WIPE LABELS from physical volume "/dev/sdc1" of volume group "taft" [y/n]? y
  WARNING: Wiping physical volume label from /dev/sdc1 of volume group "taft"
  Labels on physical volume "/dev/sdc1" successfully wiped
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  get_pv_from_vg_by_id: vg_read failed to read VG taft
  Can't open /dev/sdc2 exclusively.  Mounted filesystem?

[root@taft-02 ~]# dmsetup table
taft-taft--mirror_mlog: 0 8192 linear 8:34 384
taft-taft--mirror_mimage_1: 0 209715200 linear 8:18 384
taft-taft--mirror_mimage_0: 0 209715200 linear 8:17 384
VolGroup00-LogVol01: 0 4063232 linear 8:2 40960384
VolGroup00-LogVol00: 0 40960000 linear 8:2 384
taft-taft--mirror:
[root@taft-02 ~]# dmsetup remove taft-taft--mirror_mlog
[root@taft-02 ~]# dmsetup remove taft-taft--mirror_mimage_1
[root@taft-02 ~]# dmsetup remove taft-taft--mirror_mimage_0
[root@taft-02 ~]# pvscan
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  PV /dev/sdb1        VG taft   lvm2 [136.23 GB / 36.23 GB free]
  PV /dev/sdb2        VG taft   lvm2 [136.23 GB / 36.23 GB free]
  PV unknown device   VG taft   lvm2 [136.23 GB / 136.23 GB free]
  PV /dev/sdc2        VG taft   lvm2 [136.23 GB / 136.23 GB free]
  Total: 4 [544.92 GB] / in use: 4 [544.92 GB] / in no VG: 0 [0   ]
[root@taft-02 ~]# pvremove -ff /dev//sdb[1-2] /dev/sdc[1-2]
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid 'wgsbMr-epSC-1NO7-iic9-Plcw-LvKZ-RpA8zN'.
  Couldn't find all physical volumes for volume group taft.
  get_pv_from_vg_by_id: vg_read failed to read VG taft
  Labels on physical volume "/dev//sdb1" successfully wiped
  Couldn't find device with uuid '3YrUBq-7Xg8-rH3y-bgfi-Jos1-k2fT-UBZnI8'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid '3YrUBq-7Xg8-rH3y-bgfi-Jos1-k2fT-UBZnI8'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid '3YrUBq-7Xg8-rH3y-bgfi-Jos1-k2fT-UBZnI8'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid '3YrUBq-7Xg8-rH3y-bgfi-Jos1-k2fT-UBZnI8'.
  Couldn't find all physical volumes for volume group taft.
  get_pv_from_vg_by_id: vg_read failed to read VG taft
  Labels on physical volume "/dev//sdb2" successfully wiped
  No physical volume label read from /dev/sdc1
  Labels on physical volume "/dev/sdc1" successfully wiped
  Couldn't find device with uuid '3YrUBq-7Xg8-rH3y-bgfi-Jos1-k2fT-UBZnI8'.
  Couldn't find all physical volumes for volume group taft.
  Couldn't find device with uuid '3YrUBq-7Xg8-rH3y-bgfi-Jos1-k2fT-UBZnI8'.
  Couldn't find all physical volumes for volume group taft.
  get_pv_from_vg_by_id: vg_read failed to read VG taft
  Labels on physical volume "/dev/sdc2" successfully wiped
[root@taft-02 ~]# pvscan
  No matching physical volumes found


Version-Release number of selected component (if applicable):
[root@taft-01 ~]# rpm -q device-mapper
device-mapper-1.02.07-1.0.RHEL4
[root@taft-01 ~]# rpm -q lvm2
lvm2-2.02.06-1.0.RHEL4
[root@taft-01 ~]# rpm -q lvm2-cluster
lvm2-cluster-2.02.01-1.2.RHEL4
[root@taft-01 ~]# rpm -q cmirror-kernel-smp
cmirror-kernel-smp-2.6.9-4.2
[root@taft-01 ~]# uname -ar
Linux taft-01 2.6.9-36.ELsmp #1 SMP Tue May 9 09:15:01 EDT 2006 x86_64 x86_64
x86_64 GNU/Linux


How reproducible:
everytime
Comment 1 Jonathan Earl Brassow 2006-05-19 01:29:33 EDT
Strange.  It's probably the rpms you are using...  Device mapper has failed to
supply the "block_on_error" argument, which cluster mirror requires.  I'll try
checking out those RPMs...  Wait a minute...  The lvm2-cluster seems to be from
March 7th, 2006.  It is probably not setting the flag correctly to use
"block_on_error".  Might have to wait until that package is updated.

[root@tng1-1 ~]# lvcreate -m1 -n lv -L 500M vg
  Logical volume "lv" created
[root@tng1-1 ~]# lvs
  LV       VG         Attr   LSize   Origin Snap%  Move Log     Copy%
  LogVol00 VolGroup00 -wi-ao  13.19G
  LogVol01 VolGroup00 -wi-ao   1.00G
  lv       vg         mwi-a- 500.00M                    lv_mlog  32.80
[root@tng1-1 ~]# vgs
  VG         #PV #LV #SN Attr   VSize  VFree
  VolGroup00   1   2   0 wz--n- 14.19G     0
  vg           4   4   0 wz--nc 68.59G 67.61G
Comment 3 Corey Marthaler 2006-07-07 11:35:19 EDT
The locking errors described in this bug are documented in bz 194125. Closing...
Comment 5 Alasdair Kergon 2010-04-27 10:51:05 EDT
Assuming this VERIFIED fix got released.  Closing.
Reopen if it's not yet resolved.

Note You need to log in before you can comment on or make changes to this bug.