Bug 206089

Summary: attempting to change the minor number of a cmirror causes havoc
Product: [Retired] Red Hat Cluster Suite Reporter: Corey Marthaler <cmarthal>
Component: cmirrorAssignee: Jonathan Earl Brassow <jbrassow>
Status: CLOSED CURRENTRELEASE QA Contact: Cluster QE <mspqa-list>
Severity: medium Docs Contact:
Priority: medium    
Version: 4CC: agk, cfeist, dwysocha, mbroz
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-08-05 21:34:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Corey Marthaler 2006-09-11 22:25:10 UTC
Description of problem:
It appears that after attempting to change the minor number of a cluster mirror,
I ended up with an additional mirror server (two total) as well as additional
mirror legs.


[root@taft-04 tmp]# lvscan
  ACTIVE            '/dev/vg/mirror' [100.00 GB] inherit

[root@taft-04 tmp]# dmsetup ls
vg-mirror_mimage_1      (253, 4)
vg-mirror_mimage_0      (253, 3)
vg-mirror       (253, 5)
VolGroup00-LogVol01     (253, 1)
VolGroup00-LogVol00     (253, 0)
vg-mirror_mlog  (253, 2)

[root@taft-04 tmp]# lvchange -My --major 200 --minor 100 /dev/vg/mirror
Logical volume mirror will be deactivated temporarily. Continue? [y/n]: y
  Error locking on node taft-04: Internal lvm error, check syslog
  Error locking on node taft-02: Internal lvm error, check syslog
  Error locking on node taft-03: Internal lvm error, check syslog
  Error locking on node taft-01: Internal lvm error, check syslog
  Failed to lock mirror

Appears to be two mirror servers now, taft-01 and taft-04...

Sep 11 10:50:26 taft-01 kernel: dm-cmirror: I'm the cluster mirror log server
for I6FTGdqG
Sep 11 10:50:26 taft-01 kernel: dm-cmirror: Disk Resume::  I6FTGdqG (active)
Sep 11 10:50:26 taft-01 dmeventd[7002]: Monitoring mirror device, vg-mirror for
events
Sep 11 10:50:26 taft-01 kernel: dm-cmirror:   Live nodes        :: 1
Sep 11 10:50:26 taft-01 kernel: dm-cmirror:   In-Use Regions    :: 0
Sep 11 10:50:26 taft-01 kernel: dm-cmirror:   Good IUR's        :: 0
Sep 11 10:50:26 taft-01 kernel: dm-cmirror:   Bad IUR's         :: 0
Sep 11 10:50:26 taft-01 kernel: dm-cmirror:   Sync count        :: 100783
Sep 11 10:50:26 taft-01 kernel: dm-cmirror:   Disk Region count :: 204800
Sep 11 10:50:26 taft-01 kernel: dm-cmirror:   Region count      :: 204800
Sep 11 10:50:26 taft-01 kernel: dm-cmirror: Marked regions::
Sep 11 10:50:26 taft-01 kernel: dm-cmirror:   16816 - 24579
Sep 11 10:50:26 taft-01 kernel: dm-cmirror:   24582 - 24583
Sep 11 10:50:26 taft-01 kernel: dm-cmirror:   24585 - 24587
[...]
Sep 11 10:50:32 taft-01 kernel: dm-cmirror:   204780
Sep 11 10:50:32 taft-01 kernel: dm-cmirror:   204782 - 204787
Sep 11 10:50:32 taft-01 kernel: dm-cmirror:   204790 - 204791
Sep 11 10:50:32 taft-01 kernel: dm-cmirror:   204793 - 204795
Sep 11 10:50:32 taft-01 kernel: dm-cmirror:   204798 - 204800
Sep 11 10:50:32 taft-01 kernel: dm-cmirror:   Total = 104017
Sep 11 10:50:46 taft-01 kernel: dm-cmirror: Log already exists with uuid, I6FTGdqG
Sep 11 10:50:46 taft-01 kernel: device-mapper: dm-mirror: Error creating mirror
dirty log
Sep 11 10:50:46 taft-01 kernel: device-mapper: error adding target to table
Sep 11 10:50:46 taft-01 lvm[6847]: device-mapper: reload ioctl failed: Invalid
argument




Sep 11 11:58:49 taft-04 kernel: dm-cmirror: I'm the cluster mirror log server
for I6FTGdqG
Sep 11 11:58:49 taft-04 kernel: dm-cmirror: Disk Resume::  I6FTGdqG (active)
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   Live nodes        :: 3
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   In-Use Regions    :: 0
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   Good IUR's        :: 0
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   Bad IUR's         :: 0
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   Sync count        :: 100783
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   Disk Region count :: 204800
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   Region count      :: 204800
Sep 11 11:58:49 taft-04 kernel: dm-cmirror: Marked regions::
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   16816 - 24579
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   24582 - 24583
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   24585 - 24587
Sep 11 11:58:49 taft-04 kernel: dm-cmirror:   24590 - 24592
[...]
Sep 11 11:58:54 taft-04 kernel: dm-cmirror:   204782 - 204787
Sep 11 11:58:54 taft-04 kernel: dm-cmirror:   204790 - 204791
Sep 11 11:58:54 taft-04 kernel: dm-cmirror:   204793 - 204795
Sep 11 11:58:54 taft-04 kernel: dm-cmirror:   204798 - 204800
Sep 11 11:58:54 taft-04 kernel: dm-cmirror:   Total = 104017
Sep 11 11:59:24 taft-04 dmeventd[6853]: Monitoring mirror device, vg-mirror for
events
Sep 11 11:59:24 taft-04 kernel: dm-cmirror: Log already exists with uuid, I6FTGdqG
Sep 11 11:59:24 taft-04 kernel: device-mapper: dm-mirror: Error creating mirror
dirty log
Sep 11 11:59:24 taft-04 kernel: device-mapper: error adding target to table
Sep 11 11:59:24 taft-04 lvm[6671]: device-mapper: reload ioctl failed: Invalid
argument
                                                                               
                               

[root@taft-01 tmp]# lvs -a -o +devices
  LV                VG   Attr   LSize   Origin Snap%  Move Log         Copy% 
Devices
  mirror            vg   mwi-a- 100.00G                    mirror_mlog  56.99
mirror_mimage_0(0),mirror_mimage_1(0)
  [mirror_mimage_0] vg   iwi-ao 100.00G                                      
/dev/sdb1(0)
  [mirror_mimage_0] vg   iwi-ao 100.00G                                      
/dev/sdb3(0)
  [mirror_mimage_1] vg   iwi-ao 100.00G                                      
/dev/sdb2(0)
  [mirror_mimage_1] vg   iwi-ao 100.00G                                      
/dev/sdc1(0)
  [mirror_mlog]     vg   lwi-ao   4.00M                                      
/dev/sdc3(0)


Version-Release number of selected component (if applicable):
[root@taft-01 tmp]# uname -ar
Linux taft-01 2.6.9-42.0.2.ELsmp #1 SMP Thu Aug 17 17:57:31 EDT 2006 x86_64
x86_64 x86_64 GNU/Linux
[root@taft-01 tmp]# rpm -q lvm2
lvm2-2.02.06-6.0.RHEL4
[root@taft-01 tmp]# rpm -q lvm2-cluster
lvm2-cluster-2.02.06-7.0.RHEL4
[root@taft-01 tmp]# rpm -q device-mapper
device-mapper-1.02.07-4.0.RHEL4
[root@taft-01 tmp]# rpm -q cmirror
cmirror-1.0.1-0
[root@taft-01 tmp]# rpm -q cmirror-kernel
cmirror-kernel-2.6.9-13.0

Comment 1 Corey Marthaler 2006-09-11 22:26:25 UTC
This is reproducable.

Comment 2 Jonathan Earl Brassow 2006-09-12 18:16:35 UTC
This is the exact same problem as rename - only the cmirror package is refussing
to be screwed up by allowing multiple copies of the same table to be loaded.

There is a work-around to this pending for user-space, as well as a kernel patch
to fix the problem.



Comment 3 Jonathan Earl Brassow 2006-09-19 13:19:30 UTC
Fixed, but waiting for checkin/new packages.

Comment 4 Kiersten (Kerri) Anderson 2006-09-22 19:46:14 UTC
Devel ACK

Comment 5 Jonathan Earl Brassow 2006-10-17 14:51:06 UTC
let's put this into POST, since the fixes are now in the packages (I've
checked), but I'm not sure the package has been built.

Comment 6 Corey Marthaler 2006-11-01 16:39:31 UTC
fix verified in the latest builds:

[root@link-08 lvm]# uname -ar
Linux link-08 2.6.9-42.17.ELsmp #1 SMP Mon Oct 9 18:42:57 EDT 2006 x86_64 x86_64
x86_64 GNU/Linux
[root@link-08 lvm]# rpm -q lvm2
lvm2-2.02.13-1
[root@link-08 lvm]# rpm -q lvm2-cluster
lvm2-cluster-2.02.13-1
[root@link-08 lvm]# rpm -q device-mapper
device-mapper-1.02.12-3
[root@link-08 lvm]# rpm -q cmirror-kernel
cmirror-kernel-2.6.9-13.0


Comment 7 Chris Feist 2008-08-05 21:34:56 UTC
Fixed in latest release.