Bug 712247

Summary: device trees are being left behind after deactivation has occured
Product: Red Hat Enterprise Linux 5 Reporter: Corey Marthaler <cmarthal>
Component: lvm2Assignee: Peter Rajnoha <prajnoha>
Status: CLOSED CURRENTRELEASE QA Contact: Corey Marthaler <cmarthal>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.7CC: agk, dwysocha, heinzm, jbrassow, prajnoha, prockai, thornber, zkabelac
Target Milestone: rcKeywords: TestBlocker
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-08-09 19:50:11 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 807971    

Description Corey Marthaler 2011-06-09 22:33:57 UTC 
Description of problem:
This has been causing ReadOnly write checks to fail because zombie device entries exist and hence writes pass when they shouldn't.

I'm not exactly sure what's causing this, could be the exclusive locking, exporting, importing, broken activation, ... I'll have to boil this down better.

Test Output:

./lvmconfig
[...]
grabing the exclusive lock on grant-03
attempting to also grab an exclusive lock on grant-01
volume activation should not have worked on grant-01,
due to the current exclusive lock by grant-03
attempting to grab a non exclusive lock on grant-01
volume activation should not have worked on grant-01,
due to the current exclusive lock by grant-03
attempting to also grab an exclusive lock on grant-02
volume activation should not have worked on grant-02,
due to the current exclusive lock by grant-03
attempting to grab a non exclusive lock on grant-02
volume activation should not have worked on grant-02,
due to the current exclusive lock by grant-03
releasing the exclusive lock on grant-03
deactivating VG linear_3_8938 on grant-03


deactivating VG linear_3_8938 on grant-02
EXPORTING VOLUME GROUP on grant-02
attempting to activate on grant-03
  Volume group "linear_3_8938" is exported
IMPORTING VOLUME GROUP on grant-01

CHANGING LV WRITE ACCESS TO RO on grant-03
verifying ro access flag on on grant-03
attempting to write to lv on grant-01
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 6.8e-05 seconds, 75.3 MB/s
should not have been able to write to /dev/linear_3_8938/linear_3_89380 on grant-01



# GRANT-01
[root@grant-01 ~]# lvs -a -o +devices
  LV                        Attr   LSize   Log                 Devices
  LogVol00                  -wi-ao  64.56G                     /dev/sda2(0)
  LogVol01                  -wi-ao   9.81G                     /dev/sda2(2066)
  linear_3_89380            mrim-- 708.00M linear_3_89380_mlog linear_3_89380_mimage_0(0),linear_3_89380_mimage_1(0)
  [linear_3_89380_mimage_0] Iwi--- 708.00M                     /dev/sdc1(0)
  [linear_3_89380_mimage_1] Iwi--- 708.00M                     /dev/sdc2(0)
  [linear_3_89380_mlog]     lwi---   4.00M                     /dev/sdb6(0)
[root@grant-01 ~]# dmsetup ls
VolGroup00-LogVol01     (253, 1)
VolGroup00-LogVol00     (253, 0)
# THERE SHOULD NOT BE AN ENTRY FOR THIS DEVICE SINCE IT'S INACTIVE!!
[root@grant-01 ~]# ls /dev/linear_3_8938/linear_3_89380
/dev/linear_3_8938/linear_3_89380


# GRANT-02
[root@grant-02 ~]# lvs -a -o +devices
  LV                        Attr   LSize   Log                 Devices
  LogVol00                  -wi-ao  64.56G                     /dev/sda2(0)
  LogVol01                  -wi-ao   9.81G                     /dev/sda2(2066)
  linear_3_89380            mrim-- 708.00M linear_3_89380_mlog linear_3_89380_mimage_0(0),linear_3_89380_mimage_1(0)
  [linear_3_89380_mimage_0] Iwi--- 708.00M                     /dev/sdc1(0)
  [linear_3_89380_mimage_1] Iwi--- 708.00M                     /dev/sdc2(0)
  [linear_3_89380_mlog]     lwi---   4.00M                     /dev/sdb6(0)
[root@grant-02 ~]# dmsetup ls
VolGroup00-LogVol01     (253, 1)
VolGroup00-LogVol00     (253, 0)
[root@grant-02 ~]# ls /dev/linear_3_8938/linear_3_89380
ls: /dev/linear_3_8938/linear_3_89380: No such file or directory


# GRANT-03
[root@grant-03 ~]# lvs -a -o +devices
  LV                        Attr   LSize   Log                 Devices
  LogVol00                  -wi-ao  64.56G                     /dev/sda2(0)
  LogVol01                  -wi-ao   9.81G                     /dev/sda2(2066)
  linear_3_89380            mrim-- 708.00M linear_3_89380_mlog linear_3_89380_mimage_0(0),linear_3_89380_mimage_1(0)
  [linear_3_89380_mimage_0] Iwi--- 708.00M                     /dev/sdc1(0)
  [linear_3_89380_mimage_1] Iwi--- 708.00M                     /dev/sdc2(0)
  [linear_3_89380_mlog]     lwi---   4.00M                     /dev/sdb6(0)
[root@grant-03 ~]# dmsetup ls
VolGroup00-LogVol01     (253, 1)
VolGroup00-LogVol00     (253, 0)
[root@grant-03 ~]# ls /dev/linear_3_8938/linear_3_89380
ls: /dev/linear_3_8938/linear_3_89380: No such file or directory



Version-Release number of selected component (if applicable):
2.6.18-265.el5

lvm2-2.02.84-4.el5    BUILT: Thu Jun  2 05:04:32 CDT 2011
lvm2-cluster-2.02.84-3.el5    BUILT: Wed Apr 27 03:42:43 CDT 2011
device-mapper-1.02.63-3.el5    BUILT: Thu May 19 08:09:22 CDT 2011
device-mapper-event-1.02.63-3.el5    BUILT: Thu May 19 08:09:22 CDT 2011
cmirror-1.1.39-10.el5    BUILT: Wed Sep  8 16:32:05 CDT 2010
kmod-cmirror-0.1.22-3.el5    BUILT: Tue Dec 22 13:39:47 CST 2009


How reproducible:
Often
Comment 1 Zdenek Kabelac 2011-08-29 08:47:18 UTC 
Moving this bug to Petr, are you able to see similar issues?
Comment 2 Peter Rajnoha 2011-08-30 11:20:55 UTC 
(In reply to comment #1)
> Moving this bug to Petr, are you able to see similar issues?

No, but this is evidently a race of some kind again, so it may be hard to reproduce in different environment... But as Corey says, we need to narrow this down.

Corey, would it be possible to get the dmsetup/lvs output just after the last command that comes before trying to write to a device (that should not be there anymore)? And if the device is still active at that moment, for starters, maybe a debug log of that last "deactivate" command...
Comment 6 Corey Marthaler 2012-08-09 19:50:11 UTC 
No longer able to reproduce this issue with the latest 5.9 tree. Closing...

2.6.18-333.el5

lvm2-2.02.88-9.el5    BUILT: Wed Jul 25 10:13:00 CDT 2012
lvm2-cluster-2.02.88-9.el5    BUILT: Wed Jul 25 10:09:34 CDT 2012
device-mapper-1.02.67-2.el5    BUILT: Mon Oct 17 08:31:56 CDT 2011
device-mapper-event-1.02.67-2.el5    BUILT: Mon Oct 17 08:31:56 CDT 2011
cmirror-1.1.39-15.el5    BUILT: Thu Apr 26 17:07:01 CDT 2012
kmod-cmirror-0.1.22-3.el5    BUILT: Tue Dec 22 13:39:47 CST 2009