Bug 805300 - device-mapper: remove ioctl on failed; error when taking snapshot of inactive origin
device-mapper: remove ioctl on failed; error when taking snapshot of inactiv...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: lvm2 (Show other bugs)
6.3
x86_64 Linux
low Severity low
: rc
: ---
Assigned To: Peter Rajnoha
Cluster QE
: Regression
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-20 16:15 EDT by Corey Marthaler
Modified: 2012-06-20 11:02 EDT (History)
10 users (show)

See Also:
Fixed In Version: lvm2-2.02.95-3.el6
Doc Type: Bug Fix
Doc Text:
No Documentation needed.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-06-20 11:02:54 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Corey Marthaler 2012-03-20 16:15:37 EDT
Description of problem:
SCENARIO - [create_snap_of_inactive_origin]
Create a snapshot of an inactive origin
Making origin volume
lvcreate -L 300M snapper -n origin
Deactivating origin/snap volume(s)
lvchange -an snapper/origin


[root@grant-01 ~]# pvscan
  PV /dev/sdc1   VG snapper      lvm2 [272.46 GiB / 272.17 GiB free]
  PV /dev/sdb1   VG snapper      lvm2 [204.34 GiB / 204.34 GiB free]

[root@grant-01 ~]# vgs
  VG         #PV #LV #SN Attr   VSize   VFree  
  snapper      2   1   0 wz--n- 476.80g 476.51g

[root@grant-01 ~]# lvs -a -o +devices
  LV      VG         Attr     LSize   Origin Devices         
  origin  snapper    -wi----- 300.00m        /dev/sdc1(0)    

[root@grant-01 ~]# lvcreate -s /dev/snapper/origin -c 32 -n snap_of_inactive -L 100M
  device-mapper: remove ioctl on  failed: Device or resource busy
  Logical volume "snap_of_inactive" created

[root@grant-01 ~]# lvs -a -o +devices
  LV               VG       Attr     LSize   Origin Devices         
  origin           snapper  owi---s- 300.00m        /dev/sdc1(0)    
  snap_of_inactive snapper  swi---s- 100.00m origin /dev/sdc1(75)   


Creating snap of inactive origin
[root@grant-01 ~]# lvcreate -s /dev/snapper/origin -c 32 -n snap_of_inactive -L 100M
  device-mapper: remove ioctl on  failed: Device or resource busy
  Logical volume "snap_of_inactive" created


Version-Release number of selected component (if applicable):
2.6.32-251.el6.x86_64
lvm2-2.02.95-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
lvm2-libs-2.02.95-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
lvm2-cluster-2.02.95-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
udev-147-2.40.el6    BUILT: Fri Sep 23 07:51:13 CDT 2011
device-mapper-1.02.74-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
device-mapper-libs-1.02.74-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
device-mapper-event-1.02.74-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
device-mapper-event-libs-1.02.74-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
cmirror-2.02.95-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012


How reproducible:
Every time
Comment 1 Peter Rajnoha 2012-03-21 09:37:59 EDT
Hmm, this might be caused by the watch udev rule we've added to our 13-dm-disk.rules as part of the rebase.

The reason it shows that the ioctl failed and then the operation succeeded despite that is probably the "retry" code that gets activated - we should see that in the debug log, something like:

  #ioctl/libdm-iface.c:1687         dm remove   (253:2) NFT    [16384] (*1)
  #ioctl/libdm-iface.c:1687         dm remove   (253:2) NFT    [16384] (*2)
  ...

Where the (*X) is the retry number. The retry probably won't be used if using the debug log with the command as this will slow down the process a bit making time for udev to finish fiddling with the dev :)

(I'm thinking now whether we should show these messages as errors on retry or just log them in as a debug message only... OR removing the watch rule in RHEL as its gain is not worth the problems it brings).

Corey, please try to remove/comment out the OPTIONS+="watch" line in /lib/udev/rules.d/13-dm-disk.rules and see if you can still hit the problem...
Comment 2 Peter Rajnoha 2012-03-21 09:40:12 EDT
Oh, but we need the watch rule for the lvmetad in case the PV is created on top of a DM device. Hmmm...
Comment 3 Peter Rajnoha 2012-03-21 09:44:13 EDT
(...an alternative to that would be to send the change event ourselves, just by writing "change" to corresponding /sys/block/dm/uevent after we create the PV, so we could probably do without the watch rule)
Comment 4 Corey Marthaler 2012-03-21 15:14:36 EDT
If I comment out the 'OPTIONS+="watch"' line everything works fine.

============================================================
Iteration 50 of 50 started at Wed Mar 21 14:14:42 CDT 2012
============================================================
SCENARIO - [create_snap_of_inactive_origin]
Create a snapshot of an inactive origin
Making origin volume
lvcreate -L 300M snapper -n origin
Deactivating origin/snap volume(s)
lvchange -an snapper/origin
Creating snap of inactive origin
lvcreate -s /dev/snapper/origin -c 32 -n snap_of_inactive -L 100M
Activating origin/snap volume(s)
lvchange -ay snapper/origin 2>&1
Removing volume snapper/snap_of_inactive
Removing origin snapper/origin
Comment 5 Peter Rajnoha 2012-03-22 09:54:03 EDT
Actually, pvcreate updates the lvmetad directly so we don't need the watch rule. I'll remove it in next respin as we had cases before where this caused problems in enterprise environment (this will be a RHEL specific patch).
Comment 6 Peter Rajnoha 2012-03-27 08:49:48 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
No Documentation needed.
Comment 9 Corey Marthaler 2012-04-04 15:46:20 EDT
Fix verified in the latest rpms.

2.6.32-251.el6.x86_64
lvm2-2.02.95-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
lvm2-libs-2.02.95-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
lvm2-cluster-2.02.95-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
udev-147-2.40.el6    BUILT: Fri Sep 23 07:51:13 CDT 2011
device-mapper-1.02.74-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
device-mapper-libs-1.02.74-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
device-mapper-event-1.02.74-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
device-mapper-event-libs-1.02.74-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
cmirror-2.02.95-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012


============================================================
Iteration 10 of 10 started at Wed Apr  4 14:46:52 CDT 2012
============================================================
SCENARIO - [create_snap_of_inactive_origin]
Create a snapshot of an inactive origin
Making origin volume
lvcreate -L 300M snapper -n origin
Deactivating origin/snap volume(s)
Creating snap of inactive origin
lvcreate -s /dev/snapper/origin -c 32 -n snap_of_inactive -L 100M
Activating origin/snap volume(s)
Removing volume snapper/snap_of_inactive
Removing origin snapper/origin
Comment 10 errata-xmlrpc 2012-06-20 11:02:54 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0962.html

Note You need to log in before you can comment on or make changes to this bug.