RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 805300 - device-mapper: remove ioctl on failed; error when taking snapshot of inactive origin
Summary: device-mapper: remove ioctl on failed; error when taking snapshot of inactiv...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: lvm2
Version: 6.3
Hardware: x86_64
OS: Linux
low
low
Target Milestone: rc
: ---
Assignee: Peter Rajnoha
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-03-20 20:15 UTC by Corey Marthaler
Modified: 2012-06-20 15:02 UTC (History)
10 users (show)

Fixed In Version: lvm2-2.02.95-3.el6
Doc Type: Bug Fix
Doc Text:
No Documentation needed.
Clone Of:
Environment:
Last Closed: 2012-06-20 15:02:54 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2012:0962 0 normal SHIPPED_LIVE lvm2 bug fix and enhancement update 2012-06-19 21:12:11 UTC

Description Corey Marthaler 2012-03-20 20:15:37 UTC 
Description of problem:
SCENARIO - [create_snap_of_inactive_origin]
Create a snapshot of an inactive origin
Making origin volume
lvcreate -L 300M snapper -n origin
Deactivating origin/snap volume(s)
lvchange -an snapper/origin


[root@grant-01 ~]# pvscan
  PV /dev/sdc1   VG snapper      lvm2 [272.46 GiB / 272.17 GiB free]
  PV /dev/sdb1   VG snapper      lvm2 [204.34 GiB / 204.34 GiB free]

[root@grant-01 ~]# vgs
  VG         #PV #LV #SN Attr   VSize   VFree  
  snapper      2   1   0 wz--n- 476.80g 476.51g

[root@grant-01 ~]# lvs -a -o +devices
  LV      VG         Attr     LSize   Origin Devices         
  origin  snapper    -wi----- 300.00m        /dev/sdc1(0)    

[root@grant-01 ~]# lvcreate -s /dev/snapper/origin -c 32 -n snap_of_inactive -L 100M
  device-mapper: remove ioctl on  failed: Device or resource busy
  Logical volume "snap_of_inactive" created

[root@grant-01 ~]# lvs -a -o +devices
  LV               VG       Attr     LSize   Origin Devices         
  origin           snapper  owi---s- 300.00m        /dev/sdc1(0)    
  snap_of_inactive snapper  swi---s- 100.00m origin /dev/sdc1(75)   


Creating snap of inactive origin
[root@grant-01 ~]# lvcreate -s /dev/snapper/origin -c 32 -n snap_of_inactive -L 100M
  device-mapper: remove ioctl on  failed: Device or resource busy
  Logical volume "snap_of_inactive" created


Version-Release number of selected component (if applicable):
2.6.32-251.el6.x86_64
lvm2-2.02.95-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
lvm2-libs-2.02.95-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
lvm2-cluster-2.02.95-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
udev-147-2.40.el6    BUILT: Fri Sep 23 07:51:13 CDT 2011
device-mapper-1.02.74-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
device-mapper-libs-1.02.74-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
device-mapper-event-1.02.74-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
device-mapper-event-libs-1.02.74-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012
cmirror-2.02.95-2.el6    BUILT: Fri Mar 16 08:39:54 CDT 2012


How reproducible:
Every time
Comment 1 Peter Rajnoha 2012-03-21 13:37:59 UTC 
Hmm, this might be caused by the watch udev rule we've added to our 13-dm-disk.rules as part of the rebase.

The reason it shows that the ioctl failed and then the operation succeeded despite that is probably the "retry" code that gets activated - we should see that in the debug log, something like:

  #ioctl/libdm-iface.c:1687         dm remove   (253:2) NFT    [16384] (*1)
  #ioctl/libdm-iface.c:1687         dm remove   (253:2) NFT    [16384] (*2)
  ...

Where the (*X) is the retry number. The retry probably won't be used if using the debug log with the command as this will slow down the process a bit making time for udev to finish fiddling with the dev :)

(I'm thinking now whether we should show these messages as errors on retry or just log them in as a debug message only... OR removing the watch rule in RHEL as its gain is not worth the problems it brings).

Corey, please try to remove/comment out the OPTIONS+="watch" line in /lib/udev/rules.d/13-dm-disk.rules and see if you can still hit the problem...
Comment 2 Peter Rajnoha 2012-03-21 13:40:12 UTC 
Oh, but we need the watch rule for the lvmetad in case the PV is created on top of a DM device. Hmmm...
Comment 3 Peter Rajnoha 2012-03-21 13:44:13 UTC 
(...an alternative to that would be to send the change event ourselves, just by writing "change" to corresponding /sys/block/dm/uevent after we create the PV, so we could probably do without the watch rule)
Comment 4 Corey Marthaler 2012-03-21 19:14:36 UTC 
If I comment out the 'OPTIONS+="watch"' line everything works fine.

============================================================
Iteration 50 of 50 started at Wed Mar 21 14:14:42 CDT 2012
============================================================
SCENARIO - [create_snap_of_inactive_origin]
Create a snapshot of an inactive origin
Making origin volume
lvcreate -L 300M snapper -n origin
Deactivating origin/snap volume(s)
lvchange -an snapper/origin
Creating snap of inactive origin
lvcreate -s /dev/snapper/origin -c 32 -n snap_of_inactive -L 100M
Activating origin/snap volume(s)
lvchange -ay snapper/origin 2>&1
Removing volume snapper/snap_of_inactive
Removing origin snapper/origin
Comment 5 Peter Rajnoha 2012-03-22 13:54:03 UTC 
Actually, pvcreate updates the lvmetad directly so we don't need the watch rule. I'll remove it in next respin as we had cases before where this caused problems in enterprise environment (this will be a RHEL specific patch).
Comment 6 Peter Rajnoha 2012-03-27 12:49:48 UTC 
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
No Documentation needed.
Comment 9 Corey Marthaler 2012-04-04 19:46:20 UTC 
Fix verified in the latest rpms.

2.6.32-251.el6.x86_64
lvm2-2.02.95-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
lvm2-libs-2.02.95-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
lvm2-cluster-2.02.95-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
udev-147-2.40.el6    BUILT: Fri Sep 23 07:51:13 CDT 2011
device-mapper-1.02.74-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
device-mapper-libs-1.02.74-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
device-mapper-event-1.02.74-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
device-mapper-event-libs-1.02.74-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012
cmirror-2.02.95-3.el6    BUILT: Fri Mar 30 09:54:10 CDT 2012


============================================================
Iteration 10 of 10 started at Wed Apr  4 14:46:52 CDT 2012
============================================================
SCENARIO - [create_snap_of_inactive_origin]
Create a snapshot of an inactive origin
Making origin volume
lvcreate -L 300M snapper -n origin
Deactivating origin/snap volume(s)
Creating snap of inactive origin
lvcreate -s /dev/snapper/origin -c 32 -n snap_of_inactive -L 100M
Activating origin/snap volume(s)
Removing volume snapper/snap_of_inactive
Removing origin snapper/origin
Comment 10 errata-xmlrpc 2012-06-20 15:02:54 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0962.html
Note You need to log in before you can comment on or make changes to this bug.