Back to bug 1024347
| Who | When | What | Removed | Added |
|---|---|---|---|---|
| Peter Rajnoha | 2013-10-29 13:15:36 UTC | Status | ASSIGNED | POST |
| Peter Rajnoha | 2013-10-29 14:27:16 UTC | Doc Text | Cause: There's an event generated for any device that is being watched for changes by means of special WATCH udev rule. This udev rule is also used for LVs and it causes the /dev directory to be up-to-date with any data written to the LV (mainly the symlinks that are based on metadata, like the content of /dev/disk directory). The event is generated each time the device is closed after being open for writing. Consequence: When converting existing LVs to thin pool volume, an error message "device-mapper: remove ioctl on failed: Device or resource busy" may appear. This is caused by LVM command and udev interaction where the original LV is open for writing and then part of the LV is zeroed so it's prepared for thin pool use. Then the LV is closed which triggers the WATCH rule. Then LVM tries to remove the original volume while it can still be open by udev. This causes the error message to appear. LVM tries to remove the LV a few times before exiting with lvconvert failure. Normally, udev should process the LV quickly and LVM should proceed within next LV removal retries. Workaround (if any): Normally, users can just ignore this error message - the LV is processed correctly on next retry. If the number of retries is not enough, then lvconvert may fail as a result. If this is the case, users are encouraged to replace the 'OPTIONS+="watch"' with '#OPTIONS+="watch"' line in /lib/udev/rules.d/13-dm-disk.rules. This will cause the WATCH rule for LVM volumes to be disabled. However, this may cause the /dev content to be out-of-sync with actual metadata state stored on LV. Result: If LVM needs to retry the LV removal because it's being open in parallel, most notably by udev as described before, it issues an error message "remove ioctl failed: Device or resource busy". If this is the case, the removal is retried several times before lvconvert fails completely. | |
| Doc Type | Bug Fix | Known Issue | ||
| Eliska Slobodova | 2013-11-04 15:34:03 UTC | Doc Text | Cause: There's an event generated for any device that is being watched for changes by means of special WATCH udev rule. This udev rule is also used for LVs and it causes the /dev directory to be up-to-date with any data written to the LV (mainly the symlinks that are based on metadata, like the content of /dev/disk directory). The event is generated each time the device is closed after being open for writing. Consequence: When converting existing LVs to thin pool volume, an error message "device-mapper: remove ioctl on failed: Device or resource busy" may appear. This is caused by LVM command and udev interaction where the original LV is open for writing and then part of the LV is zeroed so it's prepared for thin pool use. Then the LV is closed which triggers the WATCH rule. Then LVM tries to remove the original volume while it can still be open by udev. This causes the error message to appear. LVM tries to remove the LV a few times before exiting with lvconvert failure. Normally, udev should process the LV quickly and LVM should proceed within next LV removal retries. Workaround (if any): Normally, users can just ignore this error message - the LV is processed correctly on next retry. If the number of retries is not enough, then lvconvert may fail as a result. If this is the case, users are encouraged to replace the 'OPTIONS+="watch"' with '#OPTIONS+="watch"' line in /lib/udev/rules.d/13-dm-disk.rules. This will cause the WATCH rule for LVM volumes to be disabled. However, this may cause the /dev content to be out-of-sync with actual metadata state stored on LV. Result: If LVM needs to retry the LV removal because it's being open in parallel, most notably by udev as described before, it issues an error message "remove ioctl failed: Device or resource busy". If this is the case, the removal is retried several times before lvconvert fails completely. | An event is generated for any device that is being watched for changes by means of a special WATCH udev rule. This udev rule is also used for logical volumes and it causes the /dev/ directory to be up-to-date with any data written to the logical volume (mainly the symlinks that are based on metadata, like the content of /dev/disk directory). The event is generated each time the device is closed after being open for writing. device-mapper: remove ioctl on failed: Device or resource busy This is caused by the LVM command and udev interaction where the original logical volume is open for writing and then part of the logical volume is zeroed so it is prepared for thin pool use. Then the logical volume is closed, which triggers the WATCH rule. Then LVM tries to remove the original volume while it can still be opened by udev. This causes the error message to appear. LVM tries to remove the logical volume a few times before exiting with lvconvert failure. Normally, udev should process the logical volume quickly and LVM should proceed within next logical volume removal retries. Normally, users can just ignore this error message; the logical volume is processed correctly on next retry. If the number of retries is not sufficient, then lvconvert can fail as a result. If this is the case, users are encouraged to comment out the OPTIONS+="watch" line in the /lib/udev/rules.d/13-dm-disk.rules file. This will cause the WATCH rule for LVM volumes to be disabled. However, this may cause the /dev/ content to be out-of-sync with actual metadata state stored on the logical volume. If LVM needs to retry the logical volume removal because it is being open in parallel, most notably by udev as described before, it issues an error message "remove ioctl failed: Device or resource busy". If this is the case, the removal is retried several times before lvconvert fails completely. |
| Nenad Peric | 2014-03-31 14:43:16 UTC | CC | nperic | |
| Peter Rajnoha | 2014-06-23 14:53:01 UTC | Status | POST | MODIFIED |
| Fixed In Version | lvm2-2.02.107-1.el6 | |||
| errata-xmlrpc | 2014-06-23 14:57:34 UTC | Status | MODIFIED | ON_QA |
| Nenad Peric | 2014-07-01 13:00:50 UTC | Status | ON_QA | ASSIGNED |
| Peter Rajnoha | 2014-07-31 12:17:30 UTC | Status | ASSIGNED | MODIFIED |
| errata-xmlrpc | 2014-07-31 12:21:21 UTC | Status | MODIFIED | ON_QA |
| Nenad Peric | 2014-07-31 12:27:11 UTC | Status | ON_QA | VERIFIED |
| Alasdair Kergon | 2014-09-18 23:11:44 UTC | Sub Component | Thin Provisioning (RHEL6) | |
| Peter Rajnoha | 2014-09-19 07:20:49 UTC | Doc Text | An event is generated for any device that is being watched for changes by means of a special WATCH udev rule. This udev rule is also used for logical volumes and it causes the /dev/ directory to be up-to-date with any data written to the logical volume (mainly the symlinks that are based on metadata, like the content of /dev/disk directory). The event is generated each time the device is closed after being open for writing. device-mapper: remove ioctl on failed: Device or resource busy This is caused by the LVM command and udev interaction where the original logical volume is open for writing and then part of the logical volume is zeroed so it is prepared for thin pool use. Then the logical volume is closed, which triggers the WATCH rule. Then LVM tries to remove the original volume while it can still be opened by udev. This causes the error message to appear. LVM tries to remove the logical volume a few times before exiting with lvconvert failure. Normally, udev should process the logical volume quickly and LVM should proceed within next logical volume removal retries. Normally, users can just ignore this error message; the logical volume is processed correctly on next retry. If the number of retries is not sufficient, then lvconvert can fail as a result. If this is the case, users are encouraged to comment out the OPTIONS+="watch" line in the /lib/udev/rules.d/13-dm-disk.rules file. This will cause the WATCH rule for LVM volumes to be disabled. However, this may cause the /dev/ content to be out-of-sync with actual metadata state stored on the logical volume. If LVM needs to retry the logical volume removal because it is being open in parallel, most notably by udev as described before, it issues an error message "remove ioctl failed: Device or resource busy". If this is the case, the removal is retried several times before lvconvert fails completely. | Cause: When converting existing logical volume to thin pool logical volume, LVM needs to open the volume temporarily for it to be initialized with zeroes at its start. However, this initialization step caused WATCH udev rule to trigger. The WATCH udev rule is set for all top-level logical volumes to update udev database records if device content changes - the watch udev rule triggers on each close of the device once if it was open for writing before. This is exactly the case of opening the volume for thin-pool initialization. All udev rules are reevaluated based on the WATCH rule which causes subsequent scanning of the device for changes. At the same time, LVM may try to close the device, ending up with an error to be issued since LVM is unable to remove open device. Consequence: There was an error message issued while converting to thin pool volume: "device-mapper: remove ioctl on failed: Device or resource busy". Fix: LVM now uses proper flags for temporary volumes which are used during conversions as intermediate step. These flags direct udev to avoid setting WATCH rule or initiate any scanning on such devices until they're properly initialized. Result: There's no message about device-mapper device removal issued anymore during logical volume conversion to thin pool logical volume. |
| Doc Type | Known Issue | Bug Fix | ||
| errata-xmlrpc | 2014-10-14 08:24:53 UTC | Status | VERIFIED | CLOSED |
| Resolution | --- | ERRATA | ||
| Last Closed | 2014-10-14 04:24:53 UTC |
Back to bug 1024347