RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 869254 - /dev/mapper/mpathX is block device not symbolic link
Summary: /dev/mapper/mpathX is block device not symbolic link
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: lvm2
Version: 6.4
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Peter Rajnoha
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On: 864368 872174
Blocks: 869253
TreeView+ depends on / blocked
 
Reported: 2012-10-23 12:14 UTC by Peter Rajnoha
Modified: 2013-02-21 08:14 UTC (History)
14 users (show)

Fixed In Version: lvm2-2.02.98-2.el6
Doc Type: Bug Fix
Doc Text:
Cause: libdevmapper incorrectly processed the udev synchronisation and device node management state if the udev synchronisation was disabled (--noudevsync option in dmsetup and --noudevsync or activation/udev_sync=0 lvm.conf option in LVM2) and at the same time we did not expect the verification of udev operations to be used (so not using --verifyudev in dmsetup and using activation/verify_udev_operations=0 LVM2 configuration option or libdevmapper users using DM_UDEV_DISABLE_LIBRARY_FALLBACK flag). Consequence: Disabling the udev synchronisation caused the udev verification to be always used, ignoring the actual state of using or not using the --verifyudev option in dmsetup or activation/verify_udev_operations configuration option in LVM2 or DM_UDEV_DISABLE_LIBRARY_FALLBACK flag. This ended up with libdevmapper/LVM2 to process the relevant nodes and symlinks in /dev and bypassing udev even if it was not desirable. Fix: The libdevmapper library has been fixed to honour the actual need for udev verification based on user's settings, not defaulting to always using it if udev synchronisation is off. Result: After the fix, the device nodes and symlinks in /dev are processed by udev if udev verification and udev synchronisation is not used at the same time.
Clone Of: 864368
Environment:
Last Closed: 2013-02-21 08:14:37 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:0501 0 normal SHIPPED_LIVE lvm2 bug fix and enhancement update 2013-02-20 21:30:45 UTC

Description Peter Rajnoha 2012-10-23 12:14:26 UTC 
Cloning for libdevmapper part of this fix for 6.4.

+++ This bug was initially created as a clone of Bug #864368 +++

Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
# rpm -q device-mapper-multipath
device-mapper-multipath-0.4.9-29.fc18.x86_64
# uname -a
Linux storageqe-02-iscsi1.lab.bos.redhat.com 3.6.0-3.fc18.x86_64 #1 SMP Wed Oct 3 13:29:15 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
#ls -l /dev/mapper/
total 0
crw-------. 1 root root  10, 236 Oct  7 20:46 control
brw-rw----. 1 root disk 253,   0 Oct  7 20:46 mpathb
brw-------. 1 root root 253,   1 Oct  7 20:46 mpathbp1
brw-------. 1 root root 253,   2 Oct  7 20:46 mpathbp2
lrwxrwxrwx. 1 root root        7 Oct  7 20:46 vg_storageqe--02--iscsi1-lv_root -> ../dm-4
lrwxrwxrwx. 1 root root        7 Oct  7 20:46 vg_storageqe--02--iscsi1-lv_swap -> ../dm-3
lrwxrwxrwx. 1 root root        7 Oct  7 20:46 VolGroup00-LogVol00 -> ../dm-5
lrwxrwxrwx. 1 root root        7 Oct  7 20:46 VolGroup00-LogVol01 -> ../dm-6

  
Actual results:


Expected results:


Additional info:

--- Additional comment from xiaoli on 2012-10-09 11:03:42 CEST ---

close it if it's by design.

--- Additional comment from prajnoha on 2012-10-09 11:44:00 CEST ---

If there are nodes instead of symlinks in /dev/mapper, it means that libdevmapper was used in "non-udev" mode and there was a fallback to original direct node creation in libdevmapper, bypassing the udev.

Mpath should work now with udev support/udev synchronisation enabled by default so the situation described here is a fallback in case there's something wrong with udev setup.

As for kpartx and mappings created for partitions defined on top of an mpath device, it has an extra "-s" parameter to enable udev synchronisation (kpartx is called within udev rules directly to automatically create these mappings).

Ben, I've noticed there's still "kpartx" call without "-s" in 62-multipath.rules: 

  RUN+="$env{MPATH_SBIN_PATH}/kpartx -a -p p $tempnode"

We should change that to use the "-s" as well... Otherwise the device is not visible from udev database accessible from libudev etc. and that one could be used as a primary source of information for the list of existing devices in the system - LVM uses that for example and if udev is bypassed, the device is not seen by LVM then (or any other subsystem using udev DB).

That's for kpartx, though I'm not quite sure now why the non-partition mpath mappings bypassed udev.

--- Additional comment from bmarzins on 2012-10-23 05:31:24 CEST ---

Is there any way to tell libdevmapper that I want udev to create the device, but I don't want to wait for that to get done? I'm really leery of having kpartx, which is called from udev, wait on udev.

--- Additional comment from prajnoha on 2012-10-23 13:37:22 CEST ---

(In reply to comment #3)
> Is there any way to tell libdevmapper that I want udev to create the device,
> but I don't want to wait for that to get done? 

Yup... However, I see another part of the problem now! The libdm part was not correctly passing flags that control udev supporting code in case udev_sync is not used! The DM_UDEV_DISABLE_LIBRARY_FALLBACK was not propagated internally while udev_sync is off, it should be fixed with this patch (I'll include that in next 6.4 build):

  http://git.fedorahosted.org/cgit/lvm2.git/commit/?id=a820a68619c10108666f8093545837233c954046

Another part of the fix is for kpartx that should use the above-mentioned flag to control libdm to not try fixing any nodes and symlinks (in case udev_sync is not used - actually, this is the preferred way as, otherwise, there's a race between udev and libdm with respect to node creation without udev_sync. So we need to gracefully let udev do the work and disable any further checks and fallbacks to direct node creation).

--- Additional comment from prajnoha on 2012-10-23 13:40:07 CEST ---

Created attachment 632040 [details]
kpartx patch to use DM_UDEV_DISABLE_LIBRARY_FLAG

Patch for kpartx attached... (we should probably make this a default setting in libdm)

--- Additional comment from prajnoha on 2012-10-23 14:00:17 CEST ---

The use of DM_UDEV_DISABLE_LIBRARY_FLAG will cause libdm to rely on udev completely to create nodes and symlinks, libdm itself will not try any checks and it won't do any corrections with this flag used.

Ben, it's up to you - you may use this flag conditionally (e.g. dmsetup uses "--verifyudev" to not use this flag and do any corrections if udev did not create the nodes/symlinks for some reason, but by default, it makes use of it). Though, the library fallback should be just for debug, I think - if udev works correctly, there's no reason to have missing nodes/symlinks even if completely relying on udev.

--- Additional comment from prajnoha on 2012-10-23 14:04:59 CEST ---

Created attachment 632052 [details]
complete patch to use DM_UDEV_DISABLE_LIBRARY_FALLBACK in mpath tools

...attaching complete patch for mpath tools

--- Additional comment from prajnoha on 2012-10-23 14:07:20 CEST ---

Created attachment 632055 [details]
complete patch to use DM_UDEV_DISABLE_LIBRARY_FALLBACK in mpath tools
Comment 1 Peter Rajnoha 2012-10-23 12:15:16 UTC 
libdevmapper part fixed with this commit:

http://git.fedorahosted.org/cgit/lvm2.git/commit/?id=a820a68619c10108666f8093545837233c954046
Comment 2 Peter Rajnoha 2012-10-23 13:04:28 UTC 
To QA:

To test this issue, you can use the following sequence (just test bare dm device, not an mpath device):

# dmsetup create test --table "0 1 zero" --noudevsync
# ls -l /dev/mapper/
total 0
crw-------. 1 root root  10, 236 Oct 23 08:31 control
brw-rw----. 1 root disk 253,   0 Oct 23 14:40 test

--> "test" is a node, libdm should have let udev to make the symlink instead (despite the operation itself is not synced with udev, but udev should still make the /dev content, not libdm). Having the node in there *is not correct* (udev creates symlinks in /dev/mapper, libdm creates nodes - but it should do that ONLY if --verifyudev is used)


After the patch applied - this is the *correct* behaviour:

# dmsetup create test --table "0 1 zero" --noudevsync
# ls -l /dev/mapper/
total 0
crw-------. 1 root root 10, 236 Oct 23 08:31 control
lrwxrwxrwx. 1 root root       7 Oct 23 14:50 test -> ../dm-0

--> "test" is a symlink, no libdm fallback has been done and libdm did not touch any of the /dev content


# dmsetup create test --table "0 1 zero" --noudevsync --noudevrules
[0] rawhide/~ # ls -l /dev/mapper/
total 0
crw-------. 1 root root 10, 236 Oct 23 08:31 control

--> *no* symlink nor any node created for "test" - we disabled udev rules and "udev verify" is disabled by default as well


[0] rawhide/~ # dmsetup create test --table "0 1 zero" --noudevsync --noudevrules --verifyudev
[0] rawhide/~ # ls -l /dev/mapper/
total 0
crw-------. 1 root root  10, 236 Oct 23 08:31 control
brw-rw----. 1 root disk 253,   0 Oct 23 14:57 test

--> "test" is a node - we disabled udev and we enabled "udev verify" - so the node has been created by libdm during "udev verify"
Comment 3 Nenad Peric 2012-10-25 10:35:31 UTC 
Adding QA Ack based on comment #2, 

This bug will be marked verified once final 6.4
regression testing has been completed.
Comment 5 Nenad Peric 2013-01-08 15:12:48 UTC 
(09:02:02) [root@r6-node01:/dev]$ dmsetup create test_dev --table "0 1 zero" --noudevsync
(09:02:14) [root@r6-node01:/dev]$ ls tes*
ls: cannot access tes*: No such file or directory

(09:02:23) [root@r6-node01:/dev/mapper]$ ls -l te*
lrwxrwxrwx. 1 root root 7 Jan  8 09:02 test_dev -> ../dm-2



(09:03:12) [root@r6-node01:/dev/mapper]$ dmsetup create test --table "0 1 zero" --noudevsync --noudevrules
(09:03:15) [root@r6-node01:/dev/mapper]$ ls -l 
total 0
crw-rw----. 1 root root 10, 58 Jan  8 03:35 control

(09:03:20) [root@r6-node01:/dev/mapper]$ cd ..
(09:03:25) [root@r6-node01:/dev]$ ls -l te*
ls: cannot access te*: No such file or directory

No symlink or node created

(09:07:16) [root@r6-node01:/dev]$ dmsetup create test_dev --table "0 1 zero" --noudevsync --noudevrules --verifyudev
(09:08:22) [root@r6-node01:/dev]$ ls -l tes*
ls: cannot access tes*: No such file or directory
(09:08:26) [root@r6-node01:/dev]$ cd mapper/
(09:08:31) [root@r6-node01:/dev/mapper]$ ls -l tes*
brw-rw----. 1 root disk 253, 3 Jan  8 09:08 test_dev


The behavior is as expected and described in Comment #2


Verifying the symlink/node creation behavior with:


lvm2-2.02.98-6.el6.x86_64
lvm2-devel-2.02.98-6.el6.x86_64
udev-147-2.43.el6.x86_64
device-mapper-1.02.77-6.el6.x86_64
Comment 6 errata-xmlrpc 2013-02-21 08:14:37 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0501.html
Note You need to log in before you can comment on or make changes to this bug.