Bug 990826

Summary: Fail to mount external usb disks as /dev/sdc1 is not created
Product: [Fedora] Fedora Reporter: Dov Grobgeld <dov.grobgeld>
Component: device-mapper-multipathAssignee: Ben Marzinski <bmarzins>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 19CC: agk, bmarzins, dwysocha, fdinitto, harald, heinzm, jboero, jonathan, lvm-team, msnitzer, prajnoha, prockai, udev-maint, zkabelac
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-02-17 16:29:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dov Grobgeld 2013-08-01 05:07:13 UTC
Description of problem:

When plugging in a new disk the new disk partitions used to automatically spring into existance in /dev/sdc1, /dev/sdc2, etc (or another disk id). This no longer happens.

Here are the relevant lines in /var/log/messages when plugging an external disk with one ntfs partition.

Aug  1 07:57:36 dovg64 kernel: [50355.578158] sd 8:0:0:0: [sdc] Assuming drive cache: write through
Aug  1 07:57:36 dovg64 kernel: [50355.590942]  sdc: sdc1
Aug  1 07:57:36 dovg64 kernel: [50355.594562] sd 8:0:0:0: [sdc] No Caching mode page present
Aug  1 07:57:36 dovg64 kernel: [50355.594568] sd 8:0:0:0: [sdc] Assuming drive cache: write through
Aug  1 07:57:36 dovg64 kernel: [50355.595709] sd 8:0:0:0: [sdc] Attached SCSI disk
Aug  1 07:57:37 dovg64 multipathd: sdc: add path (uevent)
Aug  1 07:57:37 dovg64 systemd-udevd[6962]: inotify_add_watch(7, /dev/sdc1, 10) failed: No such file or directory
Aug  1 07:57:37 dovg64 multipathd: mpathb: load table [0 976769024 multipath 0 0 1 1 service-time 0 1 1 8:32 1]
Aug  1 07:57:37 dovg64 multipathd: mpathb: event checker started
Aug  1 07:57:37 dovg64 multipathd: sdc [8:32]: path added to devmap mpathb

Similarly for a disk-on-key with two partitions (vfat and truecrypt):


Aug  1 08:02:42 dovg64 kernel: [50661.290497] sd 9:0:0:0: [sdc] No Caching mode page present
Aug  1 08:02:42 dovg64 kernel: [50661.290505] sd 9:0:0:0: [sdc] Assuming drive cache: write through
Aug  1 08:02:42 dovg64 kernel: [50661.296158] sd 9:0:0:0: [sdc] No Caching mode page present
Aug  1 08:02:42 dovg64 kernel: [50661.296165] sd 9:0:0:0: [sdc] Assuming drive cache: write through
Aug  1 08:02:42 dovg64 kernel: [50661.329044]  sdc: sdc1 sdc2
Aug  1 08:02:42 dovg64 kernel: [50661.332287] sd 9:0:0:0: [sdc] No Caching mode page present
Aug  1 08:02:42 dovg64 kernel: [50661.332293] sd 9:0:0:0: [sdc] Assuming drive cache: write through
Aug  1 08:02:42 dovg64 kernel: [50661.334550] sd 9:0:0:0: [sdc] Attached SCSI removable disk
Aug  1 08:02:42 dovg64 multipathd: sdc: add path (uevent)
Aug  1 08:02:42 dovg64 systemd-udevd[7079]: inotify_add_watch(7, /dev/sdc2, 10) failed: No such file or directory
Aug  1 08:02:42 dovg64 systemd-udevd[7062]: inotify_add_watch(7, /dev/sdc1, 10) failed: No such file or directory
Aug  1 08:02:42 dovg64 multipathd: mpatha: load table [0 15646720 multipath 0 0 1 1 service-time 0 1 1 8:32 1]
Aug  1 08:02:42 dovg64 multipathd: mpatha: event checker started
Aug  1 08:02:42 dovg64 multipathd: sdc [8:32]: path added to devmap mpatha

This bug prevents mounting any external usb media.

I have tried to externally create the nodes by mknod, but this fails as well:

mknod /dev/sdc1 b 8 33
[root@dovg64 ChuckPos]# mount /dev/sdc1 /mnt/foo
mount: /dev/sdc1 is already mounted or /mnt/foo busy

Version-Release number of selected component (if applicable):

Linux dovg64 3.10.3-300.fc19.x86_64 #1 SMP Fri Jul 26 00:00:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Fedora 19 updated as of today.

How reproducible:

Everytime.

Steps to Reproduce:
1. Enter external media.
2. Watch /var/log/messages
3. Try to mount the media.

Actual results:

The media can't be mounted.

Expected results:


Additional info:

Comment 1 Harald Hoyer 2013-08-01 07:54:46 UTC
Aug  1 07:57:37 dovg64 multipathd: mpathb: load table [0 976769024 multipath 0 0 1 1 service-time 0 1 1 8:32 1]
Aug  1 07:57:37 dovg64 multipathd: mpathb: event checker started
Aug  1 07:57:37 dovg64 multipathd: sdc [8:32]: path added to devmap mpathb

Aug  1 08:02:42 dovg64 multipathd: mpatha: load table [0 15646720 multipath 0 0 1 1 service-time 0 1 1 8:32 1]
Aug  1 08:02:42 dovg64 multipathd: mpatha: event checker started
Aug  1 08:02:42 dovg64 multipathd: sdc [8:32]: path added to devmap mpatha


Seems like you configured multipath to add your devices to a multipath device.

Comment 2 Dov Grobgeld 2013-08-01 07:58:03 UTC
How can I restore the settings of multipath to system defaults? I certainly have not on purpose done any changes to multipath (a component that I am not familiar with at all).

Comment 3 Ben Marzinski 2013-08-05 22:17:12 UTC
If you don't have multipath devices (these are storage devices with multiple paths to the storage. If you don't know whether you have one or not, then you probably don't) then you should just be able to run

# mpathconf --disable

and you should be fine. You can check by running

# multipath -c /dev/sdc

You should see the following message

/dev/sdc is not a valid multipath device path

You can also just delete the file /etc/multipath.conf

If you ever needed it again, you could just run

# mpathconf --enable

and it would recreate it for you. Let me know if this fixes the issue for you.

Comment 4 Dov Grobgeld 2013-08-07 08:29:03 UTC
Thanks. Unfortunately it does not solve the issue that I can't mount external USB media:

[root@dovg64 dov]# fdisk -l /dev/sdc

Disk /dev/sdc: 4004 MB, 4004511744 bytes, 7821312 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0e418c82

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *          32     7821311     3910640    b  W95 FAT32

[root@dovg64 dov]# multipath -c /dev/sdc
/dev/sdc is not a valid multipath device path

[root@dovg64 dov]# mount | grep sdc
[root@dovg64 dov]# mkdir /mnt/foo
[root@dovg64 dov]# mount /dev/sdc1 /mnt/foo
mount: /dev/sdc1 is already mounted or /mnt/foo busy

These are the messages from /var/log/messages when connecting the device. The failed mount command does not cause any additional messages.

Aug  7 11:26:23 dovg64 kernel: [163994.761519] usb 2-1.3: new high-speed USB device number 10 using ehci-pci
Aug  7 11:26:23 dovg64 kernel: [163994.848082] usb 2-1.3: New USB device found, idVendor=0781, idProduct=5567
Aug  7 11:26:23 dovg64 kernel: [163994.848088] usb 2-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Aug  7 11:26:23 dovg64 kernel: [163994.848093] usb 2-1.3: Product: Firebird USB Flash Drive
Aug  7 11:26:23 dovg64 kernel: [163994.848096] usb 2-1.3: Manufacturer: SanDisk
Aug  7 11:26:23 dovg64 kernel: [163994.848100] usb 2-1.3: SerialNumber: 4C532000030216118271
Aug  7 11:26:23 dovg64 kernel: [163994.849076] usb-storage 2-1.3:1.0: USB Mass Storage device detected
Aug  7 11:26:23 dovg64 kernel: [163994.849538] scsi9 : usb-storage 2-1.3:1.0
Aug  7 11:26:23 dovg64 mtp-probe: checking bus 2, device 10: "/sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.3"
Aug  7 11:26:23 dovg64 mtp-probe: bus: 2, device: 10 was not an MTP device
Aug  7 11:26:24 dovg64 kernel: [163995.853850] scsi 9:0:0:0: Direct-Access     SanDisk  Cruzer Blade     1.26 PQ: 0 ANSI: 5
Aug  7 11:26:24 dovg64 kernel: [163995.854481] sd 9:0:0:0: Attached scsi generic sg3 type 0
Aug  7 11:26:24 dovg64 kernel: [163995.856887] sd 9:0:0:0: [sdc] 7821312 512-byte logical blocks: (4.00 GB/3.72 GiB)
Aug  7 11:26:24 dovg64 kernel: [163995.858726] sd 9:0:0:0: [sdc] Write Protect is off
Aug  7 11:26:24 dovg64 kernel: [163995.859694] sd 9:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Aug  7 11:26:24 dovg64 kernel: [163995.877814]  sdc: sdc1
Aug  7 11:26:24 dovg64 kernel: [163995.885933] sd 9:0:0:0: [sdc] Attached SCSI removable disk
Aug  7 11:26:25 dovg64 multipathd: sdc: add path (uevent)
Aug  7 11:26:25 dovg64 multipathd: mpathc: load table [0 7821312 multipath 0 0 1 1 service-time 0 1 1 8:32 1]
Aug  7 11:26:25 dovg64 multipathd: mpathc: event checker started
Aug  7 11:26:25 dovg64 multipathd: sdc [8:32]: path added to devmap mpathc

Comment 5 Ben Marzinski 2013-08-07 14:46:05 UTC
Sorry. If the multipath device is already created, disabling multipath won't remove the existing devices.  You need to remove the multipath devices with

# multipath -F

This will remove all the existing multipath devices, and disabling multipath will keep new ones from being created.  If you remade your initramfs while multipath was enabled, it's possible that multipathd is getting run in the initramfs.  To solve this, you need to remake your initramfs after running

# mpathconf --disable

To remake your initramfs, you need to run

dracut -H -f <initramfs> <kernel-version>

For instance

# dracut -H -f /boot/initramfs-3.9.0-0.rc7.git1.1.fc20.x86_64.img 3.9.0-0.rc7.git1.1.fc20.x86_64

You may want to back up your existing initramfs before running this.

Comment 6 Dov Grobgeld 2013-08-07 14:55:39 UTC
Thanks! This solved the problem. I still have no idea of how multipath was turned on. And if it interfers with the ability to mount external usb media, something is broken.

Comment 7 Zdenek Kabelac 2013-10-22 09:02:34 UTC
Maybe something with udev rules for multipath devices goes wrong here ?

Comment 8 John Boero 2014-05-05 22:11:49 UTC
Bingo I just ran into this as well.  Highly annoying.  I don't remember enabling multipath either.  Fresh install on a laptop.  Why would multipath be on by default???  Or did I  Thanks for the help I'm sorted but this is odd default behavior.

Comment 9 Ben Marzinski 2014-05-06 14:48:53 UTC
Multipath is not enabled by default, unless it got enabled during the install, and that shouldn't happen unless you actually have multiple paths.  Also, on a fresh install to f19, multipath should always default to using the "find_multipaths" option, which should keep it from ever multipathing a usb device unless it is explicitly told to.

I trust you did have a /etc/mutipath.conf file. If it's still around, can you post it, so I can take a look at it? I'm not sure how you could get a multipath.conf file that would allow multipathing usb devices without either copying it from an earlier install, manually editing it, or running mpathconf a fairly complex mpathconf command.

Comment 10 John Boero 2014-05-06 14:54:59 UTC
I agree multipath shouldn't be enabled.  I don't recall configuring it at all during install.

Contents of my multipath.conf are juts one line (comment):

#Use Defaults

Comment 11 Fedora End Of Life 2015-01-09 19:12:58 UTC
This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 12 Fedora End Of Life 2015-02-17 16:29:41 UTC
Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.