Bug 513448

Summary: udev 141-4 "hangs" during boot
Product: [Fedora] Fedora Reporter: David Hull <hull>
Component: udevAssignee: Harald Hoyer <harald>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 11CC: awilliam, chris.stone, davidz, d.bz-redhat, eleaver, esandeen, harald, mcermak, psj, redhat.com, rtc, walovaton
Target Milestone: ---Keywords: CommonBugs
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: https://fedoraproject.org/wiki/Common_F11_bugs#udev-floppy
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-03-17 11:46:03 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
boot.log file from slow boot
none
This boot was the fast one which doesn't happen so often
none
This one was slow
none
Another one slow none

Description David Hull 2009-07-23 17:20:56 UTC
Description of problem:
I just installed the latest version of udev and the latest kernel today.  From /var/log/yum.log

Jul 23 09:12:45 Updated: udev-141-4.fc11.x86_64
Jul 23 09:12:56 Updated: libudev0-141-4.fc11.x86_64
Jul 23 09:14:02 Updated: kernel-firmware-2.6.29.6-213.fc11.noarch
Jul 23 09:14:08 Updated: kernel-headers-2.6.29.6-213.fc11.x86_64
Jul 23 09:14:47 Installed: kernel-devel-2.6.29.6-213.fc11.x86_64
Jul 23 09:15:56 Installed: kernel-2.6.29.6-213.fc11.x86_64

When I rebooted, I initially thought the kernel hung after printing "Starting udev:".  I tried booting into the previous kernel (kernel-2.6.29.5-191.fc11.x86_64) and found the same problem.  However, once I had some more patience, after a minute or two "OK" was printed and the boot continued normally.  I edited /etc/udev/udev.conf to add 'udev_log="debug"' and rebooted.  During the boot, the delay happened around the lines (captured as best as I could):

[delay here]
/lib/udev/devkit-disks-part-id (stdout) 'DKD_MEDIA_AVAILABLE=1'
[???] 'Entering MS-DOS parser (offset=0, size=4096)'
[long delay here]
[???] read failed (Input/output error)
[???] 'Exiting MS-DOS parser'
[???] 'Entering Apple parser'
[long delay here]
[???]

Version-Release number of selected component (if applicable):
udev-141-4.fc11.x86_64

How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 David Hull 2009-07-23 17:39:27 UTC
I rebooted again to try to do a better job of capturing the console messages.

udevd[141]: seq 973 cleanup, pid [854], status 0, 0 seconds old
[ approx 20 second delay ]
udevd-event[799]: /lib/udev/devkit-disks-part-id (stdout) 'DKD_MEDIA_AVAILABLE=1'
udevd-event[799]: /lib/udev/devkit-disks-part-id (stdout) 'Entering MS-DOS parser (offset=0, size=4096)'
[ approx 40 second delay ]
udevd-event[799]: /lib/udev/devkit-disks-part-id (stdout) 'read failed (Input/output error)'
udevd-event[799]: /lib/udev/devkit-disks-part-id (stdout) 'Exiting MS-DOS parser'
udevd-event[799]: /lib/udev/devkit-disks-part-id (stdout) 'Entering Apple parser'
[ approx 40 second delay ]
[ boot continues normally ]

Also, I will attach the /var/log/boot.log, but it does not appear to contain this part of the boot sequence.

Comment 2 David Hull 2009-07-23 17:40:13 UTC
Created attachment 354896 [details]
boot.log file from slow boot

Comment 3 Harald Hoyer 2009-07-24 06:41:05 UTC
It might be, that you don't have a floppy drive, but enabled the floppy drive in your bios.
You can disable it in the bios and/or do:

# echo "blacklist floppy" > /etc/modprobe.d/blacklist-floppy.conf

Comment 4 Harald Hoyer 2009-07-24 09:26:46 UTC
CC'ing David Zeuthen:

Should devkit-disks-part-id try to open floppies?

Comment 5 David Hull 2009-07-25 03:49:05 UTC
(In reply to comment #3)
> It might be, that you don't have a floppy drive, but enabled the floppy drive
> in your bios.

I think you're right.  I'm not at my work machine now, which is the one that prompted my bug report, but I just updated udev on my home machine and ran into much the same problem: a big pause during udev's initialization.  My home machine doesn't have a floppy, but one was configured in the bios.  I disabled it, and the udev time went from around 60 seconds to a few seconds.

I won't be in front of my work machine again for about a week, but I'll see if having a floppy configured in the bios is the problem there as well.  Thank you for your help!

Comment 6 Christopher Stone 2009-07-28 00:26:01 UTC
I got hit by this one as well. No floppy, but one set in my BIOS. Fixing BIOS fixes delay.

Really nasty bug because I thought for sure the system had crashed.  Fortunately I left my PC on while looking for a rescue DVD and the system finally booted and I was able to locate this bug.  Just relieved that I didn't need to try and rescue my OS.

Comment 7 Adam Williamson 2009-08-03 17:59:02 UTC
I've been seeing a similar delay in Rawhide, and I also don't have a floppy drive but see lots of messages about it while udev is delayed. Will confirm that blacklisting the floppy module fixes it, and post back.

-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 8 David Hull 2009-08-04 16:37:37 UTC
(In reply to comment #5)
> I won't be in front of my work machine again for about a week, but I'll see if
> having a floppy configured in the bios is the problem there as well.

I'm back in front of my work machine, and I found that it too had a floppy drive configured in the BIOS, but no floppy actually installed.  Disabling the floppy in the BIOS eliminated the udev delay at boot.

Comment 9 Adam Williamson 2009-08-04 19:15:37 UTC
Confirmed, blacklisting floppy works here. Haven't checked if I have it enabled in BIOS or not.

-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 10 William Lovaton 2009-08-12 03:57:13 UTC
Sorry but this tip doesn't work for me.  I don't have a floppy and it is disabled in the BIOS but udev usually stalls during boot.  Then I tried blacklisting the floppy module as Harald suggested but it didn't work either.

I have a Dell XPS 420 Quad Core with an integrated card reader for SD/Mini/MMC/CF/SM/xD/etc with a Fedora 11 OS with the latest updates.

I'll upload 3 pictures with the bootchart results of 3 consecutive boots, the first one was fast: 38 seconds, the other two were slow: 70 seconds.

This computer used to boot pretty quickly when I made a clean install of Fedora 11 but now after applying updates things have been very slow, it sometimes boots fast, but more often that not it will boot slowly.

Comment 11 William Lovaton 2009-08-12 03:59:35 UTC
Created attachment 357108 [details]
This boot was the fast one which doesn't happen so often

Comment 12 William Lovaton 2009-08-12 04:01:04 UTC
Created attachment 357109 [details]
This one was slow

what is vol_id doing?

Comment 13 William Lovaton 2009-08-12 04:03:57 UTC
Created attachment 357111 [details]
Another one slow

but this time there is no obvious process shown in the graph, may be a bootchart bug?

Comment 14 Harald Hoyer 2009-08-12 08:29:34 UTC
(In reply to comment #10)
> Sorry but this tip doesn't work for me.  I don't have a floppy and it is
> disabled in the BIOS but udev usually stalls during boot.  Then I tried
> blacklisting the floppy module as Harald suggested but it didn't work either.
> 
> I have a Dell XPS 420 Quad Core with an integrated card reader for
> SD/Mini/MMC/CF/SM/xD/etc with a Fedora 11 OS with the latest updates.
> 
> I'll upload 3 pictures with the bootchart results of 3 consecutive boots, the
> first one was fast: 38 seconds, the other two were slow: 70 seconds.
> 
> This computer used to boot pretty quickly when I made a clean install of Fedora
> 11 but now after applying updates things have been very slow, it sometimes
> boots fast, but more often that not it will boot slowly.  

Hmm, seems like readahead reads a lot of files.
Please open a new bug against readahead, attach /var/lib/readahead/early.sorted and disable readahead in /etc/sysconfig/readahead.

Comment 15 William Lovaton 2009-08-13 01:56:52 UTC
> Hmm, seems like readahead reads a lot of files.
> Please open a new bug against readahead, attach /var/lib/readahead/early.sorted
> and disable readahead in /etc/sysconfig/readahead.  

Done:
https://bugzilla.redhat.com/show_bug.cgi?id=517214

Comment 16 Adam Williamson 2009-08-18 21:53:21 UTC
it's probably worth noting that this bug affects a stock qemu virtual machine as created by virt-manager. i'm not entirely sure how you'd 'disable' the floppy controller on one of those, it doesn't appear to be an option in virt-manager anywhere.

-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 17 Julian Tosh 2009-08-24 04:53:51 UTC
Blacklisting the floppy as described above significantly reduced the described delay. Thanks guys!

Comment 18 ed leaver 2009-09-01 16:42:06 UTC
Hi. I'm seeing what appears to be an identical problem in Rawhide (2.6.31-0.190.rc8.fc12.x86_64 8/31/09). I've disabled my unused floppy controler in bios, and disabled readahead, but the problem persists. Setting /etc/udev/udev.conf to "debug" shows the system hanging at apparently random spots in the very lengthy and multi-process udev. The disk activity light remains solid "on" after the udev hang.  

System is Opteron 170 in Abit AT8, Uli 1575 SB, 4GB ram. The machine's main OS is CentOS 5.3 which runs just fine on SATA md raid1. The F12 rawhide is installed on an IDE disk, udev hangs regardless of whether the SATA controller is enabled.

Shall I open a bug against udev in Rawhide, or do you want to beat up on it for a while here first?  

Thanks!

Comment 19 Adam Williamson 2009-09-01 18:01:35 UTC
Open a new bug - if disabling floppy controller doesn't help, it's almost certainly not this bug. Thanks!

-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 20 ed leaver 2009-09-01 23:52:23 UTC
Well... okay. But I have the exact same problem with Leonides as with Rawhide.

Occasionally Fedora [11|12] will boot fine, udev taking less than two seconds. Most of the time (>90%) the boot process hangs in udev, with the hdd activity light lit. Must hit hardware reset to reboot. The install DVDs always work in rescue mode, which allows me to get in and run yum. Boot flags are apic=yes nomodeset (0r radeon.modeset=0). No swap files are set in fstab, / and /boot are set by UUID. Get same behavior using F12's "generic" initrd and a custom one generated by mkinitrd. 

There is no floppy drive. The floppy controller is disabled in bios, as are 1394 and SATA. 

CentOS 5.3 works fine.

I'm trying to find *something* to make the problem less persistent. Not quite desperate enough to compile a custom udev with more debug printf's. But I'm getting there.

Thanks.

Comment 21 Adam Williamson 2009-09-02 00:22:41 UTC
I'm not saying you don't have a bug, it's just not _this_ bug :). The one we've been discussing here is definitely caused by the floppy controller being enabled in the BIOS but no floppy drive present, and it has different symptoms to yours: the problem happens on every boot, but it's not a freeze, just a pause; if you wait it out, the system continues to boot.

So it's definitely a completely different problem, which is why I'm asking you to file a separate bug. Trying to track multiple different problems in a single bug report is always a bad idea.

-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 22 ed leaver 2009-09-03 05:58:11 UTC
Okay, it's Bug #521004 against Rawhide. Thanks!

Comment 23 Eric Sandeen 2009-11-23 18:20:25 UTC
I hit this in F11 as well, and can confirm that the workaround in comment #3 fixed it.

But doesn't this need a better fix; most users won't be savvy enough to know that something is wrong when boot sits there for 40s, will never apply the workaround, and will just assume that Fedora is slow by design, or something... :)

Thanks,
-Eric

Comment 24 Harald Hoyer 2009-11-24 10:57:04 UTC
Hmm, we probably could try once and generate the blacklist file if nothing is found.

In F-12 we would need either regenerate the initramfs image or add "rdblacklist=floppy" to the kernel command line.

The real fix would be, that the BIOS does not advertise the floppy to be present in the system.

Comment 26 Adam Williamson 2010-01-12 23:23:47 UTC
martin: did you really mean your comment to be private?

for me it's not really fixed; we all worked around the issue by disabling our floppy controller in the BIOS, but this is still a behavioural regression that hasn't been changed.



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 27 David Hull 2010-01-13 03:00:50 UTC
I got an e-mail notice that info was needed from me, but I'm not sure what it is.  Please ask again.