Bug 1109603

Summary: dracut unable to boot 3.16 most of the time
Product: [Fedora] Fedora Reporter: Dennis Gilmore <dennis>
Component: dracutAssignee: dracut-maint-list
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 22CC: andreas.bierfert, awilliam, bruno, dracut-maint-list, harald, Jan.van.Eldik, jonathan, jskladan, juergh, kparal, pbrobinson, pwhalen, robatino, satellitgo, tflink, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-04-08 13:54:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 245418    
Attachments:
Description Flags
rdsosreport.txt from failed boot none

Description Dennis Gilmore 2014-06-15 20:45:16 UTC
Created attachment 908948 [details]
rdsosreport.txt from failed boot

Description of problem:
Seems to be some kind of race condition in dracut it fails to mount /sysroot before the storage is ready.

How reproducible:
Most of the time

Steps to Reproduce:
1. boot system
2.
3.

Actual results:
Fedora Boot Options.
1:      Fedora (3.16.0-0.rc0.git10.1.fc21.armv7hl) 21 (Rawhide)
2:      Fedora (3.16.0-0.rc0.git5.1.fc21.armv7hl) 21 (Rawhide)
3:      Fedora (3.15.0-1.fc21.armv7hl) 21 (Rawhide)
Enter choice: 1
1:      Fedora (3.16.0-0.rc0.git10.1.fc21.armv7hl) 21 (Rawhide)
Retrieving file: /initramfs-3.16.0-0.rc0.git10.1.fc21.armv7hl.img
12898688 bytes read in 754 ms (16.3 MiB/s)
Retrieving file: /vmlinuz-3.16.0-0.rc0.git10.1.fc21.armv7hl
5762248 bytes read in 440 ms (12.5 MiB/s)
append: console=ttymxc0,115200 root=UUID=7ee85ed8-de4a-4779-8658-2daed0d35e97 ro rhgb quiet LANG=en_US.UTF-8
Retrieving file: /dtb-3.16.0-0.rc0.git10.1.fc21.armv7hl/imx6q-wandboard.dtb
29638 bytes read in 508 ms (56.6 KiB/s)
Wrong Image Format for sysboot command
ERROR: can't get kernel image!
Kernel image @ 0x11000000 [ 0x000000 - 0x57ecc8 ]
## Flattened Device Tree blob at 18000000
   Booting using the fdt blob at 0x18000000
   Loading Ramdisk to 2f3b2000, end 2ffff180 ... OK
   Loading Device Tree to 2f3a7000, end 2f3b13c5 ... OK

Starting kernel ...

[    0.276009] edma-dma-engine edma-dma-engine.0: Can't allocate PaRAM dummy slot
[    2.561245] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[    2.568770] sr_init: platform driver register failed for SR
[  OK  ] Found device /dev/disk/by-uuid/7ee85ed8-de4a-4779-8658-2daed0d35e97.
[  OK  ] Started dracut initqueue hook.
[  OK  ] Started Show Plymouth Boot Screen.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Basic System.
         Starting dracut pre-mount hook...
         Starting File System Check on /dev/disk/by-uuid/7ee8...2daed0d35e97...
[    6.100557] systemd-fsck[197]: _/: clean, 159631/1788480 files, 1664672/7507889 blocks
[  OK  ] Started File System Check on /dev/disk/by-uuid/7ee85...8-2daed0d35e97.
         Stopping File System Check on /dev/disk/by-uuid/7ee8...2daed0d35e97...
[  OK  ] Stopped File System Check on /dev/disk/by-uuid/7ee85...8-2daed0d35e97.
[  OK  ] Started dracut pre-mount hook.
         Mounting /sysroot...
[FAILED] Failed to mount /sysroot.
See 'systemctl status sysroot.mount' for details.
[DEPEND] Dependency failed for Initrd Root File System.
[DEPEND] Dependency failed for Reload Configuration from the Real Root.
[  OK  ] Stopped dracut pre-pivot and cleanup hook.
[  OK  ] Stopp         Starting File System Check on /dev/disk/by-uuid/7ee8...2daed0d35e97...

Generating "/run/initramfs/rdsosreport.txt"
[    7.013765] systemd-fsck[236]: _/: clean, 159631/1788480 files, 1664672/7507889 blocks


Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.


Expected results:
system to boot

Additional info:

Comment 1 Paul Whalen 2014-06-17 14:44:36 UTC
This seems to be resolved in RC1 (both Wandboard and Cubietruck).

Comment 2 Peter Robinson 2014-06-17 15:00:40 UTC
I suspect it's not so much fixed with rc1 but rather the problem goes away or is masked due to the rc1 kernel build not being built with debug on.

Comment 3 Peter Robinson 2014-06-25 09:19:33 UTC
I see the issue with rc2 with debug off so it's not exclusely surrounding debug timing

Comment 4 Harald Hoyer 2014-06-25 10:08:51 UTC
[    6.611881] wandboard01.ausil.us dracut-pre-mount[196]: growroot: NOCHANGE: partition 3 is size 60063118. it cannot be grown


do you have some dracut growpart module installed? If yes, does removing it and recreate your initramfs fix the issue?

I think, this is in the cloud-initramfs-tools package.

Comment 5 Peter Robinson 2014-06-25 10:30:59 UTC
For the ARM images we likely do. Why would this randomly work/not work?

Comment 6 Harald Hoyer 2014-07-08 09:19:29 UTC
(In reply to Peter Robinson from comment #5)
> For the ARM images we likely do. Why would this randomly work/not work?

don't know.. not my source code

dracut-modules-growroot is part of cloud-initramfs-tools

reassigning

Comment 7 Tim Flink 2014-07-09 17:42:19 UTC
Discussed at the 2014-07-09 Fedora 21 alpha blocker review meeting. Accepted as a blocker for Fedora 21 alpha due to violation of the following alpha release criteria [1]:

A system installed with a release-blocking desktop must boot to a log in screen where it is possible to log in to a working desktop using a user account created during installation or a 'first boot' utility.

A system installed without a graphical package set must boot to a state where it is possible to log in through at least one of the default virtual consoles. 

[1] https://fedoraproject.org/wiki/Fedora_21_Alpha_Release_Criteria#Expected_installed_system_boot_behavior

Comment 8 Juerg Haefliger 2014-08-06 11:44:17 UTC
I suspect it's an unfortunate interaction with systemd. I guess it needs to be converted to a proper systemd service. However I'm questioning its usefulness for F21. With kernels >3.8 cloud-init can grow the partition post-initrd while it's mounted so the initrd approach is not really required anymore.

Are there any objections to removing the dracut-modules-growroot package from F21?

Comment 9 Peter Robinson 2014-08-11 13:56:28 UTC
Moving this back to dracut (at least initially).

The grow root dracut plugin has been removed from the ARM images for some time and the issue is still occurring. It seems if you specify the partition it boots every time, specify the UUID not so much. In the recovery shell the device nodes can be seen. I wonder if this is a hangover form the util-linux issues from earlier

Comment 10 Harald Hoyer 2014-08-11 14:27:53 UTC
It would be nice to get a journal from a failed boot with "systemd.log_level=debug" on the kernel command line.

Comment 11 Peter Robinson 2014-08-11 14:31:41 UTC
(In reply to Harald Hoyer from comment #10)
> It would be nice to get a journal from a failed boot with
> "systemd.log_level=debug" on the kernel command line.

Yes, I'm working to get it.

Comment 12 Kamil Páral 2014-08-13 17:43:15 UTC
Discussed at the 2014-08-13 blocker review meeting:
http://meetbot.fedoraproject.org/fedora-blocker-review/2014-08-13/
<jsmith> So Peter Jones agreed to put some traces in and track down more information
<pschindl> #info it seems like bug 1109603 has needed attention and there is some work on it.

Comment 13 Josef Skladanka 2014-08-20 16:47:19 UTC
Discussed in 2014-08-20 Blocker Review Meeting [1].

Comment #12 is falsely mentioning pjnones instead of probinso (jsmith probably confusing bugs). 
#action pwhalen to ping probinso to add logs and move this forward


[1] http://meetbot.fedoraproject.org/fedora-blocker-review/2013-08-20/

Comment 14 Adam Williamson 2014-08-27 17:31:45 UTC
Discussed at 2014-08-27 blocker review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2014-08-27/f21-blocker-review.2014-08-27-15.59.log.txt . This bug has not been explicitly fixed, and is probably still theoretically lurking about, but pbrobinson and pwhalen say it is not affecting current images. The working hypothesis is that it's some kind of timing-sensitive issue and does not occur when a release (non-debug) kernel is used.

So long as that appears to be the case, we agreed to drop the release blocking status of the bug: from this point on, all F21 builds will use non-debug kernels, so this issue should not affect them. We're not closing the bug as it probably still exists, and it's *possible* it'll start affecting builds again. If that happens, we'll re-nominate it as a blocker.

Comment 15 Jaroslav Reznik 2015-03-03 16:01:59 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 22 development cycle.
Changing version to '22'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora22

Comment 16 Peter Robinson 2015-04-08 13:54:20 UTC
I think this is now fixed