Created attachment 908948 [details]
rdsosreport.txt from failed boot
Description of problem:
Seems to be some kind of race condition in dracut it fails to mount /sysroot before the storage is ready.
Most of the time
Steps to Reproduce:
1. boot system
Fedora Boot Options.
1: Fedora (3.16.0-0.rc0.git10.1.fc21.armv7hl) 21 (Rawhide)
2: Fedora (3.16.0-0.rc0.git5.1.fc21.armv7hl) 21 (Rawhide)
3: Fedora (3.15.0-1.fc21.armv7hl) 21 (Rawhide)
Enter choice: 1
1: Fedora (3.16.0-0.rc0.git10.1.fc21.armv7hl) 21 (Rawhide)
Retrieving file: /initramfs-3.16.0-0.rc0.git10.1.fc21.armv7hl.img
12898688 bytes read in 754 ms (16.3 MiB/s)
Retrieving file: /vmlinuz-3.16.0-0.rc0.git10.1.fc21.armv7hl
5762248 bytes read in 440 ms (12.5 MiB/s)
append: console=ttymxc0,115200 root=UUID=7ee85ed8-de4a-4779-8658-2daed0d35e97 ro rhgb quiet LANG=en_US.UTF-8
Retrieving file: /dtb-3.16.0-0.rc0.git10.1.fc21.armv7hl/imx6q-wandboard.dtb
29638 bytes read in 508 ms (56.6 KiB/s)
Wrong Image Format for sysboot command
ERROR: can't get kernel image!
Kernel image @ 0x11000000 [ 0x000000 - 0x57ecc8 ]
## Flattened Device Tree blob at 18000000
Booting using the fdt blob at 0x18000000
Loading Ramdisk to 2f3b2000, end 2ffff180 ... OK
Loading Device Tree to 2f3a7000, end 2f3b13c5 ... OK
Starting kernel ...
[ 0.276009] edma-dma-engine edma-dma-engine.0: Can't allocate PaRAM dummy slot
[ 2.561245] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[ 2.568770] sr_init: platform driver register failed for SR
[ OK ] Found device /dev/disk/by-uuid/7ee85ed8-de4a-4779-8658-2daed0d35e97.
[ OK ] Started dracut initqueue hook.
[ OK ] Started Show Plymouth Boot Screen.
[ OK ] Reached target Paths.
[ OK ] Reached target Basic System.
Starting dracut pre-mount hook...
Starting File System Check on /dev/disk/by-uuid/7ee8...2daed0d35e97...
[ 6.100557] systemd-fsck: _/: clean, 159631/1788480 files, 1664672/7507889 blocks
[ OK ] Started File System Check on /dev/disk/by-uuid/7ee85...8-2daed0d35e97.
Stopping File System Check on /dev/disk/by-uuid/7ee8...2daed0d35e97...
[ OK ] Stopped File System Check on /dev/disk/by-uuid/7ee85...8-2daed0d35e97.
[ OK ] Started dracut pre-mount hook.
[FAILED] Failed to mount /sysroot.
See 'systemctl status sysroot.mount' for details.
[DEPEND] Dependency failed for Initrd Root File System.
[DEPEND] Dependency failed for Reload Configuration from the Real Root.
[ OK ] Stopped dracut pre-pivot and cleanup hook.
[ OK ] Stopp Starting File System Check on /dev/disk/by-uuid/7ee8...2daed0d35e97...
[ 7.013765] systemd-fsck: _/: clean, 159631/1788480 files, 1664672/7507889 blocks
Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.
system to boot
This seems to be resolved in RC1 (both Wandboard and Cubietruck).
I suspect it's not so much fixed with rc1 but rather the problem goes away or is masked due to the rc1 kernel build not being built with debug on.
I see the issue with rc2 with debug off so it's not exclusely surrounding debug timing
[ 6.611881] wandboard01.ausil.us dracut-pre-mount: growroot: NOCHANGE: partition 3 is size 60063118. it cannot be grown
do you have some dracut growpart module installed? If yes, does removing it and recreate your initramfs fix the issue?
I think, this is in the cloud-initramfs-tools package.
For the ARM images we likely do. Why would this randomly work/not work?
(In reply to Peter Robinson from comment #5)
> For the ARM images we likely do. Why would this randomly work/not work?
don't know.. not my source code
dracut-modules-growroot is part of cloud-initramfs-tools
Discussed at the 2014-07-09 Fedora 21 alpha blocker review meeting. Accepted as a blocker for Fedora 21 alpha due to violation of the following alpha release criteria :
A system installed with a release-blocking desktop must boot to a log in screen where it is possible to log in to a working desktop using a user account created during installation or a 'first boot' utility.
A system installed without a graphical package set must boot to a state where it is possible to log in through at least one of the default virtual consoles.
I suspect it's an unfortunate interaction with systemd. I guess it needs to be converted to a proper systemd service. However I'm questioning its usefulness for F21. With kernels >3.8 cloud-init can grow the partition post-initrd while it's mounted so the initrd approach is not really required anymore.
Are there any objections to removing the dracut-modules-growroot package from F21?
Moving this back to dracut (at least initially).
The grow root dracut plugin has been removed from the ARM images for some time and the issue is still occurring. It seems if you specify the partition it boots every time, specify the UUID not so much. In the recovery shell the device nodes can be seen. I wonder if this is a hangover form the util-linux issues from earlier
It would be nice to get a journal from a failed boot with "systemd.log_level=debug" on the kernel command line.
(In reply to Harald Hoyer from comment #10)
> It would be nice to get a journal from a failed boot with
> "systemd.log_level=debug" on the kernel command line.
Yes, I'm working to get it.
Discussed at the 2014-08-13 blocker review meeting:
<jsmith> So Peter Jones agreed to put some traces in and track down more information
<pschindl> #info it seems like bug 1109603 has needed attention and there is some work on it.
Discussed in 2014-08-20 Blocker Review Meeting .
Comment #12 is falsely mentioning pjnones instead of probinso (jsmith probably confusing bugs).
#action pwhalen to ping probinso to add logs and move this forward
Discussed at 2014-08-27 blocker review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2014-08-27/f21-blocker-review.2014-08-27-15.59.log.txt . This bug has not been explicitly fixed, and is probably still theoretically lurking about, but pbrobinson and pwhalen say it is not affecting current images. The working hypothesis is that it's some kind of timing-sensitive issue and does not occur when a release (non-debug) kernel is used.
So long as that appears to be the case, we agreed to drop the release blocking status of the bug: from this point on, all F21 builds will use non-debug kernels, so this issue should not affect them. We're not closing the bug as it probably still exists, and it's *possible* it'll start affecting builds again. If that happens, we'll re-nominate it as a blocker.
This bug appears to have been reported against 'rawhide' during the Fedora 22 development cycle.
Changing version to '22'.
More information and reason for this action is here:
I think this is now fixed