Bug 1475565 - initramfs not generated at default location if /boot/<machine-id> exists
Summary: initramfs not generated at default location if /boot/<machine-id> exists
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dracut
Version: 26
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: dracut-maint-list
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: https://fedoraproject.org/wiki/Common...
Depends On:
Blocks: F27FinalBlocker
TreeView+ depends on / blocked
 
Reported: 2017-07-26 22:12 UTC by Tim Crawford
Modified: 2017-10-13 23:30 UTC (History)
21 users (show)

Fixed In Version: dracut-046-4.fc27
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-10-13 23:30:13 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
kernel-install output (3.51 KB, text/plain)
2017-08-31 14:20 UTC, Tim Crawford
no flags Details
possible fix (736 bytes, patch)
2017-09-01 08:02 UTC, Harald Hoyer
no flags Details | Diff
Patch to 20-grubby.install (1.10 KB, patch)
2017-09-30 04:40 UTC, Scott Shambarger
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Github datto dattobd issues 112 0 None closed Fedora 26 does not persist through reboot 2020-06-09 07:50:12 UTC
Github datto dattobd issues 93 0 None closed dracut may generate initrd at wrong location 2020-06-09 07:50:12 UTC

Description Tim Crawford 2017-07-26 22:12:18 UTC
Description of problem:

> If <image> is omitted or empty, then the default location /boot/initramfs-<kernel version>.img is used.

If `/boot/<machine-id>/` exists, dracut attempts to create the initramfs at `/boot/<machine-id>/$(uname -r)/initrd`.

Version-Release number of selected component (if applicable):
044-183.fc26

How reproducible:
Always

Steps to Reproduce:
1. `sudo mkdir -p /boot/$(cat /etc/machine-id)/$(uname -r)`
2. `sudo dracut -f -v`

Actual results:
dracut creates the initramfs at `/boot/<machine-id>/$(uname -r)/initrd`

Expected results:
dracut creates the initramfs at `/boot/initramfs-$(uname -r).img`

Comment 1 Neal Gompa 2017-08-04 02:06:09 UTC
This is causing me issues, as DKMS-built kernel modules aren't getting installed into the initramfs that's used by the system to boot.

Comment 2 Harald Hoyer 2017-08-07 13:32:27 UTC
Why does /boot/$(cat /etc/machine-id)/$(uname -r) exist on your machine?

Normally this is only used with native the systemd bootloader.

Comment 3 Tim Crawford 2017-08-30 13:57:55 UTC
It is being automatically generated when upgrading the kernel on this machine.

Comment 4 Harald Hoyer 2017-08-30 14:23:37 UTC
If you remove /boot/$MACHINE_ID , is it created again by kernel upgrades on your machine? It should not.

Comment 5 Tim Crawford 2017-08-30 17:06:06 UTC
Correct.

# uname -r
4.11.11-300.fc26.x86_64
# rm -rf /boot/$(cat /etc/machine-id)
# ls /boot/$(cat /etc/machine-id)
ls: cannot access '/boot/3a04f6d81a0941dca41e1d300776f3f6': No such file or directory
# dnf upgrade -q -y kernel
# ls /boot/$(cat /etc/machine-id)
4.12.8-300.fc26.x86_64/

/boot/initramfs-4.12.8-300.fc26.x86_64.img also exists after upgrading the kernel.

Comment 6 Harald Hoyer 2017-08-31 08:00:12 UTC
what is the stderr output of:

# bash -x kernel-install add $(uname -r) /lib/modules/$(uname -r)/vmlinuz

Comment 7 Tim Crawford 2017-08-31 14:20:04 UTC
Created attachment 1320609 [details]
kernel-install output

Output of `kernel-install add`, after having removed `/boot/<machine-id>`.

Comment 8 Harald Hoyer 2017-09-01 08:02:11 UTC
seems like systemd ships 20-grubby.install now, which only runs after $BOOT_DIR_ABS is created by kernel-install

Comment 9 Harald Hoyer 2017-09-01 08:02:58 UTC
Created attachment 1320860 [details]
possible fix

Comment 10 Tim Crawford 2017-09-05 12:23:33 UTC
With the patch `/boot/<machine-id>/<kver>` is removed, but `/boot/<machine-id>` still exists. Subsequent runs of dracut on that kernel fail.

# dracut -f -v
dracut: Executing: /usr/bin/dracut -f -v
dracut: Can't write to /boot/3a04f6d81a0941dca41e1d300776f3f6/4.12.9-300.fc26.x86_64: Directory /boot/3a04f6d81a0941dca41e1d300776f3f6/4.12.9-300.fc26.x86_64 does not exist or is not accessible.

Comment 11 Zbigniew Jędrzejewski-Szmek 2017-09-06 08:38:14 UTC
I don't think the patch from comment #c9 solved the issue. The location should not change when the directory exists, it's just too brittle and unexpected.

I'll submit a PR against dracut to not autodetect the directory like this instead.

Comment 12 Dalibor Pospíšil 2017-09-17 15:07:17 UTC
This also causes me that I'm not able to boot new kernel while having encrypted home and swap. The initrd is generated badly and it does not ask for password to unlock luks. After removing the /boot/$(cat /etc/machine-id) and dracut --regenerate-all --force, booting works as expected.

Comment 13 Scott Shambarger 2017-09-30 04:40:43 UTC
Created attachment 1332572 [details]
Patch to 20-grubby.install

Here's a patch to /lib/kernel/install.d/20-grubby.install to remove $BOOT_DIR_ABS, it's siblings and it's parent if:
 1 - the grubby plugin is returning 77 (meaning skip other scripts)
 2 - the directories are empty

This should fix the "leftover" directories created by previous kernel-install runs and get dracut (and scripts that use it, like plymouth-set-default-theme) to work -- provided someone hasn't already created initrd files there (possibly with good reason).

I tested this with a kernel upgrade and it worked as expected (cleaning up the extra directories).

The patch is easy to test on an installed kernel (on a non-UEFI machine!) with:
# mkdir -p /boot/$(</etc/machine-id)/empty_dir
# kernel-install add $(uname -r) /lib/modules/$(uname -r)/vmlinuz

Comment 14 Michal Sekletar 2017-10-02 07:46:37 UTC
FYI, another approach to fix this was proposed here,

https://github.com/dracutdevs/dracut/pull/281

Comment 15 Kamil Páral 2017-10-02 17:07:46 UTC
Discussed at blocker review meeting [1]:

Punt (delay decision) - We need to perform more testing to estimate the user impact better.

[1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2017-10-02/

Comment 16 Kamil Páral 2017-10-09 14:15:22 UTC
I've tried to reproduce the "failed to boot encrypted system" scenario, and I couldn't. After new kernel installation, the initrd is always generated at the correct place for me. Only when you try to update the initrd, it's generated elsewhere.

What I did:
Installed F27 Beta Workstation, looked like:

# cd /boot

# ll
drwxr-xr-x. 3 root root     4096 Sep 28 01:22 aaaaa202093c472fa61803f6078f5d7b
-rw-r--r--. 1 root root   190814 Sep 20 20:28 config-4.13.3-300.fc27.x86_64
drwx------. 4 root root     4096 Sep 28 01:22 efi
-rw-r--r--. 1 root root   184380 Feb 11  2017 elf-memtest86+-5.01
drwxr-xr-x. 2 root root     4096 Sep 28 01:19 extlinux
drwx------. 6 root root     4096 Oct  9 15:26 grub2
-rw-------. 1 root root 70718979 Oct  9 15:25 initramfs-0-rescue-3698883553db47b88ebe9db6b9a39aba.img
-rw-------. 1 root root 21952137 Oct  9 15:26 initramfs-4.13.3-300.fc27.x86_64.img
drwx------. 2 root root    16384 Oct  9 15:18 lost+found
-rw-r--r--. 1 root root   182704 Feb 11  2017 memtest86+-5.01
-rw-------. 1 root root  3879460 Sep 20 20:28 System.map-4.13.3-300.fc27.x86_64
-rwxr-xr-x. 1 root root  8233048 Oct  9 15:25 vmlinuz-0-rescue-3698883553db47b88ebe9db6b9a39aba
-rwxr-xr-x. 1 root root  8233048 Sep 20 20:29 vmlinuz-4.13.3-300.fc27.x86_64

# tree aaaaa202093c472fa61803f6078f5d7b/
aaaaa202093c472fa61803f6078f5d7b/
└── 4.13.3-300.fc27.x86_64


Initrds are still updated in proper place at this point:

# dracut -f

# ll
drwxr-xr-x. 3 root root     4096 Sep 28 01:22 aaaaa202093c472fa61803f6078f5d7b
-rw-r--r--. 1 root root   190814 Sep 20 20:28 config-4.13.3-300.fc27.x86_64
drwx------. 4 root root     4096 Sep 28 01:22 efi
-rw-r--r--. 1 root root   184380 Feb 11  2017 elf-memtest86+-5.01
drwxr-xr-x. 2 root root     4096 Sep 28 01:19 extlinux
drwx------. 6 root root     4096 Oct  9 15:26 grub2
-rw-------. 1 root root 70718979 Oct  9 15:25 initramfs-0-rescue-3698883553db47b88ebe9db6b9a39aba.img
-rw-------. 1 root root 21478164 Oct  9 15:45 initramfs-4.13.3-300.fc27.x86_64.img
drwx------. 2 root root    16384 Oct  9 15:18 lost+found
-rw-r--r--. 1 root root   182704 Feb 11  2017 memtest86+-5.01
-rw-------. 1 root root  3879460 Sep 20 20:28 System.map-4.13.3-300.fc27.x86_64
-rwxr-xr-x. 1 root root  8233048 Oct  9 15:25 vmlinuz-0-rescue-3698883553db47b88ebe9db6b9a39aba
-rwxr-xr-x. 1 root root  8233048 Sep 20 20:29 vmlinuz-4.13.3-300.fc27.x86_64

# tree aaaaa202093c472fa61803f6078f5d7b/
aaaaa202093c472fa61803f6078f5d7b/
└── 4.13.3-300.fc27.x86_64


After dnf distrosync and installing new kernel, a new /boot/<hash> directory appears:

# ll
drwxr-xr-x. 3 root root     4096 Oct  9 15:57 3698883553db47b88ebe9db6b9a39aba
drwxr-xr-x. 3 root root     4096 Sep 28 01:22 aaaaa202093c472fa61803f6078f5d7b
-rw-r--r--. 1 root root   190814 Sep 20 20:28 config-4.13.3-300.fc27.x86_64
-rw-r--r--. 1 root root   190419 Sep 28 19:14 config-4.13.4-300.fc27.x86_64
drwx------. 4 root root     4096 Sep 28 01:22 efi
-rw-r--r--. 1 root root   184380 Feb 11  2017 elf-memtest86+-5.01
drwxr-xr-x. 2 root root     4096 Sep 28 01:19 extlinux
drwx------. 6 root root     4096 Oct  9 15:58 grub2
-rw-------. 1 root root 70718979 Oct  9 15:25 initramfs-0-rescue-3698883553db47b88ebe9db6b9a39aba.img
-rw-------. 1 root root 21478164 Oct  9 15:45 initramfs-4.13.3-300.fc27.x86_64.img
-rw-------. 1 root root 21456991 Oct  9 15:58 initramfs-4.13.4-300.fc27.x86_64.img
drwx------. 2 root root    16384 Oct  9 15:18 lost+found
-rw-r--r--. 1 root root   182704 Feb 11  2017 memtest86+-5.01
-rw-------. 1 root root  3879460 Sep 20 20:28 System.map-4.13.3-300.fc27.x86_64
-rw-------. 1 root root  3661616 Sep 28 19:14 System.map-4.13.4-300.fc27.x86_64
-rwxr-xr-x. 1 root root  8233048 Oct  9 15:25 vmlinuz-0-rescue-3698883553db47b88ebe9db6b9a39aba
-rwxr-xr-x. 1 root root  8233048 Sep 20 20:29 vmlinuz-4.13.3-300.fc27.x86_64
-rwxr-xr-x. 1 root root  7471192 Sep 28 19:15 vmlinuz-4.13.4-300.fc27.x86_64

# tree aaaaa202093c472fa61803f6078f5d7b/ 3698883553db47b88ebe9db6b9a39aba/
aaaaa202093c472fa61803f6078f5d7b/
└── 4.13.3-300.fc27.x86_64
3698883553db47b88ebe9db6b9a39aba/
└── 4.13.4-300.fc27.x86_64


This time, updating initrd updates it in a wrong location:

# dracut -f

# ll
drwxr-xr-x. 3 root root     4096 Oct  9 15:57 3698883553db47b88ebe9db6b9a39aba
drwxr-xr-x. 3 root root     4096 Sep 28 01:22 aaaaa202093c472fa61803f6078f5d7b
-rw-r--r--. 1 root root   190814 Sep 20 20:28 config-4.13.3-300.fc27.x86_64
-rw-r--r--. 1 root root   190419 Sep 28 19:14 config-4.13.4-300.fc27.x86_64
drwx------. 4 root root     4096 Sep 28 01:22 efi
-rw-r--r--. 1 root root   184380 Feb 11  2017 elf-memtest86+-5.01
drwxr-xr-x. 2 root root     4096 Sep 28 01:19 extlinux
drwx------. 6 root root     4096 Oct  9 15:58 grub2
-rw-------. 1 root root 70718979 Oct  9 15:25 initramfs-0-rescue-3698883553db47b88ebe9db6b9a39aba.img
-rw-------. 1 root root 21478164 Oct  9 15:45 initramfs-4.13.3-300.fc27.x86_64.img
-rw-------. 1 root root 21456991 Oct  9 15:58 initramfs-4.13.4-300.fc27.x86_64.img
drwx------. 2 root root    16384 Oct  9 15:18 lost+found
-rw-r--r--. 1 root root   182704 Feb 11  2017 memtest86+-5.01
-rw-------. 1 root root  3879460 Sep 20 20:28 System.map-4.13.3-300.fc27.x86_64
-rw-------. 1 root root  3661616 Sep 28 19:14 System.map-4.13.4-300.fc27.x86_64
-rwxr-xr-x. 1 root root  8233048 Oct  9 15:25 vmlinuz-0-rescue-3698883553db47b88ebe9db6b9a39aba
-rwxr-xr-x. 1 root root  8233048 Sep 20 20:29 vmlinuz-4.13.3-300.fc27.x86_64
-rwxr-xr-x. 1 root root  7471192 Sep 28 19:15 vmlinuz-4.13.4-300.fc27.x86_64
[root@localhost-live boot]# date
Mon Oct  9 16:01:11 CEST 2017

# tree aaaaa202093c472fa61803f6078f5d7b/ 3698883553db47b88ebe9db6b9a39aba/
aaaaa202093c472fa61803f6078f5d7b/
└── 4.13.3-300.fc27.x86_64
3698883553db47b88ebe9db6b9a39aba/
└── 4.13.4-300.fc27.x86_64
    └── initrd


But installing a new kernel from updates-testing again puts initrd into proper location:

# ll
drwxr-xr-x. 4 root root     4096 Oct  9 16:06 3698883553db47b88ebe9db6b9a39aba
drwxr-xr-x. 3 root root     4096 Sep 28 01:22 aaaaa202093c472fa61803f6078f5d7b
-rw-r--r--. 1 root root   190814 Sep 20 20:28 config-4.13.3-300.fc27.x86_64
-rw-r--r--. 1 root root   190419 Sep 28 19:14 config-4.13.4-300.fc27.x86_64
-rw-r--r--. 1 root root   190441 Oct  5 19:17 config-4.13.5-300.fc27.x86_64
drwx------. 4 root root     4096 Sep 28 01:22 efi
-rw-r--r--. 1 root root   184380 Feb 11  2017 elf-memtest86+-5.01
drwxr-xr-x. 2 root root     4096 Sep 28 01:19 extlinux
drwx------. 6 root root     4096 Oct  9 16:06 grub2
-rw-------. 1 root root 70718979 Oct  9 15:25 initramfs-0-rescue-3698883553db47b88ebe9db6b9a39aba.img
-rw-------. 1 root root 21478164 Oct  9 15:45 initramfs-4.13.3-300.fc27.x86_64.img
-rw-------. 1 root root 21456991 Oct  9 15:58 initramfs-4.13.4-300.fc27.x86_64.img
-rw-------. 1 root root 21457211 Oct  9 16:06 initramfs-4.13.5-300.fc27.x86_64.img
drwx------. 2 root root    16384 Oct  9 15:18 lost+found
-rw-r--r--. 1 root root   182704 Feb 11  2017 memtest86+-5.01
-rw-------. 1 root root  3879460 Sep 20 20:28 System.map-4.13.3-300.fc27.x86_64
-rw-------. 1 root root  3661616 Sep 28 19:14 System.map-4.13.4-300.fc27.x86_64
-rw-------. 1 root root  3664288 Oct  5 19:17 System.map-4.13.5-300.fc27.x86_64
-rwxr-xr-x. 1 root root  8233048 Oct  9 15:25 vmlinuz-0-rescue-3698883553db47b88ebe9db6b9a39aba
-rwxr-xr-x. 1 root root  8233048 Sep 20 20:29 vmlinuz-4.13.3-300.fc27.x86_64
-rwxr-xr-x. 1 root root  7471192 Sep 28 19:15 vmlinuz-4.13.4-300.fc27.x86_64
-rwxr-xr-x. 1 root root  7475288 Oct  5 19:18 vmlinuz-4.13.5-300.fc27.x86_64

# tree aaaaa202093c472fa61803f6078f5d7b/ 3698883553db47b88ebe9db6b9a39aba/
aaaaa202093c472fa61803f6078f5d7b/
└── 4.13.3-300.fc27.x86_64
3698883553db47b88ebe9db6b9a39aba/
├── 4.13.4-300.fc27.x86_64
│   └── initrd
└── 4.13.5-300.fc27.x86_64

Comment 17 Kamil Páral 2017-10-09 14:18:15 UTC
For the record, we had additional discussion about this bug with Justin Forbes last week:

<kparal> jforbes: what do you think about security implications of not updating initramfs when dracut -f is called?
<kparal> or any other implications you can think of?
<kparal> I want to hear your thoughts in case new kernel installs generate initramfs fine, but tools trying to update it don't work
<jforbes> Possible missing drivers and such are the biggest issue
<jforbes> I mean it isn't really a workable long term solution, there are many reasons to need to update the initramfs
<kparal> aren't all drivers already present when it's generated with the new kernel installed? how would failing to update initramfs miss some drivers?
<kparal> also, what are your thoughts on this being a blocker (considering this case as described)?
<jforbes> Third part modules could be an issue, drivers not part of the kernel proper than dkms builds
<kparal> oh, I see. nvidia trying to build its module and such?
<kparal> virtualbox, etc?
<jforbes> I would be +1 to blocker
<jforbes> We also see a lot of failures for whatever reason fixed by rebuilding initramfs
<kparal> ok, thanks for details. we'll revisit this bug on our next meeting, hopefully with more testing done

Comment 18 Kamil Páral 2017-10-09 16:49:49 UTC
Discussed at blocker review meeting [1]:

punt (delay decision) - No really important breakage is known. We'll wait another week if we find something.

[1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2017-10-09/

Comment 20 Fedora Update System 2017-10-10 10:31:33 UTC
dracut-046-3.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-0fac2c45e5

Comment 21 Kamil Páral 2017-10-10 12:52:03 UTC
(In reply to Fedora Update System from comment #20)
> dracut-046-3.fc27 has been submitted as an update to Fedora 27.
> https://bodhi.fedoraproject.org/updates/FEDORA-2017-0fac2c45e5

Seems fixed.

Comment 22 Fedora Update System 2017-10-11 06:28:59 UTC
dracut-046-3.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-0fac2c45e5

Comment 23 František Zatloukal 2017-10-11 11:54:41 UTC
I've tested dracut-046-3, bug is fixed.

Comment 24 Fedora Update System 2017-10-12 09:00:40 UTC
dracut-046-3.1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-a986f116b0

Comment 25 Fedora Update System 2017-10-12 12:20:18 UTC
dracut-046-4.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-f7109f3f6c

Comment 26 Fedora Update System 2017-10-13 06:25:05 UTC
dracut-046-4.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-f7109f3f6c

Comment 27 Fedora Update System 2017-10-13 23:30:13 UTC
dracut-046-4.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.