Bug 1294532 - grub2-mkconfig generates wrong grub.cfg with multi-device btrfs root
grub2-mkconfig generates wrong grub.cfg with multi-device btrfs root
Product: Fedora
Classification: Fedora
Component: grub2 (Show other bugs)
x86_64 Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Peter Jones
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2015-12-28 15:41 EST by Juan Orti
Modified: 2016-12-20 12:26 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2016-12-20 12:26:37 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
output of grub2-mkconfig (33.18 KB, text/plain)
2016-01-07 08:16 EST, Juan Orti
no flags Details
grub.cfg generated (4.91 KB, text/plain)
2016-01-07 08:17 EST, Juan Orti
no flags Details

  None (edit)
Description Juan Orti 2015-12-28 15:41:30 EST
Description of problem:
Previously, grub2-mkconfig was generating good grub.cfg files with my multi-device btrfs root fs composed of 3 gpt partitions:

Recently, I have added a new lvm device to btrfs, so now it looks like this:
# btrfs fi show
Label: 'btrfs_raid1'  uuid: 038b2b48-fd2d-4565-b2b1-d07847ecca8c
        Total devices 4 FS bytes used 1.90TiB
        devid    1 size 1.81TiB used 1.30TiB path /dev/sdb4
        devid    2 size 1.81TiB used 1.30TiB path /dev/sdc4
        devid    3 size 1.81TiB used 1.30TiB path /dev/sdd4
        devid    4 size 1.60TiB used 0.00B path /dev/mapper/vg_hd04-lv_btrfs_hd04

btrfs-progs v4.2.2

With this configuration, grub2-mkconfig generates files syntactically wrong. The "root=" directive now is broken in multiple lines with all the devices of the btrfs. Previously it was correctly generated as root=UUID=xxxx.

Here is an example of one of the entries generated:

menuentry 'Fedora (4.2.8-300.fc23.x86_64) 23 (Workstation Edition)' --class fedora --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-4.2.8-300.fc23.x86_64-advanced-038b2b48-fd2d-4565-b2b1-d07847ecca8c' {
        set gfxpayload=keep
        insmod gzio
        insmod part_gpt
        insmod ext2
        set root='hd1,gpt2'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd1,gpt2 --hint-efi=hd1,gpt2 --hint-baremetal=ahci1,gpt2  4c1427e4-aae6-4069-a90d-8545433147f4
          search --no-floppy --fs-uuid --set=root 4c1427e4-aae6-4069-a90d-8545433147f4
        linuxefi /vmlinuz-4.2.8-300.fc23.x86_64 root=/dev/sdb4
/dev/mapper/vg_hd04-lv_btrfs_hd04 ro rootflags=subvol=root rd.lvm.lv=vg_hd04/lv_btrfs_hd04 rd.lvm.lv=vg_hd04/lv_swap crashkernel=128M resume=UUID=7a7cfbbd-f80b-4819-99ed-1f0f08f2f800 zswap.enabled=1 rhgb quiet 
        initrdefi /initramfs-4.2.8-300.fc23.x86_64.img

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Mix partitions and logical volumes for the root fs
2. grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

Actual results:
Wrong grub.cfg and fails to boot

Expected results:
kernel loaded with root=UUID=xxxx directive

Additional info:
Comment 1 Chris Murphy 2016-01-01 13:43:34 EST
I can't reproduce this with grub2-2.02-0.24.fc24.x86_64. You might try editing /etc/grub.d/10_linux so that 'set -e' is 'set -x' then rerun as 'bash -x grub2-mkconfig', without -o option, and see if you can find out why it's confused. In particular, why isn't it generating root=UUID= for the root volume? It shouldn't even be trying to specify individual member drives.

Are you sure this grub.cfg was made by grub2-mkconfig (called manually) and isn't a grubby created entry following a kernel update?
Comment 2 Juan Orti 2016-01-06 04:59:08 EST
I'm sure that the grub.cfg was made by grub2-mkconfig, I repeated the process many times and it was always generated incorrectly.

After removing the lvm device, it's been generated with root=UUID= again.

I'll try to reproduce it in a VM in the next days and try to debug it as you suggest.
Comment 3 Juan Orti 2016-01-07 08:15:42 EST
I can easily reproduce this bug in a fully updated Fedora 23 VM with this root filesystem:

[root@fedora23w-btrfs ~]# btrfs fi show
Label: 'fedora_fedora23w-btrfs'  uuid: 933ffc76-ecab-4779-9eab-7f8fcb234015
        Total devices 4 FS bytes used 4.02GiB
        devid    1 size 17.51GiB used 0.00B path /dev/vda3
        devid    2 size 20.00GiB used 2.03GiB path /dev/vdb1
        devid    3 size 20.00GiB used 2.03GiB path /dev/vdc1
        devid    4 size 20.00GiB used 2.00GiB path /dev/mapper/vg01-lv01

btrfs-progs v4.2.2

[root@fedora23w-btrfs ~]# mount 
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=1014548k,nr_inodes=253637,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/vda3 on / type btrfs (rw,relatime,seclabel,space_cache,subvolid=257,subvol=/root)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
debugfs on /sys/kernel/debug type debugfs (rw,relatime,seclabel)
tmpfs on /tmp type tmpfs (rw,seclabel)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
nfsd on /proc/fs/nfsd type nfsd (rw,relatime)
/dev/vda3 on /home type btrfs (rw,relatime,seclabel,space_cache,subvolid=258,subvol=/home)
/dev/vda1 on /boot type ext4 (rw,relatime,seclabel,data=ordered)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
tmpfs on /run/user/42 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=204928k,mode=700,uid=42,gid=42)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=204928k,mode=700)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)

I have "set -x" in /etc/grub.d/10_linux, and I attach the files generated by:

grub2-mkconfig -o /tmp/grub.cfg 2>&1 | tee /tmp/grub2-mkconfig.out
Comment 4 Juan Orti 2016-01-07 08:16 EST
Created attachment 1112455 [details]
output of grub2-mkconfig
Comment 5 Juan Orti 2016-01-07 08:17 EST
Created attachment 1112456 [details]
grub.cfg generated
Comment 6 Chris Murphy 2016-01-28 02:25:20 EST
++ /usr/sbin/grub2-probe --device /dev/vda3 /dev/vdb1 /dev/vdc1 /dev/mapper/vg01-lv01 --target=abstraction
+ abstraction=lvm

I don't think this is a Btrfs problem per se, but rather that one of the devices is using LVM, and one of the grub tests ends up causing some confusion and the wrong assumption.

Based on past experience, best to head to http://lists.gnu.org/mailman/listinfo/help-grub and post a request for help, including grub and osprober versions, and the bash -x output (I think it's short enough to go on the list).
Comment 8 Leslie Satenstein 2016-01-28 09:34:15 EST
SUSE has updated their version of os-prober to handle similar problems noted in this posting.

I was testing SUSE's Leap distribution last fall and reported a bug. In January 2116, we tested the update, and the btrfs scanning issues (Multi-disk, multi-boot system with one disk dedicated to Fedora 23 installed with btrfs).

Their version of os-prober was updated.  Perhaps Fedora maintainers should review what SUSE has done to improve os-prober.
Comment 9 Fedora End Of Life 2016-11-24 09:33:49 EST
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 10 Fedora End Of Life 2016-12-20 12:26:37 EST
Fedora 23 changed to end-of-life (EOL) status on 2016-12-20. Fedora 23 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.