Description of problem: The installation of F16 on an MDRAID array succeeds but rebooting into the new install drops you to a "grub rescue" prompt. Version-Release number of selected component (if applicable): grub2 version 1.99 release 0.2.fc16 How reproducible: Install F16 on an MDRAID array. Steps to Reproduce: 1. Install F16 on an MDRAID array. 2. Reboot. 3. Actual results: Boot to "grub rescue" prompt. Expected results: Boot to login prompt.
[root@localhost ~]# parted -l Model: ATA VBOX HARDDISK (scsi) Disk /dev/sda: 8590MB Sector size (logical/physical): 512B/512B Partition Table: gpt Number Start End Size File system Name Flags 1 1049kB 2097kB 1049kB bios_grub 2 2097kB 8181MB 8179MB ext4 software RAID boot 3 8181MB 8589MB 408MB linux-swap(v1) Model: ATA VBOX HARDDISK (scsi) Disk /dev/sdb: 8590MB Sector size (logical/physical): 512B/512B Partition Table: gpt Number Start End Size File system Name Flags 1 1049kB 2097kB 1049kB bios_grub 2 2097kB 8181MB 8179MB ext4 software RAID boot 3 8181MB 8589MB 408MB linux-swap(v1) Model: Linux Software RAID Array (md) Disk /dev/md0: 8179MB Sector size (logical/physical): 512B/512B Partition Table: loop Number Start End Size File system Flags 1 0.00B 8179MB 8179MB ext4
At the "grub rescue" prompt "set" outputs: prefix=(hd0)/boot/grub2 root=(hd0) At the "grub rescue" prompt "ls" outputs: (hd0) (hd1)
/var/log/anaconda/anaconda.program.log ends with: 19:19:40,876 INFO program: Running... grub2-install (hd0) 19:19:52,095 INFO program: Installation finished. No error reported. 19:19:52,098 INFO program: Running... grub2-install (hd1) 19:20:03,112 INFO program: Installation finished. No error reported.
Booted from the installation DVD into its rescue mode and checked grub.cfg (see below). It had "root=(md0)" and the correct UUID entries but no gpt or raid insmod lines. I added "insmod part_gpt", "insmod raid", insmod "mdraid1x", rebooted, and ended up at the "grub rescue prompt" again. [root@localhost ~]# cat /etc/grub2.cfg # # DO NOT EDIT THIS FILE # # It is automatically generated by grub2-mkconfig using templates # from /etc/grub.d and settings from /etc/default/grub # ### BEGIN /etc/grub.d/00_header ### if [ -s $prefix/grubenv ]; then load_env fi set default="0" if [ "${prev_saved_entry}" ]; then set saved_entry="${prev_saved_entry}" save_env saved_entry set prev_saved_entry= save_env prev_saved_entry set boot_once=true fi function savedefault { if [ -z "${boot_once}" ]; then saved_entry="${chosen}" save_env saved_entry fi } function load_video { insmod vbe insmod vga insmod video_bochs insmod video_cirrus } set timeout=20 ### END /etc/grub.d/00_header ### ### BEGIN /etc/grub.d/10_linux ### menuentry 'Fedora Linux, with Linux 3.0.0-1.fc16.i686.PAE' --class fedora --class gnu-linux --class gnu --class os { load_video set gfxpayload=keep insmod ext2 set root='(md0)' search --no-floppy --fs-uuid --set=root d926a2c0-3c34-413b-966d-7281ac8d3f7a echo 'Loading Linux 3.0.0-1.fc16.i686.PAE ...' linux /boot/vmlinuz-3.0.0-1.fc16.i686.PAE root=UUID=d926a2c0-3c34-413b-966d-7281ac8d3f7a ro rd.lvm=0 rd.dm=0 rd.md.uuid=9f679bf8:078e4255:3be4632f:f10ee451 KEYTABLE=us quiet SYSFONT=latarcyrheb-sun16 rhgb rd.luks=0 LANG=en_US.UTF-8 echo 'Loading initial ramdisk ...' initrd /boot/initramfs-3.0.0-1.fc16.i686.PAE.img } menuentry 'Fedora Linux, with Linux 3.0.0-1.fc16.i686.PAE (recovery mode)' --class fedora --class gnu-linux --class gnu --class os { load_video set gfxpayload=keep insmod ext2 set root='(md0)' search --no-floppy --fs-uuid --set=root d926a2c0-3c34-413b-966d-7281ac8d3f7a echo 'Loading Linux 3.0.0-1.fc16.i686.PAE ...' linux /boot/vmlinuz-3.0.0-1.fc16.i686.PAE root=UUID=d926a2c0-3c34-413b-966d-7281ac8d3f7a ro single rd.lvm=0 rd.dm=0 rd.md.uuid=9f679bf8:078e4255:3be4632f:f10ee451 KEYTABLE=us quiet SYSFONT=latarcyrheb-sun16 rhgb rd.luks=0 LANG=en_US.UTF-8 echo 'Loading initial ramdisk ...' initrd /boot/initramfs-3.0.0-1.fc16.i686.PAE.img } ### END /etc/grub.d/10_linux ### <other (empty) sections snipped>
Booted from the installation DVD into its rescue mode and re-installed grub2 with grub2-setup: grub2-setup --verbose --core-image=core2.img --directory=/boot/grub2 --device-map=/boot/grub2/device.map /dev/sda grub2-setup --verbose --core-image=core2.img --directory=/boot/grub2 --device-map=/boot/grub2/device.map /dev/sdb with core2.img built with: grub2-mkimage --verbose --directory=/usr/lib/grub2/i386-pc --prefix=/boot/grub2 --output=/boot/grub2/core2.img --format=i386-pc biosdisk boot configfile ext2 extcmd fshelp gzio mdraid1x minicmd normal part_gpt raid terminal (I wasn't sure which modules are added by default so I checked with "lsmod" on another install which modules were loaded and added the raid ones) where [root@localhost ~]# cat /boot/grub2/device.map # this device map was generated by anaconda (hd0) /dev/sda (hd1) /dev/sdb (md0) /dev/md0 Rebooted to a "grub" prompt. "set" outputs: prefix=(hd0)/boot/grub2 root=(hd0) "ls" outputs (not in this order but I don;t have that screen in front of me): (hd0) (hd0,gpt1) (hd0,gpt2) (hd0,gpt3) (hd1) (hd1,gpt1) (hd1,gpt2) (hd1,gpt3) No "(md0)" in spite of "lsmod" showing that "raid" and "mdraid1x" are loaded. But I can now boot with: set prefix=(hd0,gpt2)/boot/grub2 set root=(hd0,gpt2) linux /boot/vmlinuz... root=/dev/md0 ro initrd /boot/initramfs... boot
grub2-probe outputs everything correctly except for abstraction: [root@localhost ~]# grub2-probe --target=fs --device /dev/sda2 ext2 [root@localhost ~]# grub2-probe --target=fs --device /dev/sdb2 ext2 [root@localhost ~]# grub2-probe --target=fs --device /dev/md0 ext2 [root@localhost ~]# grub2-probe --target=fs / ext2 [root@localhost ~]# grub2-probe --target=fs_uuid --device /dev/sda2 d926a2c0-3c34-413b-966d-7281ac8d3f7a [root@localhost ~]# grub2-probe --target=fs_uuid --device /dev/sdb2 d926a2c0-3c34-413b-966d-7281ac8d3f7a [root@localhost ~]# grub2-probe --target=fs_uuid --device /dev/md0 d926a2c0-3c34-413b-966d-7281ac8d3f7a [root@localhost ~]# grub2-probe --target=fs_uuid / d926a2c0-3c34-413b-966d-7281ac8d3f7a [root@localhost ~]# grub2-probe --target=drive --device /dev/sda (hd0) [root@localhost ~]# grub2-probe --target=drive --device /dev/sda2 (hd0,gpt2) [root@localhost ~]# grub2-probe --target=drive --device /dev/sdb (hd1) [root@localhost ~]# grub2-probe --target=drive --device /dev/sdb2 (hd1,gpt2) [root@localhost ~]# grub2-probe --target=drive --device /dev/md0 (md0) [root@localhost ~]# grub2-probe --target=device / /dev/md0 [root@localhost ~]# grub2-probe --target=partmap --device /dev/sda2 gpt [root@localhost ~]# grub2-probe --target=partmap --device /dev/sdb2 gpt [root@localhost ~]# grub2-probe --target=abstraction --device /dev/sda2 [root@localhost ~]# grub2-probe --target=abstraction --device /dev/sdb2 [root@localhost ~]# grub2-probe --target=abstraction --device /dev/md0 [root@localhost ~]# grub2-probe --target=abstraction / [root@localhost ~]#
I am not sure if things were planned to change for grub2 in F16, but normally grub treats raid 1 array elements as individual file systems and accesses them read only. For this to work the arrays need to be version 0.90 or 1.0. 1.1 and 1.2 arrays will prevent that trick from working.
I logged on to reply to Bruno's post and noticed in Comment 6 that the UUID output by "grub2-probe --target=fs_uuid ..." is the same for sda1, sdb2, and md0. It didn't seem correct so I booted up my VM and the output of "search --fs_uuid d926a2c0-3c34-413b-966d-7281ac8d3f7a" at the "grub" prompt is "(hd0,gpt2) (hd1,gpt2)", which confirms the grub2-probe result but blkid disagrees: [root@localhost ~]# blkid /dev/sda2: UUID="9f679bf8-078e-4255-3be4-632ff10ee451" UUID_SUB="3c727c1b-7302-aaa7-5d73-e73f2a2d8839" LABEL="localhost.localdomain:0" TYPE="linux_raid_member" /dev/sda3: UUID="ea914222-ffe9-455e-a4ec-d5313fb94ab8" TYPE="swap" /dev/sdb2: UUID="9f679bf8-078e-4255-3be4-632ff10ee451" UUID_SUB="352c117a-cdf8-3da3-4116-713b180c2394" LABEL="localhost.localdomain:0" TYPE="linux_raid_member" /dev/sdb3: UUID="c0671e1b-f40a-4e3c-a83e-ac39f5a3b16a" TYPE="swap" /dev/md0: UUID="d926a2c0-3c34-413b-966d-7281ac8d3f7a" TYPE="ext4"
Bruno (Comment 7) FYI: [root@localhost ~]# cat /proc/mdstat Personalities : [raid1] md0 : active raid1 sda2[0] sdb2[1] 7987188 blocks super 1.0 [2/2] [UU] bitmap: 0/1 pages [0KB], 65536KB chunk [root@localhost anaconda]# grep mdadm /var/log/anaconda/anaconda.program.log 19:09:22,076 INFO program: Running... mdadm --create /dev/md0 --run --level=1 --raid-devices=2 --metadata=1.0 --bitmap=internal /dev/sda2 /dev/sdb2 19:09:22,531 ERR program: mdadm: array /dev/md0 started. My system's booting up from sda2 because (md0) isn't detected by grub and I run the following at the "grub" prompt: set prefix=(hd0,gpt2)/boot/grub2 set root=(hd0,gpt2) configfile /boot/grub2/grub.cfg and select the "Fedora 3.0.0-1.fc16.i686.PAE" entry defined by [root@localhost anaconda]# cat /etc/grub2.cfg set timeout=9 set default=0 function load_video { insmod vbe insmod vga insmod video_bochs insmod video_cirrus } menuentry 'Fedora 3.0.0-1.fc16.i686.PAE' { load_video insmod part_gpt #insmod raid #insmod mdraid1x insmod ext2 search --no-floppy --fs-uuid --set=root d926a2c0-3c34-413b-966d-7281ac8d3f7a linux /boot/vmlinuz-3.0.0-1.fc16.i686.PAE root=UUID=d926a2c0-3c34-413b-966d-7281ac8d3f7a ro rd.lvm=0 rd.dm=0 rd.md.uuid=9f679bf8:078e4255:3be4632f:f10ee451 KEYTABLE=us SYSFONT=latarcyrheb-sun16 rd.luks=0 LANG=en_US.UTF-8 vga=794 initrd /boot/initramfs-3.0.0-1.fc16.i686.PAE.img }
Since the raid modules weren't embedded in core.img by grub2-install and they weren't insmod'd by grub.cfg, I've looked at both grub2-install and grub2-mkconfig. From "/usr/sbin/grub2-install" # Then the partition map module. In order to support partition-less media, # this command is allowed to fail (--target=fs already grants us that the # filesystem will be accessible). partmap_module= for x in `"$grub_probe" --device-map="${device_map}" --target=partmap --device "${grub_device}" 2> /dev/null`; do case "$x" in netbsd | openbsd) partmap_module="$partmap_module part_bsd";; "") ;; *) partmap_module="$partmap_module part_$x";; esac done which explains why a "set" at the "grub rescue" prompt only returns "(hd0) (hd1)". From "/usr/lib/grub/grub-mkconfig_lib uses_abstraction () { device=$1 abstraction="`${grub_probe} --device ${device} --target=abstraction`" for module in ${abstraction}; do if test "x${module}" = "x$2"; then return 0 fi done return 1 } which explains why grub2-mkconfig/10_linux don't insmod the raid modules in grub.cfg I also noticed # The order in this list is critical. Be careful when modifying it. modules="$modules $disk_module" modules="$modules $fs_module $partmap_module $devabstraction_module" in "/usr/sbin/grub2-install" so, without much conviction, I rebuilt core2.img and re-installed grub with grub2-setup just in case it might help; it didn't.
Slightly OT but I'm curious. Why are all the disk and filesystem "insmod" lines in grub.cfg needed when core.img is built with the necessary modules? (As I would expect it to be normally.) I can boot with just: set prefix=(hd0,gpt2)/boot/grub2 set root=(hd0,gpt2) configfile /boot/grub2/grub.cfg and [root@localhost anaconda]# cat /etc/grub2.cfg set timeout=9 set default=0 function load_video { insmod vbe insmod vga insmod video_bochs insmod video_cirrus } menuentry 'Fedora 3.0.0-1.fc16.i686.PAE' { load_video linux /boot/vmlinuz-3.0.0-1.fc16.i686.PAE root=UUID=d926a2c0-3c34-413b-966d-7281ac8d3f7a ro rd.lvm=0 rd.dm=0 rd.md.uuid=9f679bf8:078e4255:3be4632f:f10ee451 KEYTABLE=us SYSFONT=latarcyrheb-sun16 rd.luks=0 LANG=en_US.UTF-8 vga=794 initrd /boot/initramfs-3.0.0-1.fc16.i686.PAE.img }
You do have 1.0 metadata, so that isn't the problem. (With 1.0 metadata, the raid info is stored at the end of the block device, so that one can access an element of an array, read only, as though it were not a raid device and things will work.)
Thanks. The difference between grub1 and grub2 is that grub1 sets root as "(hdX,Y) and grub2 as "(mdZ)". Since, in the default F16 install, neither "(hd0,gpt2)" nor "(md0)" are recognized, ... As a (possibly misinformed) user, I would've expected "grub-install /dev/sda" to install grub2 with the root device as "(hdX,Y)" like grub1 (and as Anaconda does) and "grub-install /dev/mdZ" to install grub2 with the root device as "(mdZ)". I've re-installed grub specifying the root device with: grub2-setup --verbose --core-image=core2.img --directory=/boot/grub2 --device-map=/boot/grub2/device.map --root-device='(hd0,gpt2)' /dev/sda grub2-setup --verbose --core-image=core2.img --directory=/boot/grub2 --device-map=/boot/grub2/device.map --root-device='(hd0,gpt2)' /dev/sdb and I'm now booting to the grub menu. Regarding the difference between metadata 0.9 and 1.0 and metadata 1.1 and 1.2, I've been intrigued since you mentioned in Comment 7, so I created an Ubuntu 11.04 VM with gpt and mdraid because it defaults to metadata 1.2. 1. I switched to the grub cli at boot and tried to boot with the same "set prefix..." and "set root" as F16; you're right, it fails. (But it works when I delete all the insmods.) 2. FYI, here's a comparison between the two VMs, should it be of interest. Fedora [root@localhost ~]# blkid /dev/sda2: UUID="9f679bf8-078e-4255-3be4-632ff10ee451" UUID_SUB="3c727c1b-7302-aaa7-5d73-e73f2a2d8839" LABEL="localhost.localdomain:0" TYPE="linux_raid_member" /dev/sdb2: UUID="9f679bf8-078e-4255-3be4-632ff10ee451" UUID_SUB="352c117a-cdf8-3da3-4116-713b180c2394" LABEL="localhost.localdomain:0" TYPE="linux_raid_member" /dev/md0: UUID="d926a2c0-3c34-413b-966d-7281ac8d3f7a" TYPE="ext4" [root@localhost ~]# ls -l /dev/disk/by-uuid/ lrwxrwxrwx 1 root root 9 Aug 22 13:33 d926a2c0-3c34-413b-966d-7281ac8d3f7a -> ../../md0 root@localhost:~# mdadm --examine /dev/sda2 | grep UUID Array UUID : 9f679bf8:078e4255:3be4632f:f10ee451 Device UUID : 3c727c1b:7302aaa7:5d73e73f:2a2d8839 root@localhost:~# mdadm --examine /dev/sdb2 | grep UUID Array UUID : 9f679bf8:078e4255:3be4632f:f10ee451 Device UUID : 352c117a:cdf83da3:4116713b:180c2394 [root@localhost ~]# mdadm --detail /dev/md0 | grep UUID UUID : 9f679bf8:078e4255:3be4632f:f10ee451 [root@localhost ~]# rpm -q --qf '%{name} %{version}-%{release}\n' util-linux util-linux 2.19.1-2.fc16 [root@localhost ~]# rpm -q --qf '%{name} %{version}-%{release}\n' grub2 grub2 1.99-0.2.fc16 [root@localhost ~]# grub2-install --version grub2-install (GRUB) 1.99~rc1 Ubuntu root@localhost:~# blkid /dev/sda2: UUID="0dae631e-c4c1-dd38-9803-d0ac0c920a79" LABEL="localhost:0" TYPE="linux_raid_member" /dev/md0: UUID="489ac9fd-e8f8-46c9-aafb-65fc6e1db63e" TYPE="ext4" /dev/sdb2: UUID="0dae631e-c4c1-dd38-9803-d0ac0c920a79" LABEL="localhost:0" TYPE="linux_raid_member" root@localhost:~# ls -l /dev/disk/by-uuid/ lrwxrwxrwx 1 root root 9 2011-08-23 13:31 489ac9fd-e8f8-46c9-aafb-65fc6e1db63e -> ../../md0 root@localhost:~# mdadm --examine /dev/sda2 | grep UUID Array UUID : 0dae631e:c4c1dd38:9803d0ac:0c920a79 Device UUID : bc8f5b99:cf735af2:e3f9fb5e:51d9ab7a root@localhost:~# mdadm --examine /dev/sdb2 | grep UUID Array UUID : 0dae631e:c4c1dd38:9803d0ac:0c920a79 Device UUID : 1afcecb8:e351edb2:cdb27a6c:5479e28b root@localhost:~# mdadm --detail /dev/md0 | grep UUID UUID : 0dae631e:c4c1dd38:9803d0ac:0c920a79 root@localhost:~# dpkg-query -W -f='${Package} ${Version}\n' util-linux util-linux 2.17.2-9.1ubuntu4 root@localhost:~# dpkg-query -W -f='${Package} ${Version}\n' grub-pc grub-pc 1.99~rc1-13ubuntu3 root@localhost:~# grub-install --version grub-install (GRUB) 1.99~rc1-13ubuntu3
I duped my VM then downloaded and installed ftp://ftp.gnu.org/gnu/grub/grub-1.99.tar.gz. It boots OK. [root@localhost ~]# grub-probe / --target=device /dev/md0 [root@localhost ~]# grub-probe / --target=drive (mduuid/9f679bf8078e42553be4632ff10ee451) [root@localhost ~]# grub-probe --device /dev/md0 --target=drive (mduuid/9f679bf8078e42553be4632ff10ee451) When I switch to the "grub" prompt at boot ls outputs: (md/0) <...all disks and partitions> set outputs prefix=(mduuid/9f679bf8078e42553be4632ff10ee451)/boot/grub root=(mduuid/9f679bf8078e42553be4632ff10ee451) FYI: [root@localhost ~]# cat /boot/grub/device.map (hd0) /dev/disk/by-id/ata-VBOX_HARDDISK_VB1fd4a442-c04e76f8 (hd1) /dev/disk/by-id/ata-VBOX_HARDDISK_VBc3c46fad-843f828f
grub2-1.99-1.fc16 has resolved the issue. Thanks.
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
(In reply to comment #0) > Description of problem: > The installation of F16 on an MDRAID array succeeds but rebooting into the new > install drops you to a "grub rescue" prompt. Me too. *** This bug has been marked as a duplicate of bug 732164 ***