Bug 728947

Summary: grub will not boot from non-primary virtio disk
Product: Red Hat Enterprise Linux 6 Reporter: Matthew Booth <mbooth>
Component: grubAssignee: Peter Jones <pjones>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Release Test Team <release-test-team>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.0   
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 728951 728953 (view as bug list) Environment:
Last Closed: 2012-05-11 19:12:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 728951, 728953, 756082    
Attachments:
Description Flags
RHEL 6 guest failing to boot with 'Error 21'
none
RHEL 6 failing to boot with 'Hard Disk Error' none

Description Matthew Booth 2011-08-08 13:05:27 UTC 
Description of problem:
A guest has 2 virtio disks: vda and vdb. /boot resides on /dev/vdb1. Grub is installed on /dev/vda, which is also where the guest is configured to boot from.

Firstly, a fresh installation in the above configuration will cause the boot to fail with 'Hard Disk Error' (screenshot rhel6-grub-hd_error)[1].

If you attempt to fix the problem by manually running 'grub-install /dev/vda' from the rescue environment you  get Error 21 trying to load stage 1.5 (screenshot rhel6-error_21).

The above configuration works fine if the disks are IDE. It also works fine if the drives are swapped so that /dev/vdb1 becomes /dev/vda1 and grub is manually installed on the new /dev/vda. The problem seems to be specific to attempting to read /boot from a drive other than the first one.

I have replicated this problem on RHEL 4.8, RHEL 5.4 and RHEL 6.0, all of which support VirtIO. I have also replicated the problem on a RHEL 5 KVM host, which uses Bochs BIOS, and an F14 host, which uses SeaBIOS. The behaviour is the same in all cases, although the error messages from RHEL 4 and RHEL 5 grub are slightly more verbose.

[1] I'd expect the behaviour to be identical immediately after installation to behaviour after running 'grub-install /dev/vda', so could this indicate a secondary problem in anaconda?
Comment 2 Matthew Booth 2011-08-08 13:28:15 UTC 
For some context, I originally hit this testing V2V conversion of a multiboot guest OS, where the first disk contained Windows XP and the second disk contained RHEL. Grub is configured to boot either OS. The conversion process enables virtio in both Windows and RHEL. After conversion, grub fails with error 21 and neither guest can be booted.

However, if you swap the 2 disks around so that RHEL, and therefore /boot, is on vda, and manually install grub to the new vda, grub can boot RHEL successfully. However, it fails trying to chainload XP, which is now on the second disk. grub.conf is obviously edited appropriate during the switchover too, so this isn't caused by it attempting to chainload the wrong partition.
Comment 3 Matthew Booth 2011-08-08 14:21:08 UTC 
Created attachment 517231 [details]
RHEL 6 guest failing to boot with 'Error 21'
Comment 4 Matthew Booth 2011-08-08 14:21:41 UTC 
Created attachment 517232 [details]
RHEL 6 failing to boot with 'Hard Disk Error'
Comment 5 Peter Jones 2011-11-20 17:27:13 UTC 
Please attach anaconda logs from installation and instructions on how to reproduce the problem.