Bug 594068 - do not use boot=on with ide disks
do not use boot=on with ide disks
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.0
All Linux
medium Severity high
: rc
: ---
Assigned To: Daniel Veillard
Virtualization Bugs
: Regression, Reopened, TestBlocker
: 631655 (view as bug list)
Depends On:
Blocks: 581275 599016 Rhel6.0LibvirtTier2 630613
  Show dependency treegraph
 
Reported: 2010-05-20 09:36 EDT by Haim
Modified: 2014-01-12 19:46 EST (History)
24 users (show)

See Also:
Fixed In Version: libvirt-0_8_1-21_el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 630613 (view as bug list)
Environment:
Last Closed: 2010-11-11 09:48:15 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Haim 2010-05-20 09:36:13 EDT
Description of problem:

using 'no hardware acceleration' option in qemu (no-kvm) results a situation where vm is stuck on booting from hard drive. 
when running same vm but using the '-enable-kvm' option, vm boots from hard drive and os is loaded. 

qemu commands: 


 - qemu process with kvm enabled - vm boots from hard drive as expected
  
PID TTY      STAT   TIME COMMAND
31956 ?        Sl     0:07 /usr/libexec/qemu-kvm -S -M rhel6.0.0 -cpu qemu64,+cx16,+ssse3,-svm -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name libvirt-pool-07 -uuid 9362025d-0906-4587-9b0e-0561c25aaa53 -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/libvirt-pool-07.monitor,server,nowait -mon chardev=monitor,mode=control -rtc base=2010-4-20T16:23:34 -boot c -drive file=/rhev/data-center/606d043c-ef9c-4c6f-848b-5bd89325c78d/d9124e52-d42a-4b0c-8657-523bc5b6733b/images/ad757a3f-bce5-4629-a91d-9d81bef59377/afd1ef06-a03d-4751-9348-2a4c1467f78f,if=none,id=drive-ide0-0-0,boot=on,format=qcow2,serial=29-a91d-9d81bef59377,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=21,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:23:71:17,bus=pci.0,addr=0x4 -usb -device usb-tablet,id=input0 -vnc 0:0,password -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3
  PID TTY      STAT   TIME COMMAND


 - qemu process with kvm disabled - vm doesn't boot from hard drive and stuck 
   on 'Booting from Hard disk...'

32322 ?        Rl     0:08 /usr/libexec/qemu-kvm -S -M rhel6.0.0 -cpu qemu64,+cx16,+ssse3,-svm -no-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name libvirt-pool-07 -uuid 9362025d-0906-4587-9b0e-0561c25aaa53 -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/libvirt-pool-07.monitor,server,nowait -mon chardev=monitor,mode=control -rtc base=2010-4-20T16:24:49 -boot cn -drive file=/rhev/data-center/606d043c-ef9c-4c6f-848b-5bd89325c78d/d9124e52-d42a-4b0c-8657-523bc5b6733b/images/ad757a3f-bce5-4629-a91d-9d81bef59377/afd1ef06-a03d-4751-9348-2a4c1467f78f,if=none,id=drive-ide0-0-0,boot=on,format=qcow2,serial=29-a91d-9d81bef59377,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=21,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:23:71:17,bus=pci.0,addr=0x4 -usb -device usb-tablet,id=input0 -vnc 0:0,password -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3

versions: 

vdsm-4.9-6.1.x86_64
qemu-kvm-0.12.1.2-2.59.el6.x86_64
libvirt-0.8.1-6.el6.x86_64
2.6.32-21.el6.x86_64 (RHEL 6)

How reproducible: always 


Steps to Reproduce:
1. start vm and make sure -no-kvm arg is provided 
2. access vm via vnc
Comment 3 Ante Karamatic 2010-05-26 10:06:06 EDT
I'm experiencing very similar thing. I'm not sure if it's the same, but in my case without kvm supported hardware, booting freezes on 'GRUB Loading'. I've done some debugging and if I run command provided by libvirt (minus -S), I get the same behaviour. But, if I replace boot=on with boot=off in -drive, it boots and works as expected.

I'm experiencing this on Ubuntu 10.04 (libvirt 0.7.5).
Comment 4 Ante Karamatic 2010-05-30 13:45:35 EDT
As a dirty workaround, I've replaced all occurances of boot=on with boot=off in whole of libvirt code. After that everything works and I guess that's the source of the problem.
Comment 5 Gleb Natapov 2010-05-31 12:19:45 EDT
Where this requirement comes from? extboot does not and never worked without KVM. It relies on some KVM limitation to function that does not exists in plane QEMU.
Comment 6 Dan Kenigsberg 2010-05-31 12:51:41 EDT
With RHEL5's kvm we could boot guests with -no-kvm. Unless stated otherwise, we would like to keep this ability with RHEL6's kvm+libvirt.

Is there a workaround, or is this feature dead?
Comment 7 Gleb Natapov 2010-05-31 13:00:00 EDT
Don't use boot=on then.  (In reply to comment #6)
> With RHEL5's kvm we could boot guests with -no-kvm. Unless stated otherwise, we
Really? With boot=on?

> would like to keep this ability with RHEL6's kvm+libvirt.
Then libvirt shouldn't specify boot=on.
  
> 
> Is there a workaround, or is this feature dead?    
don't use boot=on
Comment 8 Dan Kenigsberg 2010-05-31 13:52:11 EDT
Daniel, please ask Gleb if libvirt should avoid setting boot=on for drives while using -no-kvm? Maybe if he repeats his suggestion 3 more time we would understand the problem better.
Comment 11 Daniel Berrange 2010-06-17 12:51:34 EDT
> > With RHEL5's kvm we could boot guests with -no-kvm.
> Really? With boot=on?

Yes, that works fine with guests I'm running on RHEL5

eg this one

/usr/bin/qemu-system-x86_64 -S -M pc -no-kvm -m 409 -smp 2 -name rhel5x86_64 -uuid a29d663e-3083-1096-b8ba-a11c081e77e0 -no-kvm-pit-reinjection -monitor pty -pidfile /var/run/libvirt/qemu//rhel5x86_64.pid -boot c -drive file=/var/lib/libvirt/images/rhel5x86_64.img,if=ide,index=0,boot=on -drive file=/var/lib/libvirt/images/extra1.img,if=virtio,index=0 -drive file=/var/lib/libvirt/images/extra11.img,if=virtio,index=1 -net nic,macaddr=54:52:00:3a:a8:af,vlan=0 -net tap,fd=18,script=,vlan=0,ifname=vnet0 -net nic,macaddr=54:52:00:3a:a8:bf,vlan=1,model=e1000 -net tap,fd=19,script=,vlan=1,ifname=vnet1 -net nic,macaddr=54:52:00:3a:a8:cf,vlan=2,model=e1000 -net tap,fd=20,script=,vlan=2,ifname=vnet2 -net nic,macaddr=54:52:00:3a:a8:df,vlan=3,model=virtio -net tap,fd=21,script=,vlan=3,ifname=vnet3 -net nic,macaddr=54:52:00:3a:a8:ef,vlan=4,model=virtio -net tap,fd=22,script=,vlan=4,ifname=vnet4 -serial pty -parallel none -usb -vnc 127.0.0.1:1 -soundhw ac97



> > would like to keep this ability with RHEL6's kvm+libvirt.
> Then libvirt shouldn't specify boot=on.

If QEMU reports that it has support for 'boot=on', then libvirt will use it. 
If this flag isn't supposed to be used anymore then it needs to be removed from QEMU, otherwise the regression wrt previous behaviour needs to be fixed. libvirt can't second guess the fact that the RHEL6 kvm binary reports boot=on support, which then doesn't work in certain scenarios.
Comment 13 Daniel Berrange 2010-06-22 08:52:28 EDT
> Either libvirt needs to stop using ,boot=on for anything but virtio disks or we
> need to make qemu not act on ,boot=on for anything but virtio. The second
> option doesn't sound too good.

This has worked without trouble ever since we added it. Before we consider removing anything, we need to know why there is a regression in KVM behaviour in RHEL-6 vs RHEL5 where it was working.
Comment 14 Gleb Natapov 2010-06-22 09:22:43 EDT
(In reply to comment #13)
> > Either libvirt needs to stop using ,boot=on for anything but virtio disks or we
> > need to make qemu not act on ,boot=on for anything but virtio. The second
> > option doesn't sound too good.
> 
> This has worked without trouble ever since we added it. Before we consider
> removing anything, we need to know why there is a regression in KVM behaviour
> in RHEL-6 vs RHEL5 where it was working.

The problem was analyzed a long time ago:
http://www.mail-archive.com/kvm@vger.kernel.org/msg29091.html
I haven't check why old qemu works, my guess is old BIOS haven't locked all BIOS memory.

extboot was always just a hack to overcome bios shortcomings (not being able to boot from virtio). It was never meant to be used with ide and rhev-m people respected that. And -no-kvm option is unsupported in RHEL6, so any use of it may cause data lose, hair lose or worse.

RHEL6 bios now have native ability to boot from virtio and since scsi is unsupported we should drom boot=on and extboot entirely.
Comment 15 Daniel Berrange 2010-07-20 12:09:14 EDT
> RHEL6 bios now have native ability to boot from virtio and since scsi is
> unsupported we should drom boot=on and extboot entirely.

If boot=on is no longer required or recommended, then it should be removed from KVM command line & thus libvirt won't detect its existence and thus won't use it.
Comment 16 Dor Laor 2010-07-21 07:33:31 EDT
(In reply to comment #15)
> > RHEL6 bios now have native ability to boot from virtio and since scsi is
> > unsupported we should drom boot=on and extboot entirely.
> 
> If boot=on is no longer required or recommended, then it should be removed from
> KVM command line & thus libvirt won't detect its existence and thus won't use
> it.    

boot=on was a bad name. It actually means extboot=on.
It should only we used for ide interface. With the current qemu cmdline it is too hard to change. I asked libvirt to change that also in rhel5.
I'm sure there is a BZ against libvirt somewhere.
Comment 17 Dor Laor 2010-07-21 07:44:17 EDT
Only now saw that it was already a libvirt bug, reopen + change component back.
Comment 18 Daniel Veillard 2010-07-29 09:25:05 EDT
Unfortunately just dropping boot=on on drive device when --no-kvm is
used doesn't work in general, I just tried, if using 

-drive file=/var/lib/libvirt/images/test6.img,if=none,id=drive-virtio-disk0,boot=on,format=raw

then booting fails with 'Booting from Hard drive' and gets stuck there
but if that boot=on option is dropped, and using

-drive file=/var/lib/libvirt/images/test6.img,if=none,id=drive-virtio-disk0,format=raw

the same domain now fail to boot with

"Booting from Hard Disk..."
Boot failed: could not read the boot disk

No bootable device.

Of course that same exact domain boots fine if using kvm instead of qemu
everything else being equal.
It seems to me the recipe to remove "boot=on" in case of --no-kvm is being
used is not workable, this must be done in a more fine-grained fashing, maybe
per device type.
I'm waiting for complete description of the change to do, in the current state
the suggested approach just does not work for me.

Daniel
Comment 19 Yaniv Kaul 2010-07-29 09:30:28 EDT
What about '-boot c' ?
Comment 20 Daniel Veillard 2010-07-29 09:42:23 EDT
Actually after updating to seabios-0.5.1-2 the domain will boot ...
very slowly because of the lack of acceleration but that's normal.

Daniel
Comment 21 Daniel Veillard 2010-07-29 09:43:16 EDT
w.r.t. comment #19

Yaniv, the -boot c option was present in both cases

Daniel
Comment 22 Haim 2010-07-29 10:14:45 EDT
Daniel, do you need more information from me on this issue?
Comment 23 Daniel Veillard 2010-07-29 11:10:52 EDT
Yes, basically the problem seems not related to ide drive but to -no-kvm being
incompatible with boot=on in any kind of disk. I reproduced the problem
with a domain using only one virtio device.
So the fix for this issue seems to desactivate boot=on if -no-kvm is used

It's independant of the fact that ide drives don't need boot=on

For RHEL-6 I plan to fix only the first problem, because that's the only
one leading to non-booting domains, i.e. really need to be fixed at this point

Daniel
Comment 24 Daniel Veillard 2010-07-29 11:46:23 EDT
Patch posted upstream and internally

https://www.redhat.com/archives/libvir-list/2010-July/msg00690.html

Daniel
Comment 25 Dave Allan 2010-08-05 10:18:15 EDT
libvirt-0_8_1-21_el6 has been built in RHEL-6-candidate with the fix.

Dave
Comment 28 Mike Cao 2010-09-08 01:52:31 EDT
*** Bug 631655 has been marked as a duplicate of this bug. ***
Comment 29 releng-rhel@redhat.com 2010-11-11 09:48:15 EST
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.

Note You need to log in before you can comment on or make changes to this bug.