Description of problem: All QA automated systems rely on PXE local booting for proper provisioning and testing. All systems are configured in the BIOS to boot PXE first. When we want to provision the systems, we modify the PXE target (using RHTS or now cobbler). When we want to boot locally to run tests, we set the default PXE target to "local". KVM guests do no honor the PXE "local" target. It seems that once you boot PXE, KVM doesn't attach the already installed disks. Version-Release number of selected component (if applicable): kernel-2.6.27.5-113.fc10.x86_64 libvirt-0.4.6-3.fc10.x86_64 kvm-74-5.fc10.x86_64 How reproducible: Every time. Steps to Reproduce: 1. Set KVM guest PXE target to "Network Boot" using virt-manager 2. Boot the KVM guest. 3. In the PXE menu, type "local" Actual results: * See attached screenshot, xml, and libvirt logfile. Expected results: The system should behave as a "real" system behaves and boot the local disk. Additional info: * This makes adding KVM guests into test automation a bit funky since we'll need to do a workaround which involves: When you want to reprovision a guest: 1) virsh destroy $GUEST 2) virsh undefine $GUEST 3) Edit xml to boot off network 4) virsh define $XMLFILE 5) virsh start $GUEST We'd then need to repeat to have it boot to local disk.
Created attachment 324048 [details] Screenshot
Created attachment 324049 [details] Guest XML configuration
Created attachment 324050 [details] /var/log/libvirt/qemu/vguest2.log
Being able to boot KVM-via-PXE statefully would be highly useful for my testing in Cobbler land as well, and would help with virtual deployment (and re-deployment) of non-Linux guests.
The XML only specifies a single device for booting. Can you try setting multiple devices <boot dev='network'/> <boot dev='cdrom'/> <boot dev='hd'/> Which should tell the BIOS to try to boot network, then cdrom, then harddisk in that order.
Using ... <os> <type arch='x86_64' machine='pc'>hvm</type> <boot dev='network'/> <boot dev='cdrom'/> <boot dev='hd'/> </os> Results in ... # cat /var/log/libvirt/qemu/vguest2.log /usr/bin/qemu-kvm -S -M pc -m 1024 -smp 2 -name vguest2 -monitor pty -boot ndc -drive file=/dev/VolGroup00/vguest2,if=virtio,index=0,boot=on -net nic,macaddr=54:52:00:29:89:e5,vlan=0,model=virtio -net tap,fd=16,script=,vlan=0,ifname=vnet0 -serial pty -parallel none -usb -vnc 127.0.0.1:1 -k en-us char device redirected to /dev/pts/3 char device redirected to /dev/pts/4 Too many option ROMS Amy I doing that right?
Wow! I didn't know you could specify multiple boot devs. Using <boot dev='network'/> <boot dev='hd'/> And then pressing 'q' to not boot from networking successfully boots from disk. James, try just the above and see if it does the job for you.
Cole, what we are looking for is when the bootloader is fed the following PXE configuration it should boot from the local disk: DEFAULT local PROMPT 0 TIMEOUT 0 TOTALTIMEOUT 0 ONTIMEOUT local LABEL local LOCALBOOT 0 This will enable us to create a KVM "empty shell" that we can assign what OS it is running just based on changing the PXE configuration. Pressing "q" would be interactive and less useful -- you'd have to catch it really really quickly or you'd be reinstalling.
(In reply to comment #7) > Wow! I didn't know you could specify multiple boot devs. Using > > <boot dev='network'/> > <boot dev='hd'/> > > James, try just the above and see if it does the job for you. With those options in my XML ... my guest fails to start. # virsh dumpxml vguest2 | grep -C2 "<boot" <os> <type arch='x86_64' machine='pc'>hvm</type> <boot dev='network'/> <boot dev='hd'/> </os> <features> # virsh start vguest2 libvir: QEMU error : internal error QEMU quit during monitor startup error: Failed to start domain vguest2 # tail /var/log/libvirt/qemu/vguest2.log /usr/bin/qemu-kvm -S -M pc -m 1024 -smp 2 -name vguest2 -monitor pty -boot nc -drive file=/dev/VolGroup00/vguest2,if=virtio,index=0,boot=on -net nic,macaddr=54:52:00:29:89:e5,vlan=0,model=virtio -net tap,fd=12,script=,vlan=0,ifname=vnet0 -serial pty -parallel none -usb -vnc 127.0.0.1:1 -k en-us char device redirected to /dev/pts/3 char device redirected to /dev/pts/4 Too many option ROMS What am I missing?
jlaska: hmm, works on F9. sounds like a bug. mdehaan: you may just have to test it and see what happens. I let the guest boot to our pxe server which doesn't seem to have an explicit 'local' option. Hitting enter without a selection seems to imply local, but qemu then prompts for the boot from (n)etwork or (q)uit. Maybe qemu is smart enough to notice a 'boot from local' directive from the PXE server, and won't prompt. You'll just have to test it since I'm not sure how to go about it.
Cole, that's what james was trying to do above when he filed the bug, and I watched it happen. """ KVM guests do no honor the PXE "local" target. It seems that once you boot PXE, KVM doesn't attach the already installed disks. """ What specifically should I test?
I just wasn't sure if: not entering a selection on my pxe server & pressing enter == deliberately selecting 'boot from local' on another pxe server == having the pxe server tell the machine/VM 'hey, boot from local' (which is what I understand RHTS does). If those are all equivalent, then it sounds like qemu needs fixing to not prompt based on the pxe request.
My take on this bug is that the F10 kvm/libvirt doesn't let me specify multiple <boot> options. If that were fixed, I suspect it would open the door for PXE "local" booting.
Yes, this is a bug in KVM. The trouble is the new -drive flag and its boot=on syntax is broken wrt to normal -boot arg. We need to use boot=on for VirtIO based disks, but when we do that, then this conflicts with the option ROM for PXE boot. This is a big mess and I'm not sure how to fix it, but it certainly needs addressing somehow, because this is a valid use case
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle. Changing version to '10'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
James, Do you still have this problem if you switch from virtio to e1000? You should use this XML excerpt: <boot dev='network'/> <boot dev='hd'/>
Created attachment 324720 [details] vguest1.xml (w/ multiple <boot> and dev="virtio") Glauber, Yeah, I still seem to have this problem using virtio. # virsh start vguest1 libvir: QEMU error : internal error QEMU quit during monitor startup error: Failed to start domain vguest1 # cat /var/log/libvirt/qemu/vguest1.log /usr/bin/qemu-kvm -S -M pc -m 1024 -smp 2 -name vguest1 -monitor pty -boot nc -drive file=/dev/VolGroup00/vguest1,if=ide,index=0,boot=on -drive file=,if=ide,media=cdrom,index=2 -net nic,macaddr=54:52:00:55:c8:17,vlan=0,model=virtio -net tap,fd=14,script=,vlan=0,ifname=vnet2 -serial pty -parallel none -usb -vnc 127.0.0.1:3 -k en-us char device redirected to /dev/pts/8 char device redirected to /dev/pts/9 Too many option ROMS # virsh dumpxml vguest1 <!-- see attachment -->
Created attachment 324721 [details] vguest1.xml (w/ multiple <boot> and dev="e1000") Now with dev="e1000" # virsh start vguest1 libvir: QEMU error : internal error QEMU quit during monitor startup error: Failed to start domain vguest1 # cat /var/log/libvirt/qemu/vguest1.log /usr/bin/qemu-kvm -S -M pc -m 1024 -smp 2 -name vguest1 -monitor pty -boot nc -drive file=/dev/VolGroup00/vguest1,if=ide,index=0,boot=on -drive file=,if=ide,media=cdrom,index=2 -net nic,macaddr=54:52:00:55:c8:17,vlan=0,model=e1000 -net tap,fd=19,script=,vlan=0,ifname=vnet2 -serial pty -parallel none -usb -vnc 127.0.0.1:3 -k en-us char device redirected to /dev/pts/8 char device redirected to /dev/pts/9 Too many option ROMS
I believe the problem itself is very simple (although I don't really know a good solution without thinking a little bit...) there's only 64k of memory available for option roms, and the virtio rom that ships with our packages is... 64k in size!. So after loading the virtio PXE option rom, we're unable to keep loading option roms, in particular, the extboot option rom we need to kick out virtio boots. ;-( James said he could boot with an older rom I handled to him, which is 32k in size, and the problem os "Too many option ROMS" went away. However, he was still unable to boot from the local target, despite of the fact that he could do a local boot by pressing "q" So we really have two problems in here: The first one is that we cannot boot from our current virtio ROM, because it is too large. We can try to quick fix it by building smaller images. This should be a new BZ agains the etherboot package. And the other, the fact that roms do not honor the local target. For that, I believe we can keep using this BZ.
(In reply to comment #19) > So we really have two problems in here: > > The first one is that we cannot boot from our current virtio ROM, because it is > too large. We can try to quick fix it by building smaller images. This should > be a new BZ agains the etherboot package. Filed this as bug#473137
Apparently this is still a problem with gPXE: http://www.redhat.com/archives/fedora-virt/2009-October/msg00052.html Glauber - please take a look
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
This bug appears to have been reported against 'rawhide' during the Fedora 13 development cycle. Changing version to '13'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Still problem on Fedora 13 final + updates testing. Any change to fix this?
I have some success to boot using PXE by booting manually. May be there is too short default timeout for dhcp request. Try this: 1. start virtual machine 2. when you are prompted to press CTRL-B do it 3. try to get dhcp address running this command: dhcp net0 4. repeat step 3 until you do not get address (reply "ok") 5. boot using command: autoboot If you run "dhcp net0" command immediatelly, it will fail fist time, but second run gets IP address. Then I am able to boot from PXE.
I think local boot works well on current fedora 13 stable. Do you still have this problem? But another problem described here (timeout to boot from PXE) is still present. Should I open a new bug for this? Looks like it's enough to increase PXE network timeout by aprox. 3 seconds. Most simpler workaround is to select "Send Key -> Ctrl-Alt-Del" from menu immediatelly (or after 1-3 seconds) after guest start.
I'm still having this dhcp timeout issue on f13. Opened https://bugzilla.redhat.com/show_bug.cgi?id=638735 to track it.
This message is a reminder that Fedora 13 is nearing its end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 13. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '13'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 13's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 13 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug to the applicable version. If you are unable to change the version, please add a comment here and someone will do it for you. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Fedora 13 changed to end-of-life (EOL) status on 2011-06-25. Fedora 13 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. Thank you for reporting this bug and we are sorry it could not be fixed.
Reopen, bump to rawhide, I haven't been able to test this recently.
With virt-manager on F17 this works for me, you just need to make sure that both network and harddrive boot options are selected, otherwise the disks aren't marked as bootable and things probably won't work. Closing as WORKSFORME, please reopen if anyone still has issues on F17+
Created attachment 600144 [details] no prompt
it seems it's not even prompting for ipxe now. I think something got hardcoded into the rom by accident. Can somebody verify?
Renich, given how old and long this bug report is, let's keep it closed. If you are still experiencing a similar issue, please open a new bug report with the following info: Fedora version qemu version qemu command line (if using libvirt, /var/log/libvirt/qemu/$vmname.log) At least on F17, PXE and boot from local is working fine for me.