Red Hat Bugzilla – Bug 457979
Update QEMU to use gPXE roms for iSCSI boot support
Last modified: 2009-04-03 08:37:43 EDT
Description of problem:
To enable guests to directly boot from an iSCSI target, without libvirt one can manually start qemu-kvm with the added command line parameter:
where the .rom file came from the gPXE (etherboot) project, and provides an iSCSI initiator as an option ROM. qemu then uses this option rom to retrieve the grub boot sectors, and it's off to the races.
libvirt needs the capability of passing such extra command line parameters to qemu.
Version-Release number of selected component (if applicable):
QEMU already automatically loads option ROMs for PXE boot - any reason why the updated ROM with iSCSI support can't just be integrated into QEMU so we don't have to specify a -option-rom explicitly.
Having users of libvirt need to know about loading option ROMs for booting isn't really going to lead to a good experiance. Most applications using libvirt will never even think to enable the option ROM support, even if we have it. Making it 'just work' in upstream QEMU will ensure all applications using libvirt can use the capability without needing special args.
I'd be happy for qemu to start using the gPXE option ROMs directly for this capability. But I also suspect there may be good reasons to want to pass other command line options to qemu from libvirt in the future, so having a general method to pass such options would be beneficial, even if the specific need for -option-rom isn't needed for long.
I've spoken with upstream KVM/QEMU maintainers and they would like to switch QEMU to using gPXE option ROMS by default, so I don't want to add any option ROM support into libvirt at this time. Re-assigning this bug component to track gPXE in KVM/QEMU
NB, we also explicitly do NOT enable passing of arbitrary QEMU command line args in libvirt. The libvirt XML format needs to be hypervisor agnostic and thus requires that we come up with a standardized representation for any piece of functionality. Allowing QEMU command line args would violate this requirement.
that works for me.
Reassigning: The kvm package no longer exists in rawhide/F11, since it is now
part of 'qemu'.
The blocking issue for resolving this is finding a way to fit both extboot and gPXE option ROMs into QEMU's ROM space. Currently there just isn't enough room for both.
Xen however managed to extend ROM space for their HVM guests, so its possible the same approach they used might work with QEMU/KVM too
user: Keir Fraser <firstname.lastname@example.org>
date: Tue Jan 06 16:02:10 2009 +0000
files: tools/firmware/hvmloader/config.h tools/firmware/hvmloader/e820.h tools/firmware/hvmloader/hvmloader.c tools/firmware/hvmloader/smbios.c tools/firmware/hvmloader/util.c tools/firmware/rombios/rombios.c
rombios: Allow option ROMs to extend up to 0xEA000.
I still don't see how xen does it.
I looked at the patch, and it seems crazy to me. Reason is bios starts at 0xe0000 and goes all the way up to 0x100000. So this is probably overriding bios data, unless there's a clever trick that is not in this patch itself to allow for it.
(btw, I checked xen's firmware, not bochs-bios directly, and they do have a call 0xe0000 in the middle of it...)
That said, I've been spending some time with gpxe today, and got the following:
* virtio net rom is 54k
* rtl is about the same size.
they both go well with extboot.
The only bigger rom is e1000, and it is itself bigger than the space allowed for option roms. I boot tested the ones that fit, and they seem to work on qemu, and fail in today's kvm-userspace.git (not sure why yet). The good news is that e1000 is not _that_ bigger, and it would fit if we decreased the space reserved to vgabios (it does not need it all anyway).
However, as I digged into it, I now fail to see how this would solve matt's problem. Even if we can shape up gpxe/qemu combination to a good state, qemu will only automatically load pxe roms for the network cards it recognizes. It does not seem to be the case for the rom he is trying to load. (Matt, correct me if I'm wrong). I tried to use that rom in place of the plain rtl8139, and it does not seem to work properly.
Matt, can you give us more info on what are you trying to do, and which hardware are you trying to emulate?
I'll settle for whatever hardware I can get, as a starting point. :-) rtl8139 and virtio for starters, we can worry about other NICs later.
The use case is relatively simple - "Boot using iSCSI in Guest".
I want virtual machines to be able to boot directly off an iSCSI LUN, using an iSCSI initiator present in the Virtual Machine BIOS somehow. This exactly mimics booting a physical system, using an iSCSI boot-capable NIC.
Last summer I worked with a Dell intern to document how to boot such a system, using gPXE option ROMs loaded into qemu, as follows, using rtl8139:
qemu-kvm -m 512 –boot n –net nic,macaddr=00:16:3e:17:82:ae,vlan=0 –net tap,script=no,vlan=0,ifname=vnet0 –option-rom pci_10ec_8139.rom
In this particular setup, the gPXE option ROM iSCSI initiator gets its target information from DHCP. At some point it may be beneficial to pass this information via another method (qemu options -> BIOS iBFT -> initiator).
Once the boot loader and initrd, which are read via iSCSI, finish, the initrd carries an iSCSI initator too, reconnects, mounts the root file system, and it's off to the races.
The alternative architecture being driven pretty hard through the industry right now have the iSCSI connections terminate in the hypervisor, where the hypervisor can provide either a software or hardware-accelerated iSCSI initator. In this world, guests won't see iSCSI at all. But it breaks the "end-to-end" principle, and adds a whole lot more intelligence needed in the hypervisor, instead of focusing on making the hypervisor really really fast for network I/O.
My $0.02 -
Unless we are dealing with VT-d aren't we concerned with just a small set of emulated drivers? rtl9139 being the prominent.
And I guess in the case of VT-d we won't use gpxe option roms.
Adding to F11VirtTarget - probably a stretch, though
The hard part of it from F11 POV is to get a gpxe package.
However, if the users are willing to provide their own roms, I believe it can be reasonably targeted for F11. The qemu part is not that difficult, I have already sent part of the patches upstream, discussed the other part with anthony, and although not yet accepted, didn't seem to find many resistance.
Per #11, a gpxe package is up for review.
glommer sent patch upstream:
Okay, applied in rawhide:
* Thu Apr 02 2009 Glauber Costa <email@example.com> - 2:0.10-4
- Support botting gpxe roms.