Bug 1224954

Summary: add ipxe bios based on VF hardware to support booting from VF
Product: [oVirt] vdsm Reporter: Ido Barkan <ibarkan>
Component: GeneralAssignee: Dan Kenigsberg <danken>
Status: CLOSED DEFERRED QA Contact: Lukas Svaty <lsvaty>
Severity: high Docs Contact:
Priority: low    
Version: ---CC: alkaplan, bazulay, bugs, danken, ecohen, huding, juzhang, knoel, kraxel, lsurette, mburman, mgoldboi, oourfali, rbalakri, virt-maint, xfu, xwei, yeylon
Target Milestone: ---Keywords: Reopened
Target Release: ---Flags: ylavi: ovirt-4.0.0?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?
Hardware: x86_64   
OS: Linux   
Whiteboard: virt
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-01-04 07:40:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Virt RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 869804    
Attachments:
Description Flags
domain xml of the guest none

Description Ido Barkan 2015-05-26 09:50:32 UTC
Description of problem:
Trying to boot a VM using seabios that has a single NIC which is directly attached by virsh. qemu-kvm refuses to boot from PXE since it does not recognize any NICs. The host uses an SRIOV card and a SRIOV VF is directly attached to the vm. SRIOV card is Intel.


Version-Release number of selected component (if applicable):
drivers on host:
$ ethtool -i enp2s0f0
driver: igb
version: 5.2.13-k
firmware-version: 1.5.4

packages:
qemu-kvm-tools-rhev-1.5.3-86.el7.1.x86_64
qemu-kvm-ev-2.1.2-23.el7_1.3.x86_64
libvirt-daemon-kvm-1.2.8-16.el7.x86_64
qemu-kvm-common-ev-2.1.2-23.el7_1.3.x86_64
qemu-2.0.0-1.el7.3.x86_64
seabios-bin-1.7.5-8.el7.noarch

kernel:
$ uname -r
3.10.0-201.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1. direct attach a VF NIC
2. power on and try to vhange to boot order to prefer the nic
3.

Actual results:
The nic option isn't offered by seabios (disk and cd options are)

Expected results:
seabios should use igbvf driver in order to use the directly attached nic.

Additional info:

Comment 2 Xiaoqing Wei 2015-05-27 05:53:18 UTC
Hi Ido,

Since this is a PXE related issue,
could you pls paste your pxe package version here ?
eg: rpm -qa | grep -i ipxe

Thanks,
Xiaoqing.

Comment 3 Ido Barkan 2015-05-27 10:41:34 UTC
ipxe-roms-qemu-20130517-6.gitc4bce43.el7.noarch

AFAIU this is not pxe issue. pxe is just the sumptom of the fact that seabios does not support SRIOV. But I might be wrong.

Comment 4 Gerd Hoffmann 2015-06-01 07:39:43 UTC
> Actual results:
> The nic option isn't offered by seabios (disk and cd options are)

boot rom missing most likely.

Please attach 'lspci -v' output for the nic (as seen by the guest).
Please attach libvirt domain xml for the guest (or full qemu command line).

Comment 5 Ido Barkan 2015-06-01 10:13:58 UTC
lspci -v [vf nic part]:

00:08.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
        Subsystem: Hewlett-Packard Company Device 31ff
        Physical Slot: 8
        Flags: bus master, fast devsel, latency 0
        Memory at fe050000 (64-bit, non-prefetchable) [size=16K]
        Memory at fe054000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [70] MSI-X: Enable+ Count=3 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Kernel driver in use: igbvf

Comment 6 Ido Barkan 2015-06-01 10:15:30 UTC
Created attachment 1033247 [details]
domain xml of the guest

obtained by:
[root@nari04 ~]# virsh -r list
 Id    Name                           State
----------------------------------------------------
 32    real-os-2                      running
[root@nari04 ~]# virsh -r dumpxml real-os-2 > real-os.xml

Comment 7 Gerd Hoffmann 2015-06-01 11:30:06 UTC
(In reply to Ido Barkan from comment #5)
> lspci -v [vf nic part]:
> 
> 00:08.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev
> 01)
>         Subsystem: Hewlett-Packard Company Device 31ff
>         Physical Slot: 8
>         Flags: bus master, fast devsel, latency 0
>         Memory at fe050000 (64-bit, non-prefetchable) [size=16K]
>         Memory at fe054000 (64-bit, non-prefetchable) [size=16K]
>         Capabilities: [70] MSI-X: Enable+ Count=3 Masked-
>         Capabilities: [a0] Express Endpoint, MSI 00
>         Kernel driver in use: igbvf

Yea, no option rom present.  Configuration issue, you need to supply one.

nilsson root ~# grep "82576 Virtual Function" /usr/share/hwdata/pci.ids 
        10ca  82576 Virtual Function
nilsson root ~# rpm -ql ipxe-roms | grep 10ca
/usr/share/ipxe/808610ca.rom

So, ipxe seems to have a ipxe rom for that hardware.
Try to configure that for the nic.

libvirt has support for that (see https://libvirt.org/formatdomain.html#elementsHostDevSubsys, 'rom' element).

Don't know how to do that with ovirt though.

Comment 8 Ido Barkan 2015-06-08 08:00:24 UTC
great thanks!
So we probably should support it in Ovirt.

Comment 9 Ido Barkan 2015-06-10 07:05:06 UTC
Gerd, it seems that this way is not standard.

I don't even have this package on my el7.1 and after a chat in libvirt-list I wa told that:
"""(for example, I have an 82576 SRIOV card on my Fedora 22 system, and a
*much* newer version of ipxe installed:

  ipxe-roms-qemu-20150407-1.gitdc795b9f.fc22.noarch

but do not have the file 808610ca.rom (or *any* file with 10ca in the
name) in /usr/share/ipxe)"""

Is there a standard way or you expect the SRIOV card user to find the ROM by himself?

Comment 10 Gerd Hoffmann 2015-06-10 07:23:57 UTC
(In reply to Ido Barkan from comment #9)
> Gerd, it seems that this way is not standard.
> 
> I don't even have this package on my el7.1 and after a chat in libvirt-list
> I wa told that:
> """(for example, I have an 82576 SRIOV card on my Fedora 22 system, and a
> *much* newer version of ipxe installed:
> 
>   ipxe-roms-qemu-20150407-1.gitdc795b9f.fc22.noarch
> 
> but do not have the file 808610ca.rom (or *any* file with 10ca in the
> name) in /usr/share/ipxe)"""

"ipxe-roms-qemu.rpm" carries only the roms for the hardware emulated by qemu, and qemu has a rpm dependency on it so this is installed on every virt host.

"ipxe-roms.rpm" has everything else, but isn't installed by default.

As you pass-through physical hardware you have to install the latter.
It is in the optional channel for rhel7.

> Is there a standard way or you expect the SRIOV card user to find the ROM by
> himself?

Naming convention used by ipxe is "${vendorid}${deviceid}.rom" for the filename.  "8086" is vendor intel, "10ca" is the the vf nic device.

Comment 13 Dan Kenigsberg 2016-01-04 07:40:11 UTC
Closing until a customer request requests the feature of booting from a VF.