Bug 1377656

Summary: No vfio device displayed on seabios boot menu
Product: Red Hat Enterprise Linux 7 Reporter: aihua liang <aliang>
Component: seabiosAssignee: Alex Williamson <alex.williamson>
Status: CLOSED NOTABUG QA Contact: Yanan Fu <yfu>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.3CC: aliang, chayang, juzhang, knoel, ngu, shuang, virt-maint, xfu, xutian, yfu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-09-21 09:30:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
seabios_esc.log
none
boot_menu
none
PF Device displayed in seabios boot menu none

Description aihua liang 2016-09-20 10:08:18 UTC
Created attachment 1202807 [details]
seabios_esc.log

Description of problem:

 No vfio device displayed on seabios boot menu.


Version-Release number of selected component (if applicable):
 kernel version: 3.10.0-505.el7.x86_64
 qemu-kvm-rhev version: 3.10.0-505.el7.x86_64
 seabios version: 3.10.0-505.el7.x86_64 

How reproducible:
 100%


Steps to Reproduce:
1.Enable iommu(intel_iommu=on) and load vfio/vfio-pci(modprobe vfio/vfio-pci)
  
2.Get Devices:
#lspci
.....
04:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
04:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
....

3.Create a virtual Ethernet Card.
#echo 1 > /sys/bus/pci/devices/0000\:04\:00.1/sriov_numvfs

4.Check Devices:
#lspci
.....
04:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
04:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
04:0a.0 Ethernet controller: Intel Corporation XL710/X710 Virtual Function (rev 02)
....
 
5.Check Device Vendor/Device ID.
# lspci -n -s 04:00.0
04:00.0 0200: 8086:1583 (rev 02)
# lspci -n -s 04:0a.0
04:0a.0 0200: 8086:154c (rev 02)

6.Unbind device from host.
#echo 0000:04:00.0 > /sys/bus/pci/devices/0000\:04\:00.0/driver/unbind 
#echo 0000:04:0a.0 > /sys/bus/pci/devices/0000\:04\:0a.0/driver/unbind

7.Assign Device to vfio-pci
# echo "8086 1583" >/sys/bus/pci/drivers/vfio-pci/new_id 
# echo "8086 1583" >/sys/bus/pci/drivers/vfio-pci/remove_id 
# echo "8086 154c" >/sys/bus/pci/drivers/vfio-pci/new_id
# echo "8086 154c" >/sys/bus/pci/drivers/vfio-pci/remove_id

8.Start guest with assigned device:
usr/libexec/qemu-kvm -name rhel7 \
-machine pc,accel=kvm,usb=off,vmport=off \
-cpu host \
-m 4096 \
-realtime mlock=off \
-smp 8 \
-uuid 1534fa42-4818-4493-9f67-eee5ba758385 \
-no-user-config -nodefaults \
-chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/test1,server,nowait \
-mon chardev=qmp_id_catch_monitor,id=monitor,mode=readline \
-no-hpet \
-boot menu=on,splash-time=6000 \
-vga qxl \
-vnc 0:3 \
-netdev tap,id=hostnet10 \
-device virtio-net-pci,netdev=hostnet10,id=net10,mac=50:54:00:49:b2:5f \
-spice ipv4,port=5000,disable-ticketing \
-enable-kvm \
-monitor stdio \
-device vfio-pci,host=04:00.0,id=hostnet_PF \
-chardev socket,id=ovmf_log,path=/var/tmp/seabios,server,nowait \
-device isa-debugcon,chardev=ovmf_log,iobase=0x402 \
-drive file=/home/rhel6_8.qcow2,if=none,id=drive-ide0-0-0-0,format=qcow2,cache=none,aio=native,snapshot=off \
-device ide-hd,bus=ide.0,drive=drive-ide0-0-0-0,id=ide0-0-0-0 \
-device vfio-pci,host=04:0a.0,id=hostnet_VF \

9.Press ESC to enter boot menu, check the menu, then continue the boot process.


Actual results:
 No vfio device displayed on seabios boot menu.
 But we can see the assigned device in guest,
  # lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04)
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:04.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
00:05.0 Ethernet controller: Intel Corporation XL710/X710 Virtual Function (rev 02

Expected results:
 vfio device displayed on seabios boot menu.

Additional info:
 a.boot log---attachment:seabios_esc.log
 b.Screen capture of boot menu ---attachment:boot_menu
 b.seabios-1.9.1-5 also have this problem.

Comment 1 aihua liang 2016-09-20 10:09:03 UTC
Created attachment 1202808 [details]
boot_menu

Comment 2 Karen Noel 2016-09-20 10:13:04 UTC
> Version-Release number of selected component (if applicable):
>  kernel version: 3.10.0-505.el7.x86_64
>  qemu-kvm-rhev version: 3.10.0-505.el7.x86_64
>  seabios version: 3.10.0-505.el7.x86_64 

Please update the qemu-kvm-rhev and seabios versions. Thanks.

Comment 3 Yanan Fu 2016-09-20 11:00:51 UTC
Hi aliang, i think you should confirm if there exist one rom for this device.
path: /sys/bus/pci/devices/$DEVICE/ 

Without rom file, it is normal.

Comment 5 Alex Williamson 2016-09-20 12:00:55 UTC
Agree with comment 3, unless the device has a *bootable* ROM this is the expected behavior.  SR-IOV VFs do not have a ROM by definition so the expected results are wrong for this device.  Only the PF even has a chance of being bootable without specifying a romfile for the device.  If host dmesg reports an invalid rom file message then the rom is not bootable.

Comment 6 Yanan Fu 2016-09-21 05:29:59 UTC
Check result:
This is a configuration issue, not a product bug.
XL710 nic need to be configured in "Device Setting" in BIOS.
"Device Setting" ---> XL710 nic --->"NIC Configuration" ---> "Legacy Boot Protocol" ---> "PXE" 
By default, the legacy boot protocol is "NONE", so we can not see the PF in seabios even if the device has one bootable rom.

Comment 7 aihua liang 2016-09-21 09:29:05 UTC
Configured as yanan suggested, PF device can be displayed on seabios boot menu, see attachment.

****************Cmds to start guest***********
/usr/libexec/qemu-kvm -name rhel7 \
-machine pc,accel=kvm,usb=off,vmport=off \
-cpu host \
-m 4096 \
-realtime mlock=off \
-smp 8 \
-uuid 1534fa42-4818-4493-9f67-eee5ba758385 \
-no-user-config -nodefaults \
-chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/test1,server,nowait \
-mon chardev=qmp_id_catch_monitor,id=monitor,mode=readline \
-no-hpet \
-boot menu=on,splash-time=6000 \
-vga qxl \
-vnc 0:3 \
-netdev tap,id=hostnet10 \
-device virtio-net-pci,netdev=hostnet10,id=net10,mac=50:54:00:49:b2:5f \
-spice ipv4,port=5000,disable-ticketing \
-enable-kvm \
-monitor stdio \
-chardev socket,id=ovmf_log,path=/var/tmp/seabios,server,nowait \
-device isa-debugcon,chardev=ovmf_log,iobase=0x402 \
-drive file=/home/rhel6_8.qcow2,if=none,id=drive-ide0-0-0-0,format=qcow2,cache=none,aio=native,snapshot=off \
-device ide-hd,bus=ide.0,drive=drive-ide0-0-0-0,id=ide0-0-0-0 \
-device vfio-pci,host=04:00.0,id=hostnet_PF \
*********************************

When i create ipxe rom for VF device, it seemed not be supported. 

So,close the bug temporarily as "NOTABUG".

Comment 8 aihua liang 2016-09-21 09:29:55 UTC
Created attachment 1203187 [details]
PF Device displayed in seabios boot menu