Bug 1601318

Summary: Failed to define/create guest with vf as hostdev interface with boot order
Product: Red Hat Enterprise Linux 7 Reporter: chhu
Component: libvirtAssignee: Laine Stump <laine>
Status: CLOSED ERRATA QA Contact: chhu
Severity: medium Docs Contact:
Priority: urgent    
Version: 7.6CC: dyuan, lmen, meili, mtessun, xuzhang, yafu, yalzhang
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-4.5.0-10.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1630393 (view as bug list) Environment:
Last Closed: 2018-10-30 09:58:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1630393    
Attachments:
Description Flags
guest xml
none
libvirtd log none

Description chhu 2018-07-16 05:19:03 UTC
Created attachment 1459049 [details]
guest xml

Description of problem:
Failed to define/create guest with vf as hostdev interface with boot order

Version-Release number of selected component (if applicable):
libvirt-4.5.0-3.el7.x86_64
qemu-kvm-rhev-2.12.0-7.el7.x86_64
kernel: 3.10.0-918.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Prepare an env with 82576 VF, and guest xml as below:

<domain type='kvm'>
  <name>r7</name>
  <memory unit='KiB'>2048576</memory>
  <currentMemory unit='KiB'>2048576</currentMemory>
  <vcpu placement='static' cpuset='0-1'>2</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.6.0'>hvm</type>
    <bootmenu enable='yes' timeout='3000'/>
    <bios useserial='yes' rebootTimeout='3000'/>
  </os>
......
    <interface type='hostdev' managed='yes'>
      <mac address='00:16:3e:77:e2:ed'/>
      <source>
      <address type='pci' domain='0x0000' bus='0x08' slot='0x10' function='0x3'/>
      </source>
      <boot order='1'/>
      <rom bar='on' file='/usr/share/ipxe/808610ca.rom'/>
    </interface>

# cat t.xml|grep boot
    <bootmenu enable='yes' timeout='3000'/>
    <bios useserial='yes' rebootTimeout='3000'/>
  <on_reboot>restart</on_reboot>
      <boot order='1'/>

# cat t.xml|grep order
      <boot order='1'/>

# rpm -ql ipxe-roms|grep 808610ca.rom
/usr/share/ipxe/808610ca.rom

2. Try to define/create a guest with VF, failed with error message:
# virsh define t.xml
error: Failed to define domain from t.xml
error: unsupported configuration: boot order '1' used for more than one device

# virsh create t.xml
error: Failed to create domain from t.xml
error: unsupported configuration: boot order '1' used for more than one device

Actal results:
In step2: get error message

Expected results:
In step3: define/create guest successfully


Additional info:
- guest xml
- libvirtd log

Comment 2 chhu 2018-07-16 05:33:09 UTC
Created attachment 1459053 [details]
libvirtd log

Comment 3 yalzhang@redhat.com 2018-07-18 11:18:48 UTC
modify the xml to set boot order in hostdev type interface also report error as below:

# virsh edit yal
error: unsupported configuration: boot order '1' used for more than one device
Failed. Try again? [y,n,i,f,?]:

Comment 5 Laine Stump 2018-09-06 22:03:53 UTC
The problem is in virDomainDefCollectBootOrder, which was added in commit 5b75a4, first appearing libvirt-4.4.0, and is called for every device. The oddity is that <interface type='hostdev'> is on both the network device list *and* on the hostdev device list, so it gets checked twice, and the 2nd time is incorrectly seen as a conflict.

Comment 6 Laine Stump 2018-09-07 01:15:28 UTC
Fix posted upstream:

https://www.redhat.com/archives/libvir-list/2018-September/msg00226.html

Comment 7 Laine Stump 2018-09-12 15:26:25 UTC
Fix pushed upstream:

commit 7ea7342996d74591e00bcbf14b1eb3995f77a199
Author: Laine Stump <laine>
Date:   Thu Sep 6 21:09:45 2018 -0400

    conf: correct false boot order error during domain parse

Comment 11 chhu 2018-09-20 06:31:37 UTC
Test on packages:
libvirt-4.5.0-10.el7.x86_64
qemu-kvm-rhev-2.12.0-16.el7.x86_64


Test steps:
1. VF as hostdev interface + boot order: PASSED
1) Prepare guest with xml as below:
  <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.6.0'>hvm</type>
    <bootmenu enable='yes' timeout='3000'/>
    <bios useserial='yes' rebootTimeout='3000'/>
  </os>
......
    <interface type='hostdev' managed='yes'>
      <mac address='00:16:3e:77:e2:ed'/>
      <source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x10' function='0x2'/>
      </source>
      <boot order='1'/>
      <rom bar='on' file='/usr/share/ipxe/808610ed.rom'/>
    </interface>

2) Start the guest, and use virt-viewer to choose the iPXE to boot, it can get dhcp ip and boot from network.

3) Change the bootmenu enable='no', repeat 1)-2), no boot menu displayed and the guest boot from network automatically.

4) Delete the <bootmenu/> element, repeat 1)~2), no boot menu displayed and the guest boot from network automatically.

2. VF as hostdev interface + boot dev: PASSED
1) Prepare guest with xml as below:
  <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.6.0'>hvm</type>
    <boot dev='network'/>
    <bootmenu enable='yes' timeout='3000'/>
    <bios useserial='yes' rebootTimeout='3000'/>
  </os>
......
    <interface type='hostdev' managed='yes'>
      <mac address='00:16:3e:77:e2:ed'/>
      <source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x10' function='0x2'/>
      </source>
      <rom bar='on' file='/usr/share/ipxe/808610ed.rom'/>
    </interface>

2) Do steps as above 1)-4) in step1.

Comment 12 chhu 2018-09-20 06:33:55 UTC
3. PF passthrough as device  + boot order: PASSED
1) Prepare guest with xml as below:
  <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.6.0'>hvm</type>
    <bootmenu enable='yes' timeout='6000'/>
    <bios useserial='yes' rebootTimeout='3000'/>
  </os>
......
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <boot order='1'/>
      <rom bar='on' file='/usr/share/ipxe/808610fb.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>

2) Do steps as above 1)-4) in step1.

Comment 13 chhu 2018-09-25 03:22:21 UTC
Thanks jiyan verified this bug with RHV test scenarios.

Set the bug status to "VERIFIED", according to comment11-12 and the test with RHV scenarios.

Comment 15 errata-xmlrpc 2018-10-30 09:58:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3113