Bug 738095 - Need an xml switch to disable rombar of assigned devices
Summary: Need an xml switch to disable rombar of assigned devices
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.2
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Laine Stump
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-09-13 21:39 UTC by Don Dutile (Red Hat)
Modified: 2011-12-06 11:29 UTC (History)
9 users (show)

Fixed In Version: libvirt-0.9.4-14.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-12-06 11:29:02 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2011:1513 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2011-12-06 01:23:30 UTC

Description Don Dutile (Red Hat) 2011-09-13 21:39:24 UTC
Description of problem:
The default for -M rhel6.0.0 use to have rombar=0 set;
for rhel6.2.0 rombar is no longer set & it defaults to 1;
this causes guests with device-assigned 82574LMs to fail.

The only way to workaround such a problem,
is for the user to switch from virsh/libvirt-based guest startup
to direct qemu-kvm cmdline startup, adding rombar=0 to the host device
specification.

A libvirt-based method to disable PCI device rom would solve the regression
seen when the guest is updated from 6.0 to 6.2.


Version-Release number of selected component (if applicable):


How reproducible:
See this bz for reproducibility:
https://bugzilla.redhat.com/show_bug.cgi?id=735931

Steps to Reproduce:
1. boot guest with 82574-L(M) assigned to it:
2. guest fails to boot; fails when trying to configure 82574
3.
  
Actual results:


Expected results:
Boots successfully with assigned device w/mapped rom.

Additional info:
qemu-kvm has a command-line option that can turn off the rombar mapping of an assigned device:
   rombar=0

e.g., -device pci-assign,host=01:00.0,rombar=0,bus=pci.0,addr=0x7

If libvirt provided an xml schema to add that to an assigned device, e.g.,
<devices>
   <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
      </source>
   <rombar='0'/>   <===== new xml schema
   </hostdev>
</devices>

such that it would be added to the -device options after pci-assign & host-addr,
that would provide a workaround for new guest definitions that my cause device-assignment failures with respect to older guest definitions.

Comment 2 Laine Stump 2011-09-15 18:37:55 UTC
Gerd,

libvirt of course can't include this option on the commandline if the version of qemu being run doesn't support it. But the help output doesn't list whether or not it's there.


Did rombar show up in the same release as -device? Or was it later? Can you tell me the particular release? Maybe there was another feature that *is* reported in the help output that came up at the same time?

Comment 3 Gerd Hoffmann 2011-09-16 09:27:02 UTC
IIRC this is qemu 0.12 (same version which switches from bochs bios to seabios).

Comment 4 Laine Stump 2011-09-19 17:38:45 UTC
One concern I have is that with "<rom bar='on|off'/>", we're adding another element that can't really be re-used when we discover other PCI driver items that need to be configurable. For other drivers we put these in "<driver>" (eg event_idx), but in this case that doesn't really seem appropriate (correct me if I'm wrong) because this is a user-visible item that is at least semi-standard, while the things in <driver> have been implementation details of the driver.

Another question - what should the default be? If no rombar is specified, do we leave it out (which could lead to regressions like the current one if no extra action is taken)? Or do we always add the option, setting it to the "old" default of 0? I guess the question is really "how much regression is acceptable?"

Comment 5 Don Dutile (Red Hat) 2011-09-19 18:25:06 UTC
(In reply to comment #4)
> One concern I have is that with "<rom bar='on|off'/>", we're adding another
> element that can't really be re-used when we discover other PCI driver items
> that need to be configurable. For other drivers we put these in "<driver>" (eg
> event_idx), but in this case that doesn't really seem appropriate (correct me
> if I'm wrong) because this is a user-visible item that is at least
> semi-standard, while the things in <driver> have been implementation details of
> the driver.
> 
> Another question - what should the default be? If no rombar is specified, do we
> leave it out (which could lead to regressions like the current one if no extra
> action is taken)? Or do we always add the option, setting it to the "old"
> default of 0? I guess the question is really "how much regression is
> acceptable?"

First, you have 'needinfo' set but didn't identify which person you wanted the info from.... so here's my shot at it.

rombar is device-specific, in this case, it's a PCI bus property.

So, it shouldn't be associated with 'driver' (as in pci-assign, usb-host, etc.),
but with pci-assign only.

I also recommend not adding it at all to the command line if not requested to be set; let qemu defaults drive it.

Comment 6 Laine Stump 2011-09-21 16:35:26 UTC
(In reply to comment #5)

> First, you have 'needinfo' set but didn't identify which person you wanted the
> info from.... so here's my shot at it.

Right, because I thought it was something that more than one person might want to comment on, particularly the default setting, which is sometimes a contentious issue.

I've posted a patch upstream which implements this as discussed above (except that the xml is "<rom bar='0'/> (note the extra space):

https://www.redhat.com/archives/libvir-list/2011-September/msg00855.html

Currently the patch assumes presence of rombar if the qemu version is >= 0.12. If the patch for Bug 678731 were pushed, we could instead look in the output of "qemu-kvm -device pci-assign,?". I'm not sure if that would be a net gain or net loss.

Once the above patch is accepted/pushed upstream I will post it for inclusion in a RHEL build.

Comment 7 Don Dutile (Red Hat) 2011-09-21 17:27:48 UTC
patch for 678731 was posted & included upstream qeme.
Gerd posted it to rhvirt-patches list, and last time I checked, 
Markus & I ack'd it;
so we need another ack to get into 6.2-qemu-kvm; ok sent a request;
alex williamson will do that.

So, it's in upstream qemu; don't know how/when avi pulls it into qemu-kvm upstream; in queue for rhel6.2.

is that good enough for libvirt to change it based on -device pci-assign,? probe?

Comment 8 Laine Stump 2011-09-27 16:42:22 UTC
That would be good for a controlled environment like RHEL, but we decided that it would be more accurate for the upstream patch to use the qemu version, and that it's best for the RHEL patch to match the upstream patch. So in the end it just checks version anyway.

The upstream patch has been pushed, and was rebased and sent to rhvirt-patches for inclusion in RHEL:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2011-September/msg00882.html

commit dc79852af88a6ee8b03f8fd7e48e039bae056ed3
Author: Laine Stump <laine@laine.org>
Date:   Tue Sep 20 13:31:52 2011 -0400

    qemu: add ability to set PCI device "rombar" on or off
    
    This patch was made in response to:
    
      https://bugzilla.redhat.com/show_bug.cgi?id=738095
    
    In short, qemu's default for the rombar setting (which makes the
    firmware ROM of a PCI device visible/not on the guest) was previously
    0 (not visible), but they recently changed the default to 1
    (visible). Unfortunately, there are some PCI devices that fail in the
    guest when rombar is 1, so the setting must be exposed in libvirt to
    prevent a regression in behavior (it will still require explicitly
    setting <rom bar='off'/> in the guest XML).
    
    rombar is forced on/off by adding:
    
      <rom bar='on|off'/>
    
    inside a <hostdev> element that defines a PCI device. It is currently
    ignored for all other types of devices.

Comment 11 Alex Jia 2011-09-29 08:47:11 UTC
This issue has been resolved on rhel6.2 Beta(2.6.32-202.el6.x86_64) with libvirt-0.9.4-14.el6.x86_64 and qemu-kvm-0.12.1.2-2.183.el6.x86_64, so move the bug to VERIFIED status, the following is test report for rom bar='on|off':

# virsh dumpxml vr-rhel5u4-x86_64-kvm
......
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x19' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
      <rom bar='off'/>
    </hostdev>
......

# ps -ef|grep qemu-kvm|grep -v grep
qemu      3950     1 58 16:25 ?        00:00:09 /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name vr-rhel5u4-x86_64-kvm -uuid 894d5b0f-42c3-c13d-ee81-f55a5313cecc -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vr-rhel5u4-x86_64-kvm.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/var/lib/libvirt/images/vr-rhel5u4-x86_64-kvm,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,fd=23,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:7b:91:a4,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -k en-us -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device pci-assign,host=00:19.0,id=hostdev0,configfd=24,bus=pci.0,addr=0x6,rombar=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

# virsh dumpxml vr-rhel5u4-x86_64-kvm
......
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x19' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
      <rom bar='on'/>
    </hostdev>
......


# ps -ef|grep qemu-kvm|grep -v grep
qemu      4258     1 21 16:32 ?        00:00:01 /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name vr-rhel5u4-x86_64-kvm -uuid 894d5b0f-42c3-c13d-ee81-f55a5313cecc -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vr-rhel5u4-x86_64-kvm.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/var/lib/libvirt/images/vr-rhel5u4-x86_64-kvm,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,fd=23,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:7b:91:a4,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -k en-us -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device pci-assign,host=00:19.0,id=hostdev0,configfd=24,bus=pci.0,addr=0x6,rombar=1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

Comment 12 errata-xmlrpc 2011-12-06 11:29:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html


Note You need to log in before you can comment on or make changes to this bug.