Bug 499678 - libvirt should be able to reset a PCI function even if it causes other unused devices/functions to be reset
Summary: libvirt should be able to reset a PCI function even if it causes other unused...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: libvirt
Version: 11
Hardware: All
OS: Linux
high
high
Target Milestone: ---
Assignee: Mark McLoughlin
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: F11VirtTarget
TreeView+ depends on / blocked
 
Reported: 2009-05-07 16:02 UTC by Charles Rose
Modified: 2009-09-04 04:09 UTC (History)
9 users (show)

Fixed In Version: 0.6.2-17.fc11
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-09-04 04:09:45 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
lspci -tv (1.58 KB, text/plain)
2009-05-07 16:02 UTC, Charles Rose
no flags Details
lspci -vvv (33.17 KB, text/plain)
2009-05-07 16:50 UTC, Charles Rose
no flags Details
Syslog with virsh nodedev-reset (86.57 KB, text/plain)
2009-05-07 17:09 UTC, Charles Rose
no flags Details
Syslog with virt-manager PCI Assignment (4.49 KB, text/plain)
2009-05-07 17:10 UTC, Charles Rose
no flags Details
lspci -t -v output (3.40 KB, text/plain)
2009-05-08 06:20 UTC, Paniraj
no flags Details
lspci -vvv output (91.49 KB, text/plain)
2009-05-08 06:21 UTC, Paniraj
no flags Details

Description Charles Rose 2009-05-07 16:02:29 UTC
Created attachment 342865 [details]
lspci -tv

While testing virt-manager's device assignment with the following steps:

  * Run virt-manager, open an existing guest and go to the details tab
  * Click "Add hardware", choose "Physical host device" and click "Forward"
  * Choose the appropriate device from the drop down list (This is the second LOM):
     "01:00.1 NetXtreme II BCM5716 Gigabit Ethernet"
  * Click "Forward" and "Finish"

and it failed with:

Uncaught error adding device: Could not detach PCI device: this function is not supported by the hypervisor: No PCI reset capability available for 0000:01:00.1

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/addhardware.py", line 571, in finish
    self.add_hostdev()
  File "/usr/share/virt-manager/virtManager/addhardware.py", line 607, in add_hostdev
    self._dev.setup()
  File "/usr/lib/python2.6/site-packages/virtinst/VirtualHostDevice.py", line 208, in setup
    raise RuntimeError(_("Could not detach PCI device: %s" % str(e)))
RuntimeError: Could not detach PCI device: this function is not supported by the hypervisor: No PCI reset capability available for 0000:01:00.1

I will test this with another network card.

Comment 1 Charles Rose 2009-05-07 16:03:27 UTC
I am using python-virtinst-0.400.3-8.fc11

Comment 2 Mark McLoughlin 2009-05-07 16:36:33 UTC
charles: could you attach the output of "lspci -vvv" and "lspi -t -v" ? Are there any libvirtd messages output to /var/log/messages at the time?

it looks like this is one function of a multi-function PCI device, and you're hitting this case:

/* Secondary Bus Reset is our sledgehammer - it resets all
 * devices behind a bus.
 */
static int
pciTrySecondaryBusReset(virConnectPtr conn, pciDevice *dev)
{
...
    /* For now, we just refuse to do a secondary bus reset                   
     * if there are other devices/functions behind the bus.
     * In future, we could allow it so long as those devices
     * are not in use by the host or other guests.
     */
    if (pciBusContainsOtherDevices(conn, dev)) {
	VIR_WARN("Other devices on bus with %s, not doing bus reset",
                 dev->name);
        return -1;
    }

or this case:

/* Power management reset attempts to reset a device using a
 * D-state transition from D3hot to D0. Note, in detect_pm_reset()
 * above we require the device supports a full internal reset.
 */
static int
pciTryPowerManagementReset(virConnectPtr conn, pciDevice *dev)
{
...
    /* For now, we just refuse to do a power management reset
     * if there are other functions on this device.
     * In future, we could allow it so long as those functions
     * are not in use by the host or other guests.
     */
    if (pciDeviceContainsOtherFunctions(conn, dev)) {
	VIR_WARN("%s contains other functions, not resetting", dev->name);
        return -1;
    }

We need to improve this code so that it will reset reset a function, even if the reset affects other functions/devices, so long as the device has been detached in the host and is not in use by any other guest

Comment 3 Mark McLoughlin 2009-05-07 16:45:56 UTC
clarifying ...

before we can reset:

  1) a function on a multi-function device, using PM reset
  2) a device sharing a bus with other devices, using a bus reset

we must check that the other affected functions/devices:

  a) are detached in the host - i.e. bound to pci-stub
  b) not in use by any other guest
  c) not in use by this guest, in the case of hotplug

Comment 4 Charles Rose 2009-05-07 16:50:42 UTC
Created attachment 342879 [details]
lspci -vvv

Comment 5 Charles Rose 2009-05-07 17:09:17 UTC
Created attachment 342887 [details]
Syslog with virsh nodedev-reset

Comment 6 Charles Rose 2009-05-07 17:10:57 UTC
Created attachment 342889 [details]
Syslog with virt-manager PCI Assignment

Comment 7 Mark McLoughlin 2009-05-07 17:17:19 UTC
Right, there we go - libvirt wants to do a Secondary Bus Reset, but it can't because it would also reset the other port on the broadcom NIC:

  warning : Other devices on bus with 0000:01:00.1, not doing bus reset

By the way, even if we fix libvirt as I describe above, you will not be able to use one port in the guest while the other port is in use by the host or another guest

Comment 8 Daniel Berrangé 2009-05-07 17:26:34 UTC
Urgh, we really ought to get this kind of message back to the application

The error reported is:

"Could not detach PCI device: this function is not supported by
the hypervisor: No PCI reset capability available for 0000:01:00.1"

It should really have said something more like

"Could not detach PCI device: Other in-use devices on the same bus as device 0000:01:00.1, and FLR is not available"

Comment 9 Mark McLoughlin 2009-05-07 17:41:25 UTC
(In reply to comment #8)

> It should really have said something more like
> 
> "Could not detach PCI device: Other in-use devices on the same bus as device
> 0000:01:00.1, and FLR is not available"  

Agreed it should be better, all the possible reasons are:

  1) device doesn't implement FLR
  2) can't use bus reset because it affects other devices
  3) device doesn't implement PM reset
  4) can't use PM reset on multi-function device
  5) something went wrong doing FLR, SBR or PMR

Comment 10 Paniraj 2009-05-08 06:18:23 UTC
Additinal info to the issue:

I updated the packages again today. Added a Intel dual port NIC. Tried adding this add-on NIC to a guest. I got the same error as before.


Tried, resetting the NIC from virtual shell, I got the same error. (No PCI reset capability available for 0000:84:00.0)

Am attaching lspci outputs of my config.

Comment 11 Paniraj 2009-05-08 06:20:11 UTC
Created attachment 343051 [details]
lspci -t -v output

Comment 12 Paniraj 2009-05-08 06:21:10 UTC
Created attachment 343052 [details]
lspci -vvv output

Comment 13 Noura El hawary 2009-06-09 21:48:22 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 15 Fedora Update System 2009-08-19 16:34:52 UTC
libvirt-0.6.2-16.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/libvirt-0.6.2-16.fc11

Comment 16 Fedora Update System 2009-08-20 20:58:11 UTC
libvirt-0.6.2-17.fc11 has been pushed to the Fedora 11 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update libvirt'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-8790

Comment 17 Fedora Update System 2009-09-04 04:09:12 UTC
libvirt-0.6.2-17.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.