This service will be undergoing maintenance at 00:00 UTC, 2016-09-28. It is expected to last about 1 hours
Bug 499678 - libvirt should be able to reset a PCI function even if it causes other unused devices/functions to be reset
libvirt should be able to reset a PCI function even if it causes other unused...
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: libvirt (Show other bugs)
11
All Linux
high Severity high
: ---
: ---
Assigned To: Mark McLoughlin
Fedora Extras Quality Assurance
:
Depends On:
Blocks: F11VirtTarget
  Show dependency treegraph
 
Reported: 2009-05-07 12:02 EDT by Charles Rose
Modified: 2009-09-04 00:09 EDT (History)
9 users (show)

See Also:
Fixed In Version: 0.6.2-17.fc11
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-09-04 00:09:45 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
lspci -tv (1.58 KB, text/plain)
2009-05-07 12:02 EDT, Charles Rose
no flags Details
lspci -vvv (33.17 KB, text/plain)
2009-05-07 12:50 EDT, Charles Rose
no flags Details
Syslog with virsh nodedev-reset (86.57 KB, text/plain)
2009-05-07 13:09 EDT, Charles Rose
no flags Details
Syslog with virt-manager PCI Assignment (4.49 KB, text/plain)
2009-05-07 13:10 EDT, Charles Rose
no flags Details
lspci -t -v output (3.40 KB, text/plain)
2009-05-08 02:20 EDT, Paniraj
no flags Details
lspci -vvv output (91.49 KB, text/plain)
2009-05-08 02:21 EDT, Paniraj
no flags Details

  None (edit)
Description Charles Rose 2009-05-07 12:02:29 EDT
Created attachment 342865 [details]
lspci -tv

While testing virt-manager's device assignment with the following steps:

  * Run virt-manager, open an existing guest and go to the details tab
  * Click "Add hardware", choose "Physical host device" and click "Forward"
  * Choose the appropriate device from the drop down list (This is the second LOM):
     "01:00.1 NetXtreme II BCM5716 Gigabit Ethernet"
  * Click "Forward" and "Finish"

and it failed with:

Uncaught error adding device: Could not detach PCI device: this function is not supported by the hypervisor: No PCI reset capability available for 0000:01:00.1

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/addhardware.py", line 571, in finish
    self.add_hostdev()
  File "/usr/share/virt-manager/virtManager/addhardware.py", line 607, in add_hostdev
    self._dev.setup()
  File "/usr/lib/python2.6/site-packages/virtinst/VirtualHostDevice.py", line 208, in setup
    raise RuntimeError(_("Could not detach PCI device: %s" % str(e)))
RuntimeError: Could not detach PCI device: this function is not supported by the hypervisor: No PCI reset capability available for 0000:01:00.1

I will test this with another network card.
Comment 1 Charles Rose 2009-05-07 12:03:27 EDT
I am using python-virtinst-0.400.3-8.fc11
Comment 2 Mark McLoughlin 2009-05-07 12:36:33 EDT
charles: could you attach the output of "lspci -vvv" and "lspi -t -v" ? Are there any libvirtd messages output to /var/log/messages at the time?

it looks like this is one function of a multi-function PCI device, and you're hitting this case:

/* Secondary Bus Reset is our sledgehammer - it resets all
 * devices behind a bus.
 */
static int
pciTrySecondaryBusReset(virConnectPtr conn, pciDevice *dev)
{
...
    /* For now, we just refuse to do a secondary bus reset                   
     * if there are other devices/functions behind the bus.
     * In future, we could allow it so long as those devices
     * are not in use by the host or other guests.
     */
    if (pciBusContainsOtherDevices(conn, dev)) {
	VIR_WARN("Other devices on bus with %s, not doing bus reset",
                 dev->name);
        return -1;
    }

or this case:

/* Power management reset attempts to reset a device using a
 * D-state transition from D3hot to D0. Note, in detect_pm_reset()
 * above we require the device supports a full internal reset.
 */
static int
pciTryPowerManagementReset(virConnectPtr conn, pciDevice *dev)
{
...
    /* For now, we just refuse to do a power management reset
     * if there are other functions on this device.
     * In future, we could allow it so long as those functions
     * are not in use by the host or other guests.
     */
    if (pciDeviceContainsOtherFunctions(conn, dev)) {
	VIR_WARN("%s contains other functions, not resetting", dev->name);
        return -1;
    }

We need to improve this code so that it will reset reset a function, even if the reset affects other functions/devices, so long as the device has been detached in the host and is not in use by any other guest
Comment 3 Mark McLoughlin 2009-05-07 12:45:56 EDT
clarifying ...

before we can reset:

  1) a function on a multi-function device, using PM reset
  2) a device sharing a bus with other devices, using a bus reset

we must check that the other affected functions/devices:

  a) are detached in the host - i.e. bound to pci-stub
  b) not in use by any other guest
  c) not in use by this guest, in the case of hotplug
Comment 4 Charles Rose 2009-05-07 12:50:42 EDT
Created attachment 342879 [details]
lspci -vvv
Comment 5 Charles Rose 2009-05-07 13:09:17 EDT
Created attachment 342887 [details]
Syslog with virsh nodedev-reset
Comment 6 Charles Rose 2009-05-07 13:10:57 EDT
Created attachment 342889 [details]
Syslog with virt-manager PCI Assignment
Comment 7 Mark McLoughlin 2009-05-07 13:17:19 EDT
Right, there we go - libvirt wants to do a Secondary Bus Reset, but it can't because it would also reset the other port on the broadcom NIC:

  warning : Other devices on bus with 0000:01:00.1, not doing bus reset

By the way, even if we fix libvirt as I describe above, you will not be able to use one port in the guest while the other port is in use by the host or another guest
Comment 8 Daniel Berrange 2009-05-07 13:26:34 EDT
Urgh, we really ought to get this kind of message back to the application

The error reported is:

"Could not detach PCI device: this function is not supported by
the hypervisor: No PCI reset capability available for 0000:01:00.1"

It should really have said something more like

"Could not detach PCI device: Other in-use devices on the same bus as device 0000:01:00.1, and FLR is not available"
Comment 9 Mark McLoughlin 2009-05-07 13:41:25 EDT
(In reply to comment #8)

> It should really have said something more like
> 
> "Could not detach PCI device: Other in-use devices on the same bus as device
> 0000:01:00.1, and FLR is not available"  

Agreed it should be better, all the possible reasons are:

  1) device doesn't implement FLR
  2) can't use bus reset because it affects other devices
  3) device doesn't implement PM reset
  4) can't use PM reset on multi-function device
  5) something went wrong doing FLR, SBR or PMR
Comment 10 Paniraj 2009-05-08 02:18:23 EDT
Additinal info to the issue:

I updated the packages again today. Added a Intel dual port NIC. Tried adding this add-on NIC to a guest. I got the same error as before.


Tried, resetting the NIC from virtual shell, I got the same error. (No PCI reset capability available for 0000:84:00.0)

Am attaching lspci outputs of my config.
Comment 11 Paniraj 2009-05-08 02:20:11 EDT
Created attachment 343051 [details]
lspci -t -v output
Comment 12 Paniraj 2009-05-08 02:21:10 EDT
Created attachment 343052 [details]
lspci -vvv output
Comment 13 Noura El hawary 2009-06-09 17:48:22 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 15 Fedora Update System 2009-08-19 12:34:52 EDT
libvirt-0.6.2-16.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/libvirt-0.6.2-16.fc11
Comment 16 Fedora Update System 2009-08-20 16:58:11 EDT
libvirt-0.6.2-17.fc11 has been pushed to the Fedora 11 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update libvirt'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-8790
Comment 17 Fedora Update System 2009-09-04 00:09:12 EDT
libvirt-0.6.2-17.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.