Bug 510679 - host pci device hot-unplug is not implemented
host pci device hot-unplug is not implemented
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: libvirt (Show other bugs)
All Linux
low Severity medium
: rc
: ---
Assigned To: Jiri Denemark
Virtualization Bugs
Depends On:
Blocks: 5.4/TechnicalNotes
  Show dependency treegraph
Reported: 2009-07-10 04:58 EDT by Shaohui
Modified: 2011-01-13 17:51 EST (History)
14 users (show)

See Also:
Fixed In Version: libvirt-0.8.2-1.el5
Doc Type: Bug Fix
Doc Text:
Hotunplug of PCI devices is not supported in this release. This feature will be introduced in a future update.
Story Points: ---
Clone Of:
Last Closed: 2011-01-13 17:51:46 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
log file from virt-manager (53.53 KB, text/plain)
2009-07-12 22:47 EDT, Shaohui
no flags Details
dumpxml from virsh (1.20 KB, text/plain)
2009-07-14 05:20 EDT, Shaohui
no flags Details
xend.log (1.46 KB, text/plain)
2009-07-14 11:35 EDT, Jiri Denemark
no flags Details

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2011:0060 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2011-01-12 12:22:30 EST

  None (edit)
Description Shaohui 2009-07-10 04:58:45 EDT
Description of problem:

We can assign NIC devices to guest statically, and we can see it in the
virt-manager control-panel. it lists all the assigned device in the guest.

virt-manager has a button to remove the assigned device, but after I click the
button, the device is still here. it does not remove the device in fact.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
Actual results:

Expected results:

Additional info:
Comment 1 Cole Robinson 2009-07-12 20:06:20 EDT
Thanks for the report. Can you please attach ~/.virt-manager/virt-manager.log?
Comment 2 Shaohui 2009-07-12 22:47:01 EDT
Created attachment 351418 [details]
log file from virt-manager
Comment 4 Cole Robinson 2009-07-13 09:32:11 EDT
Relevant portion:

[Mon, 13 Jul 2009 10:43:26 virt-manager 6706] DEBUG (domain:1111) Hostdev xpath string: /domain/devices/hostdev[@type='pci' and source/address/@bus='0x00' and source/address/@slot='0x19' and source/address/@function='0x0' and source/address/@domain='0x0000']
[Mon, 13 Jul 2009 10:43:26 virt-manager 6706] DEBUG (details:1767) Device could not be hotUNplugged: POST operation failed: xend_post: error from xen daemon: (xend.err "(22, 'Invalid argument')")

So this is likely a libvirt or xen bug. Reassigning to libvirt for now.

Shaohui, can you attach the XML of guest (virsh dumpxml <vmname>) with the host device attached? Thanks.
Comment 6 Shaohui 2009-07-14 05:20:14 EDT
Created attachment 351566 [details]
dumpxml from virsh

    <hostdev mode='subsystem' type='pci' managed='no'>
        <address domain='0x0000' bus='0x00' slot='0x19' function='0x0'/>

I see a suspected field managed='no'. it might be the root cause.
Comment 7 Daniel Berrange 2009-07-14 05:26:34 EDT
For the Libvirt Xen driver, PCI devices are always setup with managed='no', only QEMU allows for managed='yes'.

I'm thinking it looks like a XenD bug - could you provide the /var/log/xen/xend.log file from just after the failure to unplug. It should give us more info about the errro.
Comment 8 Jiri Denemark 2009-07-14 11:35:59 EDT
Created attachment 351605 [details]

I was able to reproduce it and attached is what shows in xend.log when hitting the remove button in virt-manager.
Comment 9 Jiri Denemark 2009-07-14 11:58:16 EDT
I investigated a bit further and xend gets a request with the following arguments:
{'type': ['`\xfc\x8d\t\xff\x7f'],
 'force': ['0'],
 'dev': ['\xd8B\xfe '],
 'rm_cfg': ['1'],
 'op': ['device_destroy']}

I guess the type should have been 'pci'.
Comment 10 Bill Burns 2009-07-15 13:54:51 EDT
Dan, any reply to Jirka's comment #9.
Comment 11 Daniel Berrange 2009-07-15 14:41:20 EDT
Hang on a minute. We never actually implemented PCI device hotplug/unplug for the libvirt Xen driver, so I'm surprised its even getting as far as XenD.
Comment 12 Daniel Berrange 2009-07-15 14:45:31 EDT
Oh I tell a lie, we partially implemented it. Hotplug should work, but hotunplug will not.

Unfortunately  we left some bogus code in virDomainXMLDevID() method, 

    } else if (dev->type == VIR_DOMAIN_DEVICE_HOSTDEV &&
               dev->data.hostdev->mode == VIR_DOMAIN_HOSTDEV_MODE_SUBSYS &&
               dev->data.hostdev->source.subsys.type == VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI) {
    } else {
        virXendError(NULL, VIR_ERR_NO_SUPPORT,
                     "%s", _("hotplug of device type not supported"));
        return -1;

    return 0;

so it'll return success, even though its not implemented. We should remove that test for PCI hostdevs here, so you'll get a proper error from libvirt, and then target a real implementation for RHEL-5.5
Comment 17 Daniel Berrange 2009-08-20 08:00:31 EDT
Release note added. If any revisions are required, please set the 
"requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

New Contents:
Hotunplug of PCI devices is not supported in this release. This feature will be introduced in a future update.
Comment 20 Daniel Veillard 2010-01-15 03:07:47 EST
Okay, at this point it's clear it's too late for 5.5, so retargetted for 5.6,

Comment 23 Xu Jiajun 2010-02-11 09:23:31 EST
Test with RHEL5.5 Beta, pci device hot-unplug can not work in Virt-Manager for both Xen and KVM.
Comment 24 Xu Jiajun 2010-02-11 12:09:05 EST
(In reply to comment #23)
> Test with RHEL5.5 Beta, pci device hot-unplug can not work in Virt-Manager for
> both Xen and KVM.    

Seems what I met is another issue while the assigned device is MFD behind a bridge. If the device is single device, the hot-unplug can work in RHEL5.5 Beta. So pls. skip my comments in comment 23. 

I will open another bug for the MFD hotplug issue.
Comment 25 Dave Allan 2010-07-06 15:18:57 EDT
Re: comment 24, are you saying that the problem is fixed?
Comment 27 Jiri Denemark 2010-09-02 07:57:01 EDT
Fixed in libvirt-0.8.2-1.el5
Comment 30 Jiri Denemark 2010-10-14 07:40:42 EDT
The comment #29 is irrelevant, this bug is for xen hypervisor, not kvm.
Comment 31 koka xiong 2010-10-21 00:49:53 EDT
Verified on Server-x86_64-Xen and Client-i386-Xen
ia64-Xen doesn't support modprobe acpiphp

1. # virsh nodedev-list --tree
2. # virsh nodedev-dumpxml pci_8086_10bd
3. Dettach this network device from host
 virsh nodedev-dettach pci_8086_10bd
4. Add the following code to <guest> xml file
<hostdev mode='subsystem' type='pci'>
<address bus='0' slot='25' function='0'/>
5.virsh start <guest>
6.run #modprobe acpiphp on guest # 
7.On virt-manager remove the device,device can be removed
Comment 33 errata-xmlrpc 2011-01-13 17:51:46 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.