Bug 728203

Summary: PCI device can not be auto reassigned to host after detach with managed=yes
Product: Red Hat Enterprise Linux 6 Reporter: weizhang <weizhan>
Component: libvirtAssignee: Osier Yang <jyang>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: medium    
Version: 6.2CC: ajia, dallan, dyuan, ebenes, eblake, mvadkert, mzhan, rwu
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.9.4-6.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 06:23:34 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 584498, 846801, 846802    

Description weizhang 2011-08-04 07:25:19 EDT
Description of problem:
After attach and detach device with managed=yes, the pci device can not be auto reassigned to host

Version-Release number of selected component (if applicable):
kernel-2.6.32-177.el6.x86_64
libvirt-0.9.4-1.el6.x86_64
qemu-kvm-0.12.1.2-2.176.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1. prepare a guest and start it
2. # virsh nodedev-list --tree
  |    
  +- pci_0000_00_19_0
  |   |
  |   +- net_eth0_b8_ac_6f_3b_0b_bc

# virsh nodedev-dumpxml pci_0000_00_19_0
<device>
  <name>pci_0000_00_19_0</name>
  <parent>computer</parent>
  <driver>
    <name>pci-stub</name>
  </driver>
  <capability type='pci'>
    <domain>0</domain>
    <bus>0</bus>
    <slot>25</slot>
    <function>0</function>
    <product id='0x10de'>82567LM-3 Gigabit Network Connection</product>
    <vendor id='0x8086'>Intel Corporation</vendor>
  </capability>
</device>

# cat nodedev.xml
          <hostdev mode='subsystem' type='pci' managed='yes'>
            <source>
              <address bus='0' slot='25' function='0'/>
            </source>
          </hostdev>

# modprobe -r kvm_intel
# modprobe -r kvm
# modprobe kvm allow_unsafe_assigned_interrupts=1
# modprobe kvm_intel
# virsh attach-device guest nodedev.xml

3. virsh detach-device guest nodedev.xml
  
Actual results:
after step3, the eth0 can not be find on # ifconfig -a
and # service network restart  can not start network

Expected results:
the eth0 device can be auto reassigned

Additional info:
on version libvirt-0.8.7-18.el6.x86_64, the eth0 device can be auto reassigned after detach with managed=yes, so it is a regression bug
Comment 3 Osier Yang 2011-08-15 08:12:22 EDT
IMHO, "ifconfig -a" and "service network start" are not sufficient to check if the device is reassigned to host. As for a network device, there might be some other reasons which will cause "ifconfig -a" won't list the device, and "service network start" fail. For example, the network config file "ifcfg-eth0" can't be found.

The sufficient way in my opinion is checking if the device is still bound to the pci-stub driver. E.g.

# ls /sys/devices/pci0000:00/0000:00:00.0 -l

The symbol link "driver" will point to the current driver for the device.

And actually what libvirt does for "detach with managed" is unbind the device from pci-stub driver first, and bind it to the original driver then.

So if you find it's still bound to pci-stub, it's bug indeed, if it's bound to the original driver, it's not bug.
Comment 4 weizhang 2011-08-16 01:56:31 EDT
(In reply to comment #3)
> The sufficient way in my opinion is checking if the device is still bound to
> the pci-stub driver. E.g.
> 
> # ls /sys/devices/pci0000:00/0000:00:00.0 -l
> 
> The symbol link "driver" will point to the current driver for the device.
> 
> And actually what libvirt does for "detach with managed" is unbind the device
> from pci-stub driver first, and bind it to the original driver then.
> 
> So if you find it's still bound to pci-stub, it's bug indeed, if it's bound to
> the original driver, it's not bug.

I check after detach-device with "managed=yes" from guest and do 
# ls /sys/devices/pci0000\:00/0000\:00\:19.0 -l
find that the driver still bound to pci-stub
lrwxrwxrwx. 1 root root      0 Aug 16 02:04 driver -> ../../../bus/pci/drivers/pci-stub, so it's bug indeed.
Comment 5 Osier Yang 2011-08-17 09:24:35 EDT
commit 2ce90ea296eec5a16148fc8c7e2054ded60e1d32
Author: Osier Yang <jyang@redhat.com>
Date:   Wed Aug 17 20:58:33 2011 +0800

    qemu: Init reattaching related members of pciDevice before reattach
    
    Otherwise the device will still be bound to pci-stub driver even
    it's set as "managed=yes" when do detaching. Of course, it won't
    triger any driver reprobing too.

Patch pushed to upstream,
Comment 9 Osier Yang 2011-08-23 08:35:33 EDT
http://post-office.corp.redhat.com/archives/rhvirt-patches/2011-August/msg00568.html

Patch sent internally, move to MODIFIED.
Comment 10 Eric Blake 2011-08-23 09:57:58 EDT
POST not modified, until 0.9.4-6 is built.
Comment 12 weizhang 2011-08-28 23:31:22 EDT
verify pass on 
kernel-2.6.32-191.el6.x86_64
libvirt-0.9.4-6.el6.x86_64
qemu-kvm-0.12.1.2-2.184.el6.x86_64

after detach device from guest with "managed='yes'", the host can auto reattach the device again
Comment 13 errata-xmlrpc 2011-12-06 06:23:34 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html