Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 713697

Summary: Failed to load driver back after reattach node device
Product: Red Hat Enterprise Linux 6 Reporter: zhanghaiyan <yoyzhang>
Component: libvirtAssignee: Gunannan Ren <gren>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: urgent Docs Contact:
Priority: medium    
Version: 6.2CC: dallan, ddutile, dyuan, gren, jiachen, mshao, mzhan, rwu, zpeng
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.9.3-4.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 11:15:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description zhanghaiyan 2011-06-16 09:30:40 UTC 
Description of problem:
Failed to load driver back after reattach node device

Version-Release number of selected component (if applicable):
libvirt-0.9.1-1.el6.x86_64
qemu-kvm-0.12.1.2-2.162.el6.x86_64
2.6.32-131.0.15.el6.x86_64

How reproducible:
6/6

Steps to Reproduce:
1. # virsh nodedev-list --tree
computer
 |
  +- net_lo_00_00_00_00_00_00
  +- net_virbr0_nic_52_54_00_db_c3_3f
  +- pci_0000_00_00_0
  +- pci_0000_00_01_0
  |   |
  |   +- pci_0000_01_00_0
  |   +- pci_0000_01_00_1
  |     
  +- pci_0000_00_03_0
  +- pci_0000_00_03_2
  +- pci_0000_00_03_3
  +- pci_0000_00_19_0
  |   |
  |   +- net_eth0_00_25_64_a7_1f_4d
......
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e

2.# virsh nodedev-dettach pci_0000_00_19_0
Device pci_0000_00_19_0 dettached
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_00_19_0
Device pci_0000_00_19_0 reset
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub

3. # virsh nodedev-reattach pci_0000_00_19_0
Device pci_0000_00_19_0 re-attached
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_00_19_0
Device pci_0000_00_19_0 reset
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub

4.# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:db:c3:3f brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:db:c3:3f brd ff:ff:ff:ff:ff:ff
# ifconfig
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:43 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3914 (3.8 KiB)  TX bytes:3914 (3.8 KiB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:DB:C3:3F  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:45 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:8244 (8.0 KiB)

  
Actual results:
In step3, after reattach&reset node device, pci-e1000e is not added back
In step4, eth0 is not back

Expected results:
In step3, after reattach&reset node device, pci-e1000e should be added back
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
In step4, eth0 is back

Additional info:
1. This bug cannot be reproduced on libvirt-0.8.7-18.el6.x86_64.rpm (the release version in rhel61), so this is a regression bug.
2. This bug cannot be reproduced through echo bind/unbind file, so it is not a kernel bug.
1)Before dettaching the node device
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
2) Detach the node device
# cd /sys/bus/pci/devices/0000\:00\:19.0/driver/
# ls
0000:00:19.0  bind  module  new_id  remove_id  uevent  unbind
# echo 0000\:00\:19.0 > /sys/bus/pci/devices/0000\:00\:19.0/driver/unbind 
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/devices/pci0000:00/0000:00:19.0/driver
3) Reattach the node device
# echo 0000\:00\:19.0 > /sys/bus/pci/drivers/e1000e/bind 
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:db:c3:3f brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:db:c3:3f brd ff:ff:ff:ff:ff:ff
6: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:25:64:a7:1f:4d brd ff:ff:ff:ff:ff:ff
# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:25:64:A7:1F:4D  
          inet addr:10.66.5.54  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::225:64ff:fea7:1f4d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1053 (1.0 KiB)  TX bytes:3505 (3.4 KiB)
          Interrupt:21 Memory:febe0000-fec00000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:960 (960.0 b)  TX bytes:960 (960.0 b)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:DB:C3:3F  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:3789 (3.7 KiB)
Comment 7 Min Zhan 2011-06-21 06:05:29 UTC 
I have tried with 82576 card. In new libvirt pkg there is still the same problem as above. But in old libvirt pkg the bug does not exist. So I add Regression Keywords. Details pls refer to the following:

New Pkg:
libvirt-0.9.1-1.el6.x86_64
qemu-kvm-0.12.1.2-2.162.el6.x86_64
2.6.32-131.0.15.el6.x86_64

Steps:
1. # virsh nodedev-list --tree
...
  +- pci_0000_00_01_0
  |   |
  |   +- pci_0000_03_00_0
  |   |   |
  |   |   +- net_eth0_00_1b_21_39_8b_18
  |   |     
  |   +- pci_0000_03_00_1
  |       |
  |       +- net_eth1_00_1b_21_39_8b_19

...
# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:1B:21:39:8B:18  
          inet addr:10.66.4.218  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::21b:21ff:fe39:8b18/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7524 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1724 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2762743 (2.6 MiB)  TX bytes:205807 (200.9 KiB)
          Memory:f4800000-f4820000 

eth1      Link encap:Ethernet  HWaddr 00:1B:21:39:8B:19  
          inet addr:10.66.4.219  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::21b:21ff:fe39:8b19/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6317 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1355 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1566626 (1.4 MiB)  TX bytes:188221 (183.8 KiB)
          Memory:f4820000-f4840000

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb

2.# virsh nodedev-dettach pci_0000_03_00_1
Device pci_0000_03_00_1 dettached
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub

3. # virsh nodedev-reattach pci_0000_03_00_1
Device pci_0000_03_00_1 re-attached

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub

4. # ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:1B:21:39:8B:18  
          inet addr:10.66.4.218  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::21b:21ff:fe39:8b18/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7615 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1724 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2769516 (2.6 MiB)  TX bytes:205807 (200.9 KiB)
          Memory:f4800000-f4820000 

eth2      Link encap:Ethernet  HWaddr D8:D3:85:7E:61:9B  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:17 
...


# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:18 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether d8:d3:85:7e:61:9b brd ff:ff:ff:ff:ff:ff
5: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
8: sit0: <NOARP> mtu 1480 qdisc noop state DOWN 
    link/sit 0.0.0.0 brd 0.0.0.0


You can find 
- In step3, after reattach&reset node device, pci-igb is not added back
- In step4, eth1 is not back


-------------------
Old pkg:libvirt-0.8.7-18.el6.x86_64

Steps:
1. # readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb

2. # virsh nodedev-dettach pci_0000_03_00_1
Device pci_0000_03_00_1 dettached

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub

3. # virsh nodedev-reattach pci_0000_03_00_1
Device pci_0000_03_00_1 re-attached

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb

# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb

4. # ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:18 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether d8:d3:85:7e:61:9b brd ff:ff:ff:ff:ff:ff
5: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
8: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:19 brd ff:ff:ff:ff:ff:ff


For old libvirt pkg, you can find
- In step3, after reattach&reset node device, pci-igb is added back
- In step4, eth1 is back
Comment 8 Jason Chen 2011-06-21 09:32:48 UTC 
I have tried with 82576 card. In new libvirt pkg there is still the same
problem as above.

1.New Pkgs:
libvirt-0.9.2-1.el6
kernel-2.6.32-156.el6
qemu-kvm-0.12.1.2-2.165.el6
netcf-0.1.8-1.el6

2.Steps & outputs:
[root@localhost ~]# virsh nodedev-list --tree
computer
 |
  +- net_lo_00_00_00_00_00_00
  +- net_virbr0_nic_52_54_00_b7_9a_4e
  +- pci_0000_00_00_0
  +- pci_0000_00_01_0
  |   |
  |   +- pci_0000_03_00_0
  |   |   |
  |   |   +- net_eth0_00_1b_21_39_8b_18
  |   |     
  |   +- pci_0000_03_00_1
  |       |
  |       +- net_eth1_00_1b_21_39_8b_19
  ...

[root@localhost ~]# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/igb
[root@localhost ~]# virsh nodedev-dettach pci_0000_03_00_0
Device pci_0000_03_00_0 dettached

[root@localhost ~]# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
[root@localhost ~]# virsh nodedev-reset pci_0000_03_00_0
Device pci_0000_03_00_0 reset

[root@localhost ~]# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
[root@localhost ~]# virsh nodedev-reattach pci_0000_03_00_0
Device pci_0000_03_00_0 re-attached

[root@localhost ~]# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
[root@localhost ~]# virsh nodedev-reset pci_0000_03_00_0
Device pci_0000_03_00_0 reset

[root@localhost ~]# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
[root@localhost ~]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:19 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether d8:d3:85:7e:61:9b brd ff:ff:ff:ff:ff:ff
5: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
[root@localhost ~]# ifconfig 
eth1      Link encap:Ethernet  HWaddr 00:1B:21:39:8B:19  
          inet addr:10.66.4.219  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::21b:21ff:fe39:8b19/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1494 errors:0 dropped:0 overruns:0 frame:0
          TX packets:187 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:130018 (126.9 KiB)  TX bytes:27218 (26.5 KiB)
          Memory:f4820000-f4840000 

eth2      Link encap:Ethernet  HWaddr D8:D3:85:7E:61:9B  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:17 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:40 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:4750 (4.6 KiB)  TX bytes:4750 (4.6 KiB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:B7:9A:4E  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:4397 (4.2 KiB)
Comment 9 Don Dutile (Red Hat) 2011-06-21 15:39:03 UTC 
Laine: if 82576 fails now, that's a *big* regression....

reporter: virsh detach and virsh attach use libvirt's pci-managed feature/support.  it is unnecessary to do additional reset's bind, unbind, etc.

reporter: please re-test using just detach & attach, no additional commands.
I suspect that the multiple resets may be causing a problem...
Comment 10 Min Zhan 2011-06-22 07:34:01 UTC 
According to comment 9, I retested without reset operation, but the bug still exists.
# rpm -q libvirt
libvirt-0.9.2-1.el6.x86_64
1. # readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/igb
2. # virsh nodedev-dettach pci_0000_03_00_0
Device pci_0000_03_00_0 dettached
# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
3. # virsh nodedev-reattach pci_0000_03_00_0
Device pci_0000_03_00_0 re-attached
# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
4. Check #ip link or #ifconfig, both cannot see the node device.
Comment 11 Laine Stump 2011-06-22 07:41:38 UTC 
I just verified this problem on a system with a tg3 ethernet. My test is to just run the following:

  virsh nodedeve-dettach pci_0000_09_00_0
  sleep 2
  virsh nodedeve-dettach pci_0000_09_00_0
  ifconfig eth1

With libvirt 0.8.7-18 I can run this multiple times, and each time the ethernet device reappears in the ifconfig output. When I install just libvirt-0.9.2-1 (changing no other packages), the same set of commands leads to:

  eth1: error fetching interface information: Device not found

So there definitely is a problem, and it seems not related to doing extra resets.
Comment 13 Don Dutile (Red Hat) 2011-06-22 13:00:34 UTC 
Have you tried with rhel6.1's qemu-kvm:
    qemu-kvm-0.12.1.2-2.160.el6.x86_64

to ensure it's isolated to latest (in-devel rhel6.2) libvirt 
(libvirt-0.9.2-1.el6) ?

It looks like it is due to the libvirt rebase, so regression is the correct tag on this bz right now...
Comment 14 Laine Stump 2011-06-22 16:55:24 UTC 
Don - I'm using qemu-kvm-0.12.1.2-2.160.el6.x86_64. The only package I change between the failure case and success case is libvirt.

Looks like a bisect is needed!
Comment 15 Dave Allan 2011-06-22 17:45:37 UTC 
(In reply to comment #7)
> I have tried with 82576 card. In new libvirt pkg there is still the same
> problem as above. But in old libvirt pkg the bug does not exist. So I add
> Regression Keywords. Details pls refer to the following:

Fair enough, this is indeed a regression.  Thanks for the retest.
Comment 16 dyuan 2011-07-01 11:45:14 UTC 
*** Bug 718193 has been marked as a duplicate of this bug. ***
Comment 17 dyuan 2011-07-01 11:46:43 UTC 
https://bugzilla.redhat.com/show_bug.cgi?id=718193#c1
“I proposed a patch into upstream and tried to fix it.
https://www.redhat.com/archives/libvir-list/2011-July/msg00013.html”
Comment 18 Dave Allan 2011-07-07 20:23:51 UTC 
Hi Guannan, thanks for your patch!  Since we are rebasing libvirt for 6.2, I've put the BZ in POST.  

commit 416814e66afa876dba2d384b87c22fbd08e08fcd
Author: Guannan Ren <gren>
Date:   Sun Jul 3 20:09:44 2011 +0800

    pci: initialize state values on reattach
    
    add a new API pciDeviceReAttachInit() in pci.c to initialize state values for nodedev reattach
    
    Initialize three state value of device driver to 1. This is just for a new call to
    qemudNodeDeviceReAttach()
Comment 19 Don Dutile (Red Hat) 2011-07-07 21:10:59 UTC 
I hate to burst your bubble, but this is a problem in 6.1....
this bz should be marked as a dupe of 696877; i believe 719252 would be marked similarly.

So, unless you're backporting the rebase to 6.1, a patch is needed in 6.1 libvirt.

please read the last couple comments on latest test results in 696877;
if delay added to detach, before next attach, then it works.
also fails if guest is killed/destroyed vs. cleanly shutdown....
(that maybe 719252...)
Comment 20 Dave Allan 2011-07-08 02:44:37 UTC 
(In reply to comment #19)
> I hate to burst your bubble, but this is a problem in 6.1....
> this bz should be marked as a dupe of 696877; i believe 719252 would be marked
> similarly.
> 
> So, unless you're backporting the rebase to 6.1, a patch is needed in 6.1
> libvirt.
> 
> please read the last couple comments on latest test results in 696877;
> if delay added to detach, before next attach, then it works.
> also fails if guest is killed/destroyed vs. cleanly shutdown....
> (that maybe 719252...)

Don, I'm not following you; this BZ is about the load/unload of the stub driver when a device is attached/detached.  It doesn't have anything to do with qemu behavior, and it's a regression caused by a post 6.1 libvirt commit.
Comment 21 Don Dutile (Red Hat) 2011-07-08 14:05:56 UTC 
(In reply to comment #20)
> (In reply to comment #19)
> > I hate to burst your bubble, but this is a problem in 6.1....
> > this bz should be marked as a dupe of 696877; i believe 719252 would be marked
> > similarly.
> > 
> > So, unless you're backporting the rebase to 6.1, a patch is needed in 6.1
> > libvirt.
> > 
> > please read the last couple comments on latest test results in 696877;
> > if delay added to detach, before next attach, then it works.
> > also fails if guest is killed/destroyed vs. cleanly shutdown....
> > (that maybe 719252...)
> 
> Don, I'm not following you; this BZ is about the load/unload of the stub driver
> when a device is attached/detached.  It doesn't have anything to do with qemu
> behavior, and it's a regression caused by a post 6.1 libvirt commit.

my bad. I didn't read the whole history.  I saw a failure at detach .... 

it's not the same as bz's above.
Comment 22 zhe peng 2011-07-18 03:21:45 UTC 
verify with:
libvirt-0.9.3-5.el6
qemu-kvm-0.12.1.2-2.169.el6

# virsh nodedev-list --tree
..............
 +- net_lo_00_00_00_00_00_00
  +- net_virbr0_nic_52_54_00_0c_28_97
  +- pci_0000_00_00_0
  +- pci_0000_00_01_0
--
  +- pci_0000_00_19_0
  |   |
  |   +- net_eth0_1c_6f_65_06_ba_42
  |     
  +- pci_0000_00_1a_0
.............
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
# virsh nodedev-dettach pci_0000_00_19_0
Device pci_0000_00_19_0 dettached
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_00_19_0
Device pci_0000_00_19_0 reset
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub

# virsh nodedev-reattach pci_0000_00_19_0
Device pci_0000_00_19_0 re-attached
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
# virsh nodedev-reset pci_0000_00_19_0
Device pci_0000_00_19_0 reset
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
#ip link show eth0
12: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 1c:6f:65:06:ba:42 brd ff:ff:ff:ff:ff:ff

#ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 1C:6F:65:06:BA:42  
          inet addr:10.66.6.166  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::1e6f:65ff:fe06:ba42/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1571 errors:0 dropped:0 overruns:0 frame:0
          TX packets:74 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:326182 (318.5 KiB)  TX bytes:9804 (9.5 KiB)
          Interrupt:16 Memory:fc300000-fc320000 


verify with 82576 card:
# lspci | grep 82576
03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
# virsh nodedev-list --tree
computer
........
+- pci_0000_00_01_0
  |   |
  |   +- pci_0000_03_00_0
  |   |   |
  |   |   +- net_eth0_00_1b_21_39_8b_18
  |   |     
  |   +- pci_0000_03_00_1
  |   |   |
  |   |   +- net_eth1_00_1b_21_39_8b_19
  |   |     
  |   +- pci_0000_03_10_0
  |   |   |
  |   |   +- net_eth3_be_bc_e4_85_89_0e
  |   |     
  |   +- pci_0000_03_10_1
  |   |   |
  |   |   +- net_eth4_3e_55_25_0c_7f_99
  |   |     
........
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb
# virsh nodedev-dettach pci_0000_03_00_1
Device pci_0000_03_00_1 dettached
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub
# ifconfig eth1
eth1: error fetching interface information: Device not found

# virsh nodedev-reattach pci_0000_03_00_1
Device pci_0000_03_00_1 re-attached
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb
# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb
#ip link show eth1
31: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:19 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 2a:87:e6:a8:29:83
    vf 1 MAC 62:d1:c5:87:15:9c
    vf 2 MAC 0a:02:07:e2:93:c9
    vf 3 MAC 06:4b:19:7e:68:8a
    vf 4 MAC c6:90:20:be:7d:ab
    vf 5 MAC 4a:9f:71:ce:b9:a3
    vf 6 MAC 36:2b:36:40:d6:78
#ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:1B:21:39:8B:19  
          inet addr:10.66.4.219  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::21b:21ff:fe39:8b19/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:274 errors:0 dropped:0 overruns:4 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:50423 (49.2 KiB)  TX bytes:2769 (2.7 KiB)
          Memory:f4820000-f4840000
Comment 24 Min Zhan 2011-07-18 04:35:15 UTC 
Move to Verified according to Comment #22.
Comment 25 errata-xmlrpc 2011-12-06 11:15:15 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html