Bug 713697

Summary: Failed to load driver back after reattach node device
Product: Red Hat Enterprise Linux 6 Reporter: zhanghaiyan <yoyzhang>
Component: libvirtAssignee: Gunannan Ren <gren>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: urgent Docs Contact:
Priority: medium    
Version: 6.2CC: dallan, ddutile, dyuan, gren, jiachen, mshao, mzhan, rwu, zpeng
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.9.3-4.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 11:15:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description zhanghaiyan 2011-06-16 09:30:40 UTC
Description of problem:
Failed to load driver back after reattach node device

Version-Release number of selected component (if applicable):
libvirt-0.9.1-1.el6.x86_64
qemu-kvm-0.12.1.2-2.162.el6.x86_64
2.6.32-131.0.15.el6.x86_64

How reproducible:
6/6

Steps to Reproduce:
1. # virsh nodedev-list --tree
computer
 |
  +- net_lo_00_00_00_00_00_00
  +- net_virbr0_nic_52_54_00_db_c3_3f
  +- pci_0000_00_00_0
  +- pci_0000_00_01_0
  |   |
  |   +- pci_0000_01_00_0
  |   +- pci_0000_01_00_1
  |     
  +- pci_0000_00_03_0
  +- pci_0000_00_03_2
  +- pci_0000_00_03_3
  +- pci_0000_00_19_0
  |   |
  |   +- net_eth0_00_25_64_a7_1f_4d
......
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e

2.# virsh nodedev-dettach pci_0000_00_19_0
Device pci_0000_00_19_0 dettached
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_00_19_0
Device pci_0000_00_19_0 reset
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub

3. # virsh nodedev-reattach pci_0000_00_19_0
Device pci_0000_00_19_0 re-attached
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_00_19_0
Device pci_0000_00_19_0 reset
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub

4.# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:db:c3:3f brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:db:c3:3f brd ff:ff:ff:ff:ff:ff
# ifconfig
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:43 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3914 (3.8 KiB)  TX bytes:3914 (3.8 KiB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:DB:C3:3F  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:45 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:8244 (8.0 KiB)

  
Actual results:
In step3, after reattach&reset node device, pci-e1000e is not added back
In step4, eth0 is not back

Expected results:
In step3, after reattach&reset node device, pci-e1000e should be added back
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
In step4, eth0 is back

Additional info:
1. This bug cannot be reproduced on libvirt-0.8.7-18.el6.x86_64.rpm (the release version in rhel61), so this is a regression bug.
2. This bug cannot be reproduced through echo bind/unbind file, so it is not a kernel bug.
1)Before dettaching the node device
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
2) Detach the node device
# cd /sys/bus/pci/devices/0000\:00\:19.0/driver/
# ls
0000:00:19.0  bind  module  new_id  remove_id  uevent  unbind
# echo 0000\:00\:19.0 > /sys/bus/pci/devices/0000\:00\:19.0/driver/unbind 
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/devices/pci0000:00/0000:00:19.0/driver
3) Reattach the node device
# echo 0000\:00\:19.0 > /sys/bus/pci/drivers/e1000e/bind 
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:db:c3:3f brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:db:c3:3f brd ff:ff:ff:ff:ff:ff
6: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:25:64:a7:1f:4d brd ff:ff:ff:ff:ff:ff
# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:25:64:A7:1F:4D  
          inet addr:10.66.5.54  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::225:64ff:fea7:1f4d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1053 (1.0 KiB)  TX bytes:3505 (3.4 KiB)
          Interrupt:21 Memory:febe0000-fec00000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:960 (960.0 b)  TX bytes:960 (960.0 b)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:DB:C3:3F  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:3789 (3.7 KiB)

Comment 7 Min Zhan 2011-06-21 06:05:29 UTC
I have tried with 82576 card. In new libvirt pkg there is still the same problem as above. But in old libvirt pkg the bug does not exist. So I add Regression Keywords. Details pls refer to the following:

New Pkg:
libvirt-0.9.1-1.el6.x86_64
qemu-kvm-0.12.1.2-2.162.el6.x86_64
2.6.32-131.0.15.el6.x86_64

Steps:
1. # virsh nodedev-list --tree
...
  +- pci_0000_00_01_0
  |   |
  |   +- pci_0000_03_00_0
  |   |   |
  |   |   +- net_eth0_00_1b_21_39_8b_18
  |   |     
  |   +- pci_0000_03_00_1
  |       |
  |       +- net_eth1_00_1b_21_39_8b_19

...
# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:1B:21:39:8B:18  
          inet addr:10.66.4.218  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::21b:21ff:fe39:8b18/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7524 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1724 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2762743 (2.6 MiB)  TX bytes:205807 (200.9 KiB)
          Memory:f4800000-f4820000 

eth1      Link encap:Ethernet  HWaddr 00:1B:21:39:8B:19  
          inet addr:10.66.4.219  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::21b:21ff:fe39:8b19/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6317 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1355 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1566626 (1.4 MiB)  TX bytes:188221 (183.8 KiB)
          Memory:f4820000-f4840000

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb

2.# virsh nodedev-dettach pci_0000_03_00_1
Device pci_0000_03_00_1 dettached
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub

3. # virsh nodedev-reattach pci_0000_03_00_1
Device pci_0000_03_00_1 re-attached

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub

4. # ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:1B:21:39:8B:18  
          inet addr:10.66.4.218  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::21b:21ff:fe39:8b18/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7615 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1724 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2769516 (2.6 MiB)  TX bytes:205807 (200.9 KiB)
          Memory:f4800000-f4820000 

eth2      Link encap:Ethernet  HWaddr D8:D3:85:7E:61:9B  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:17 
...


# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:18 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether d8:d3:85:7e:61:9b brd ff:ff:ff:ff:ff:ff
5: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
8: sit0: <NOARP> mtu 1480 qdisc noop state DOWN 
    link/sit 0.0.0.0 brd 0.0.0.0


You can find 
- In step3, after reattach&reset node device, pci-igb is not added back
- In step4, eth1 is not back


-------------------
Old pkg:libvirt-0.8.7-18.el6.x86_64

Steps:
1. # readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb

2. # virsh nodedev-dettach pci_0000_03_00_1
Device pci_0000_03_00_1 dettached

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub

3. # virsh nodedev-reattach pci_0000_03_00_1
Device pci_0000_03_00_1 re-attached

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb

# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset

# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb

4. # ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:18 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether d8:d3:85:7e:61:9b brd ff:ff:ff:ff:ff:ff
5: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
8: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:19 brd ff:ff:ff:ff:ff:ff


For old libvirt pkg, you can find
- In step3, after reattach&reset node device, pci-igb is added back
- In step4, eth1 is back

Comment 8 Jason Chen 2011-06-21 09:32:48 UTC
I have tried with 82576 card. In new libvirt pkg there is still the same
problem as above.

1.New Pkgs:
libvirt-0.9.2-1.el6
kernel-2.6.32-156.el6
qemu-kvm-0.12.1.2-2.165.el6
netcf-0.1.8-1.el6

2.Steps & outputs:
[root@localhost ~]# virsh nodedev-list --tree
computer
 |
  +- net_lo_00_00_00_00_00_00
  +- net_virbr0_nic_52_54_00_b7_9a_4e
  +- pci_0000_00_00_0
  +- pci_0000_00_01_0
  |   |
  |   +- pci_0000_03_00_0
  |   |   |
  |   |   +- net_eth0_00_1b_21_39_8b_18
  |   |     
  |   +- pci_0000_03_00_1
  |       |
  |       +- net_eth1_00_1b_21_39_8b_19
  ...

[root@localhost ~]# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/igb
[root@localhost ~]# virsh nodedev-dettach pci_0000_03_00_0
Device pci_0000_03_00_0 dettached

[root@localhost ~]# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
[root@localhost ~]# virsh nodedev-reset pci_0000_03_00_0
Device pci_0000_03_00_0 reset

[root@localhost ~]# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
[root@localhost ~]# virsh nodedev-reattach pci_0000_03_00_0
Device pci_0000_03_00_0 re-attached

[root@localhost ~]# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
[root@localhost ~]# virsh nodedev-reset pci_0000_03_00_0
Device pci_0000_03_00_0 reset

[root@localhost ~]# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
[root@localhost ~]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:19 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether d8:d3:85:7e:61:9b brd ff:ff:ff:ff:ff:ff
5: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:b7:9a:4e brd ff:ff:ff:ff:ff:ff
[root@localhost ~]# ifconfig 
eth1      Link encap:Ethernet  HWaddr 00:1B:21:39:8B:19  
          inet addr:10.66.4.219  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::21b:21ff:fe39:8b19/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1494 errors:0 dropped:0 overruns:0 frame:0
          TX packets:187 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:130018 (126.9 KiB)  TX bytes:27218 (26.5 KiB)
          Memory:f4820000-f4840000 

eth2      Link encap:Ethernet  HWaddr D8:D3:85:7E:61:9B  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:17 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:40 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:4750 (4.6 KiB)  TX bytes:4750 (4.6 KiB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:B7:9A:4E  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:4397 (4.2 KiB)

Comment 9 Don Dutile (Red Hat) 2011-06-21 15:39:03 UTC
Laine: if 82576 fails now, that's a *big* regression....

reporter: virsh detach and virsh attach use libvirt's pci-managed feature/support.  it is unnecessary to do additional reset's bind, unbind, etc.

reporter: please re-test using just detach & attach, no additional commands.
I suspect that the multiple resets may be causing a problem...

Comment 10 Min Zhan 2011-06-22 07:34:01 UTC
According to comment 9, I retested without reset operation, but the bug still exists.
# rpm -q libvirt
libvirt-0.9.2-1.el6.x86_64
1. # readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/igb
2. # virsh nodedev-dettach pci_0000_03_00_0
Device pci_0000_03_00_0 dettached
# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
3. # virsh nodedev-reattach pci_0000_03_00_0
Device pci_0000_03_00_0 re-attached
# readlink /sys/bus/pci/devices/0000\:03\:00.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
4. Check #ip link or #ifconfig, both cannot see the node device.

Comment 11 Laine Stump 2011-06-22 07:41:38 UTC
I just verified this problem on a system with a tg3 ethernet. My test is to just run the following:

  virsh nodedeve-dettach pci_0000_09_00_0
  sleep 2
  virsh nodedeve-dettach pci_0000_09_00_0
  ifconfig eth1

With libvirt 0.8.7-18 I can run this multiple times, and each time the ethernet device reappears in the ifconfig output. When I install just libvirt-0.9.2-1 (changing no other packages), the same set of commands leads to:

  eth1: error fetching interface information: Device not found

So there definitely is a problem, and it seems not related to doing extra resets.

Comment 13 Don Dutile (Red Hat) 2011-06-22 13:00:34 UTC
Have you tried with rhel6.1's qemu-kvm:
    qemu-kvm-0.12.1.2-2.160.el6.x86_64

to ensure it's isolated to latest (in-devel rhel6.2) libvirt 
(libvirt-0.9.2-1.el6) ?

It looks like it is due to the libvirt rebase, so regression is the correct tag on this bz right now...

Comment 14 Laine Stump 2011-06-22 16:55:24 UTC
Don - I'm using qemu-kvm-0.12.1.2-2.160.el6.x86_64. The only package I change between the failure case and success case is libvirt.

Looks like a bisect is needed!

Comment 15 Dave Allan 2011-06-22 17:45:37 UTC
(In reply to comment #7)
> I have tried with 82576 card. In new libvirt pkg there is still the same
> problem as above. But in old libvirt pkg the bug does not exist. So I add
> Regression Keywords. Details pls refer to the following:

Fair enough, this is indeed a regression.  Thanks for the retest.

Comment 16 dyuan 2011-07-01 11:45:14 UTC
*** Bug 718193 has been marked as a duplicate of this bug. ***

Comment 17 dyuan 2011-07-01 11:46:43 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=718193#c1
“I proposed a patch into upstream and tried to fix it.
https://www.redhat.com/archives/libvir-list/2011-July/msg00013.html
Comment 18 Dave Allan 2011-07-07 20:23:51 UTC
Hi Guannan, thanks for your patch!  Since we are rebasing libvirt for 6.2, I've put the BZ in POST.  

commit 416814e66afa876dba2d384b87c22fbd08e08fcd
Author: Guannan Ren <gren>
Date:   Sun Jul 3 20:09:44 2011 +0800

    pci: initialize state values on reattach
    
    add a new API pciDeviceReAttachInit() in pci.c to initialize state values for nodedev reattach
    
    Initialize three state value of device driver to 1. This is just for a new call to
    qemudNodeDeviceReAttach()

Comment 19 Don Dutile (Red Hat) 2011-07-07 21:10:59 UTC
I hate to burst your bubble, but this is a problem in 6.1....
this bz should be marked as a dupe of 696877; i believe 719252 would be marked similarly.

So, unless you're backporting the rebase to 6.1, a patch is needed in 6.1 libvirt.

please read the last couple comments on latest test results in 696877;
if delay added to detach, before next attach, then it works.
also fails if guest is killed/destroyed vs. cleanly shutdown....
(that maybe 719252...)

Comment 20 Dave Allan 2011-07-08 02:44:37 UTC
(In reply to comment #19)
> I hate to burst your bubble, but this is a problem in 6.1....
> this bz should be marked as a dupe of 696877; i believe 719252 would be marked
> similarly.
> 
> So, unless you're backporting the rebase to 6.1, a patch is needed in 6.1
> libvirt.
> 
> please read the last couple comments on latest test results in 696877;
> if delay added to detach, before next attach, then it works.
> also fails if guest is killed/destroyed vs. cleanly shutdown....
> (that maybe 719252...)

Don, I'm not following you; this BZ is about the load/unload of the stub driver when a device is attached/detached.  It doesn't have anything to do with qemu behavior, and it's a regression caused by a post 6.1 libvirt commit.

Comment 21 Don Dutile (Red Hat) 2011-07-08 14:05:56 UTC
(In reply to comment #20)
> (In reply to comment #19)
> > I hate to burst your bubble, but this is a problem in 6.1....
> > this bz should be marked as a dupe of 696877; i believe 719252 would be marked
> > similarly.
> > 
> > So, unless you're backporting the rebase to 6.1, a patch is needed in 6.1
> > libvirt.
> > 
> > please read the last couple comments on latest test results in 696877;
> > if delay added to detach, before next attach, then it works.
> > also fails if guest is killed/destroyed vs. cleanly shutdown....
> > (that maybe 719252...)
> 
> Don, I'm not following you; this BZ is about the load/unload of the stub driver
> when a device is attached/detached.  It doesn't have anything to do with qemu
> behavior, and it's a regression caused by a post 6.1 libvirt commit.

my bad. I didn't read the whole history.  I saw a failure at detach .... 

it's not the same as bz's above.

Comment 22 zhe peng 2011-07-18 03:21:45 UTC
verify with:
libvirt-0.9.3-5.el6
qemu-kvm-0.12.1.2-2.169.el6

# virsh nodedev-list --tree
..............
 +- net_lo_00_00_00_00_00_00
  +- net_virbr0_nic_52_54_00_0c_28_97
  +- pci_0000_00_00_0
  +- pci_0000_00_01_0
--
  +- pci_0000_00_19_0
  |   |
  |   +- net_eth0_1c_6f_65_06_ba_42
  |     
  +- pci_0000_00_1a_0
.............
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
# virsh nodedev-dettach pci_0000_00_19_0
Device pci_0000_00_19_0 dettached
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_00_19_0
Device pci_0000_00_19_0 reset
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub

# virsh nodedev-reattach pci_0000_00_19_0
Device pci_0000_00_19_0 re-attached
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
# virsh nodedev-reset pci_0000_00_19_0
Device pci_0000_00_19_0 reset
# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e
#ip link show eth0
12: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 1c:6f:65:06:ba:42 brd ff:ff:ff:ff:ff:ff

#ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 1C:6F:65:06:BA:42  
          inet addr:10.66.6.166  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::1e6f:65ff:fe06:ba42/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1571 errors:0 dropped:0 overruns:0 frame:0
          TX packets:74 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:326182 (318.5 KiB)  TX bytes:9804 (9.5 KiB)
          Interrupt:16 Memory:fc300000-fc320000 


verify with 82576 card:
# lspci | grep 82576
03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
# virsh nodedev-list --tree
computer
........
+- pci_0000_00_01_0
  |   |
  |   +- pci_0000_03_00_0
  |   |   |
  |   |   +- net_eth0_00_1b_21_39_8b_18
  |   |     
  |   +- pci_0000_03_00_1
  |   |   |
  |   |   +- net_eth1_00_1b_21_39_8b_19
  |   |     
  |   +- pci_0000_03_10_0
  |   |   |
  |   |   +- net_eth3_be_bc_e4_85_89_0e
  |   |     
  |   +- pci_0000_03_10_1
  |   |   |
  |   |   +- net_eth4_3e_55_25_0c_7f_99
  |   |     
........
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb
# virsh nodedev-dettach pci_0000_03_00_1
Device pci_0000_03_00_1 dettached
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub
# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/pci-stub
# ifconfig eth1
eth1: error fetching interface information: Device not found

# virsh nodedev-reattach pci_0000_03_00_1
Device pci_0000_03_00_1 re-attached
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb
# virsh nodedev-reset pci_0000_03_00_1
Device pci_0000_03_00_1 reset
# readlink /sys/bus/pci/devices/0000\:03\:00.1/driver/ -f
/sys/bus/pci/drivers/igb
#ip link show eth1
31: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:19 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 2a:87:e6:a8:29:83
    vf 1 MAC 62:d1:c5:87:15:9c
    vf 2 MAC 0a:02:07:e2:93:c9
    vf 3 MAC 06:4b:19:7e:68:8a
    vf 4 MAC c6:90:20:be:7d:ab
    vf 5 MAC 4a:9f:71:ce:b9:a3
    vf 6 MAC 36:2b:36:40:d6:78
#ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:1B:21:39:8B:19  
          inet addr:10.66.4.219  Bcast:10.66.7.255  Mask:255.255.252.0
          inet6 addr: fe80::21b:21ff:fe39:8b19/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:274 errors:0 dropped:0 overruns:4 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:50423 (49.2 KiB)  TX bytes:2769 (2.7 KiB)
          Memory:f4820000-f4840000

Comment 24 Min Zhan 2011-07-18 04:35:15 UTC
Move to Verified according to Comment #22.

Comment 25 errata-xmlrpc 2011-12-06 11:15:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html