Bug 2111319
| Summary: | the network in win2016 guest can't work after failover vf migraion from BCM57810 and 82599ES network cards | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Yanhui Ma <yama> |
| Component: | virtio-win | Assignee: | ybendito |
| virtio-win sub component: | virtio-win-prewhql | QA Contact: | Yanhui Ma <yama> |
| Status: | CLOSED ERRATA | Docs Contact: | Jiri Herrmann <jherrman> |
| Severity: | medium | ||
| Priority: | medium | CC: | chayang, coli, jherrman, jinzhao, juzhang, lvivier, qizhu, virt-maint, vrozenfe, yalzhang, ybendito |
| Version: | 9.1 | Keywords: | Triaged |
| Target Milestone: | rc | Flags: | pm-rhel:
mirror+
|
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Windows | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Known Issue | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2023-11-07 08:29:32 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
I think this must be debugged on windows side first to understand what happens. Yuri, I think this BZ is for you too (like BZ2090712) Failover vf migration is only supported in RHV and it is technical preview. It is not supported in OSP and CNV. So set the priority to medium. If anything wrong, please correct me. Should be fixed in https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=53700760 It works well with following package versions and "-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off".
qemu-kvm-7.2.0-14.el9_2.x86_64
virtio-win driver:
100.93.104.23900
C:\Windows\system32>ipconfig /all
Windows IP Configuration
Host Name . . . . . . . . . . . . : WIN-U33AVC7B2CM
Primary Dns Suffix . . . . . . . :
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : lab.eng.pek2.redhat.com
Ethernet adapter Ethernet 3:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Red Hat VirtIO Ethernet Adapter #3
Physical Address. . . . . . . . . : 52-54-00-AA-1C-EF
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
Ethernet adapter Ethernet 2:
Connection-specific DNS Suffix . : lab.eng.pek2.redhat.com
Description . . . . . . . . . . . : Red Hat VirtIO Ethernet Adapter #2
Physical Address. . . . . . . . . : 52-54-00-01-16-16
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . : 2620:52:0:49d2:f5df:637a:2f63:4a2f(Preferred)
Link-local IPv6 Address . . . . . : fe80::f5df:637a:2f63:4a2f%6(Preferred)
IPv4 Address. . . . . . . . . . . : 10.73.211.168(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.254.0
Lease Obtained. . . . . . . . . . : Friday, July 21, 2023 4:37:45 AM
Lease Expires . . . . . . . . . . : Friday, July 21, 2023 11:36:30 PM
Default Gateway . . . . . . . . . : fe80::52c7:903:533b:88e1%6
10.73.211.254
DHCP Server . . . . . . . . . . . : 10.73.2.108
DHCPv6 IAID . . . . . . . . . . . : 122835968
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2C-47-5F-53-9A-E9-2D-4B-32-11
DNS Servers . . . . . . . . . . . : 10.72.17.5
10.68.5.26
NetBIOS over Tcpip. . . . . . . . : Enabled
Ethernet adapter Ethernet 7:
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Intel(R) 82599 Virtual Function
Physical Address. . . . . . . . . : 52-54-00-AA-1C-EF
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::393e:afea:c449:cccd%11(Preferred)
IPv4 Address. . . . . . . . . . . : 192.168.53.200(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Lease Obtained. . . . . . . . . . : Friday, July 21, 2023 11:39:03 AM
Lease Expires . . . . . . . . . . : Friday, July 21, 2023 11:49:03 AM
Default Gateway . . . . . . . . . : 192.168.53.2
DHCP Server . . . . . . . . . . . : 192.168.53.6
DHCPv6 IAID . . . . . . . . . . . : 324162560
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2C-47-5F-53-9A-E9-2D-4B-32-11
DNS Servers . . . . . . . . . . . : 192.168.53.2
NetBIOS over Tcpip. . . . . . . . : Enabled
Tunnel adapter isatap.{DD6493D0-1320-4D34-8038-6679E1D16381}:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft ISATAP Adapter
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Tunnel adapter Teredo Tunneling Pseudo-Interface:
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . : 2001:0:348b:fb58:143c:3a72:f5b6:2c57(Preferred)
Link-local IPv6 Address . . . . . : fe80::143c:3a72:f5b6:2c57%3(Preferred)
Default Gateway . . . . . . . . . :
DHCPv6 IAID . . . . . . . . . . . : 134217728
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2C-47-5F-53-9A-E9-2D-4B-32-11
NetBIOS over Tcpip. . . . . . . . : Disabled
Tunnel adapter isatap.lab.eng.pek2.redhat.com:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . : lab.eng.pek2.redhat.com
Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
C:\Windows\system32>ping 192.168.53.6
Pinging 192.168.53.6 with 32 bytes of data:
Reply from 192.168.53.6: bytes=32 time<1ms TTL=64
Reply from 192.168.53.6: bytes=32 time<1ms TTL=64
Reply from 192.168.53.6: bytes=32 time<1ms TTL=64
Reply from 192.168.53.6: bytes=32 time<1ms TTL=64
Ping statistics for 192.168.53.6:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
C:\Windows\system32>ping 192.168.53.101
Pinging 192.168.53.101 with 32 bytes of data:
Reply from 192.168.53.101: bytes=32 time<1ms TTL=64
Reply from 192.168.53.101: bytes=32 time<1ms TTL=64
Reply from 192.168.53.101: bytes=32 time<1ms TTL=64
Reply from 192.168.53.101: bytes=32 time<1ms TTL=64
Ping statistics for 192.168.53.101:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
C:\Windows\system32>
According to comment 5, first pre-verified the bug. Here is the new bug with 'ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'. Bug 2224964 - after windows guest failover vf migration with non-intel adapters, the network can't work, unless adding '-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off' Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virtio-win bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:6388 |
Description of problem: After live migrating win2016 guest with failover vf from BCM57810 and 82599ES network cards separately on src and dst hosts, ping dst host will fail in windows guest and the network info of Device manager is src netowrk card info, not the dst network card info. The guest is hung when rebooting the guest and scan the hardware changes via Device Manager. Version-Release number of selected component (if applicable): # rpm -q qemu-kvm qemu-kvm-7.0.0-9.el9.x86_64 [root@dell-per730-28 home]# uname -r 5.14.0-133.el9.x86_64 source nic info: NetXtreme II BCM57810 10 Gigabit Ethernet destination nic info: 82599ES 10-Gigabit SFI/SFP+ Network Connection How reproducible: 100% Steps to Reproduce: 1.create vf on both src host and dst host echo 1 > /sys/bus/pci/devices/0000\:1a\:00.1/sriov_numvfs 2. create failover-vf and failover-bridge network on both src and dst host # virsh net-dumpxml failover-bridge <network connections='1'> <name>failover-bridge</name> <uuid>1943a508-b0b7-4274-be5a-6f0143d10f40</uuid> <forward mode='bridge'/> <bridge name='br0'/> </network> # virsh net-dumpxml failover-vf <network connections='1'> <name>failover-vf</name> <uuid>4319b666-8f4b-410a-886f-17b6df772224</uuid> <forward mode='hostdev' managed='yes'> <address type='pci' domain='0x0000' bus='0x1a' slot='0x08' function='0x2'/> </forward> </network> 3. boot win2016 guest with failover vf on src host <interface type='network'> <mac address='52:54:00:aa:1c:ef'/> <source network='failover-bridge'/> <model type='virtio'/> <teaming type='persistent'/> <alias name='ua-test'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </interface> <interface type='network'> <mac address='52:54:00:aa:1c:ef'/> <source network='failover-vf'/> <teaming type='transient' persistent='ua-test'/> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </interface> 4. live migrating the guest 5. after migration, check the network in guest 6. reboot the guest or scan the hardware changes via Device manager in guest Actual results: after step 3, network info are correct, ping can succeed. after step 5, ping dst host ip succeeds, ping src host ip fails. The nic info in Device Manager is BCM57810, not 82599ES. See attachment. ipconfig /all Windows IP Configuration Host Name . . . . . . . . . . . . : WIN-6O8PO0LB52A Primary Dns Suffix . . . . . . . : Node Type . . . . . . . . . . . . : Hybrid IP Routing Enabled. . . . . . . . : No WINS Proxy Enabled. . . . . . . . : No DNS Suffix Search List. . . . . . : lab.eng.pek2.redhat.com Ethernet adapter Ethernet Instance 0 3: Connection-specific DNS Suffix . : lab.eng.pek2.redhat.com Description . . . . . . . . . . . : Red Hat VirtIO Ethernet Adapter #3 Physical Address. . . . . . . . . : 52-54-00-00-16-16 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes IPv6 Address. . . . . . . . . . . : 2620:52:0:49d2:b48b:fb85:cc13:d246(Preferred) Link-local IPv6 Address . . . . . : fe80::b48b:fb85:cc13:d246%7(Preferred) IPv4 Address. . . . . . . . . . . : 10.73.211.186(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.254.0 Lease Obtained. . . . . . . . . . : Wednesday, July 27, 2022 2:42:50 AM Lease Expires . . . . . . . . . . : Thursday, July 28, 2022 2:42:46 AM Default Gateway . . . . . . . . . : fe80::52c7:903:533b:88e1%7 10.73.211.254 DHCP Server . . . . . . . . . . . : 10.73.2.108 DHCPv6 IAID . . . . . . . . . . . : 122835968 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2A-6F-99-05-9A-7A-64-D7-19-B5 DNS Servers . . . . . . . . . . . : 10.73.2.107 10.73.2.108 10.66.127.10 NetBIOS over Tcpip. . . . . . . . : Enabled Ethernet adapter Ethernet Instance 0 4: Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Red Hat VirtIO Ethernet Adapter #4 Physical Address. . . . . . . . . : 52-54-00-AA-1C-EF DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::9528:5d3e:a846:ee90%2(Preferred) IPv4 Address. . . . . . . . . . . : 192.168.43.102(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Lease Obtained. . . . . . . . . . : Wednesday, July 27, 2022 2:42:42 AM Lease Expires . . . . . . . . . . : Wednesday, July 27, 2022 2:52:42 AM Default Gateway . . . . . . . . . : 192.168.43.2 DHCP Server . . . . . . . . . . . : 192.168.43.6 DHCPv6 IAID . . . . . . . . . . . : 38949888 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2A-6F-99-05-9A-7A-64-D7-19-B5 DNS Servers . . . . . . . . . . . : 192.168.43.2 NetBIOS over Tcpip. . . . . . . . : Enabled Tunnel adapter isatap.lab.eng.pek2.redhat.com: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : lab.eng.pek2.redhat.com Description . . . . . . . . . . . : Microsoft ISATAP Adapter Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Tunnel adapter isatap.{387B8C0A-C757-46F0-A319-4E4EE4163740}: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2 Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes C:\Windows\system32>ping 192.168.43.6 Pinging 192.168.43.6 with 32 bytes of data: Reply from 192.168.43.6: bytes=32 time<1ms TTL=64 Reply from 192.168.43.6: bytes=32 time<1ms TTL=64 Reply from 192.168.43.6: bytes=32 time<1ms TTL=64 Reply from 192.168.43.6: bytes=32 time<1ms TTL=64 Ping statistics for 192.168.43.6: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), C:\Windows\system32>ping 192.168.43.101 Pinging 192.168.43.101 with 32 bytes of data: Request timed out. Request timed out. Request timed out. Request timed out. Ping statistics for 192.168.43.101: Packets: Sent = 4, Received = 0, Lost = 4 (100% loss), C:\Windows\system32> after step6, guest is hung, see attachment. Expected results: Network info is 82599ES after migration. And ping both src and dst host ip can succeed. Additional info: After exchanging src and dst nic, network of guest works well after migration, no the issue.