RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2111319 - the network in win2016 guest can't work after failover vf migraion from BCM57810 and 82599ES network cards
Summary: the network in win2016 guest can't work after failover vf migraion from BCM57...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: virtio-win
Version: 9.1
Hardware: x86_64
OS: Windows
medium
medium
Target Milestone: rc
: ---
Assignee: ybendito
QA Contact: Yanhui Ma
Jiri Herrmann
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-07-27 03:24 UTC by Yanhui Ma
Modified: 2023-11-07 09:31 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-11-07 08:29:32 UTC
Type: ---
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-129177 0 None None None 2022-07-27 03:25:56 UTC
Red Hat Product Errata RHBA-2023:6388 0 None None None 2023-11-07 08:30:07 UTC

Description Yanhui Ma 2022-07-27 03:24:56 UTC 
Description of problem:

After live migrating win2016 guest with failover vf from BCM57810 and 82599ES network cards separately on src and dst hosts, ping dst host will fail in windows guest and the network info of Device manager is src netowrk card info, not the dst network card info. The guest is hung when rebooting the guest and scan the hardware changes via Device Manager.


Version-Release number of selected component (if applicable):

# rpm -q qemu-kvm
qemu-kvm-7.0.0-9.el9.x86_64
[root@dell-per730-28 home]# uname -r
5.14.0-133.el9.x86_64

source nic info:
NetXtreme II BCM57810 10 Gigabit Ethernet

destination nic info:
82599ES 10-Gigabit SFI/SFP+ Network Connection

How reproducible:

100%

Steps to Reproduce:
1.create vf on both src host and dst host

echo 1 > /sys/bus/pci/devices/0000\:1a\:00.1/sriov_numvfs

2. create failover-vf and failover-bridge network on both src and dst host

# virsh net-dumpxml failover-bridge 
<network connections='1'>
  <name>failover-bridge</name>
  <uuid>1943a508-b0b7-4274-be5a-6f0143d10f40</uuid>
  <forward mode='bridge'/>
  <bridge name='br0'/>
</network>

# virsh net-dumpxml failover-vf
<network connections='1'>
  <name>failover-vf</name>
  <uuid>4319b666-8f4b-410a-886f-17b6df772224</uuid>
  <forward mode='hostdev' managed='yes'>
    <address type='pci' domain='0x0000' bus='0x1a' slot='0x08' function='0x2'/>
  </forward>
</network>


3. boot win2016 guest with failover vf on src host
    <interface type='network'>
      <mac address='52:54:00:aa:1c:ef'/>
      <source network='failover-bridge'/>
      <model type='virtio'/>
      <teaming type='persistent'/>
      <alias name='ua-test'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </interface>
    <interface type='network'>
      <mac address='52:54:00:aa:1c:ef'/>
      <source network='failover-vf'/>
      <teaming type='transient' persistent='ua-test'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </interface>

4. live migrating the guest
5. after migration, check the network in guest
6. reboot the guest or scan the hardware changes via Device manager in guest


Actual results:

after step 3, network info are correct, ping can succeed.
after step 5, ping dst host ip succeeds, ping src host ip fails. The nic info in Device Manager is BCM57810, not 82599ES. See attachment.

ipconfig /all

Windows IP Configuration

   Host Name . . . . . . . . . . . . : WIN-6O8PO0LB52A
   Primary Dns Suffix  . . . . . . . : 
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : lab.eng.pek2.redhat.com

Ethernet adapter Ethernet Instance 0 3:

   Connection-specific DNS Suffix  . : lab.eng.pek2.redhat.com
   Description . . . . . . . . . . . : Red Hat VirtIO Ethernet Adapter #3
   Physical Address. . . . . . . . . : 52-54-00-00-16-16
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   IPv6 Address. . . . . . . . . . . : 2620:52:0:49d2:b48b:fb85:cc13:d246(Preferred) 
   Link-local IPv6 Address . . . . . : fe80::b48b:fb85:cc13:d246%7(Preferred) 
   IPv4 Address. . . . . . . . . . . : 10.73.211.186(Preferred) 
   Subnet Mask . . . . . . . . . . . : 255.255.254.0
   Lease Obtained. . . . . . . . . . : Wednesday, July 27, 2022 2:42:50 AM
   Lease Expires . . . . . . . . . . : Thursday, July 28, 2022 2:42:46 AM
   Default Gateway . . . . . . . . . : fe80::52c7:903:533b:88e1%7
                                       10.73.211.254
   DHCP Server . . . . . . . . . . . : 10.73.2.108
   DHCPv6 IAID . . . . . . . . . . . : 122835968
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2A-6F-99-05-9A-7A-64-D7-19-B5
   DNS Servers . . . . . . . . . . . : 10.73.2.107
                                       10.73.2.108
                                       10.66.127.10
   NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter Ethernet Instance 0 4:

   Connection-specific DNS Suffix  . : 
   Description . . . . . . . . . . . : Red Hat VirtIO Ethernet Adapter #4
   Physical Address. . . . . . . . . : 52-54-00-AA-1C-EF
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::9528:5d3e:a846:ee90%2(Preferred) 
   IPv4 Address. . . . . . . . . . . : 192.168.43.102(Preferred) 
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Lease Obtained. . . . . . . . . . : Wednesday, July 27, 2022 2:42:42 AM
   Lease Expires . . . . . . . . . . : Wednesday, July 27, 2022 2:52:42 AM
   Default Gateway . . . . . . . . . : 192.168.43.2
   DHCP Server . . . . . . . . . . . : 192.168.43.6
   DHCPv6 IAID . . . . . . . . . . . : 38949888
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2A-6F-99-05-9A-7A-64-D7-19-B5
   DNS Servers . . . . . . . . . . . : 192.168.43.2
   NetBIOS over Tcpip. . . . . . . . : Enabled

Tunnel adapter isatap.lab.eng.pek2.redhat.com:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : lab.eng.pek2.redhat.com
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter isatap.{387B8C0A-C757-46F0-A319-4E4EE4163740}:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : 
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

C:\Windows\system32>ping 192.168.43.6

Pinging 192.168.43.6 with 32 bytes of data:
Reply from 192.168.43.6: bytes=32 time<1ms TTL=64
Reply from 192.168.43.6: bytes=32 time<1ms TTL=64
Reply from 192.168.43.6: bytes=32 time<1ms TTL=64
Reply from 192.168.43.6: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.43.6:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),




C:\Windows\system32>ping 192.168.43.101

Pinging 192.168.43.101 with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 192.168.43.101:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Windows\system32>

after step6, guest is hung, see attachment.

Expected results:

Network info is 82599ES after migration. And ping both src and dst host ip can succeed. 

Additional info:

After exchanging src and dst nic, network of guest works well after migration, no the issue.
Comment 1 Laurent Vivier 2022-07-27 06:56:17 UTC 
I think this must be debugged on windows side first to understand what happens.
Comment 2 Laurent Vivier 2022-08-08 15:40:00 UTC 
Yuri, I think this BZ is for you too (like BZ2090712)
Comment 3 Yanhui Ma 2023-03-24 04:36:58 UTC 
Failover vf migration is only supported in RHV and it is technical preview. It is not supported in OSP and CNV. So set the priority to medium.
If anything wrong, please correct me.
Comment 4 ybendito 2023-07-13 09:52:38 UTC 
Should be fixed in https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=53700760
Comment 5 Yanhui Ma 2023-07-21 05:56:25 UTC 
It works well with following package versions and "-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off".
qemu-kvm-7.2.0-14.el9_2.x86_64
virtio-win driver:
100.93.104.23900


C:\Windows\system32>ipconfig /all

Windows IP Configuration

   Host Name . . . . . . . . . . . . : WIN-U33AVC7B2CM
   Primary Dns Suffix  . . . . . . . : 
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : lab.eng.pek2.redhat.com

Ethernet adapter Ethernet 3:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : 
   Description . . . . . . . . . . . : Red Hat VirtIO Ethernet Adapter #3
   Physical Address. . . . . . . . . : 52-54-00-AA-1C-EF
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes

Ethernet adapter Ethernet 2:

   Connection-specific DNS Suffix  . : lab.eng.pek2.redhat.com
   Description . . . . . . . . . . . : Red Hat VirtIO Ethernet Adapter #2
   Physical Address. . . . . . . . . : 52-54-00-01-16-16
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   IPv6 Address. . . . . . . . . . . : 2620:52:0:49d2:f5df:637a:2f63:4a2f(Preferred) 
   Link-local IPv6 Address . . . . . : fe80::f5df:637a:2f63:4a2f%6(Preferred) 
   IPv4 Address. . . . . . . . . . . : 10.73.211.168(Preferred) 
   Subnet Mask . . . . . . . . . . . : 255.255.254.0
   Lease Obtained. . . . . . . . . . : Friday, July 21, 2023 4:37:45 AM
   Lease Expires . . . . . . . . . . : Friday, July 21, 2023 11:36:30 PM
   Default Gateway . . . . . . . . . : fe80::52c7:903:533b:88e1%6
                                       10.73.211.254
   DHCP Server . . . . . . . . . . . : 10.73.2.108
   DHCPv6 IAID . . . . . . . . . . . : 122835968
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2C-47-5F-53-9A-E9-2D-4B-32-11
   DNS Servers . . . . . . . . . . . : 10.72.17.5
                                       10.68.5.26
   NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter Ethernet 7:

   Connection-specific DNS Suffix  . : 
   Description . . . . . . . . . . . : Intel(R) 82599 Virtual Function
   Physical Address. . . . . . . . . : 52-54-00-AA-1C-EF
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::393e:afea:c449:cccd%11(Preferred) 
   IPv4 Address. . . . . . . . . . . : 192.168.53.200(Preferred) 
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Lease Obtained. . . . . . . . . . : Friday, July 21, 2023 11:39:03 AM
   Lease Expires . . . . . . . . . . : Friday, July 21, 2023 11:49:03 AM
   Default Gateway . . . . . . . . . : 192.168.53.2
   DHCP Server . . . . . . . . . . . : 192.168.53.6
   DHCPv6 IAID . . . . . . . . . . . : 324162560
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2C-47-5F-53-9A-E9-2D-4B-32-11
   DNS Servers . . . . . . . . . . . : 192.168.53.2
   NetBIOS over Tcpip. . . . . . . . : Enabled

Tunnel adapter isatap.{DD6493D0-1320-4D34-8038-6679E1D16381}:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : 
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Teredo Tunneling Pseudo-Interface:

   Connection-specific DNS Suffix  . : 
   Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   IPv6 Address. . . . . . . . . . . : 2001:0:348b:fb58:143c:3a72:f5b6:2c57(Preferred) 
   Link-local IPv6 Address . . . . . : fe80::143c:3a72:f5b6:2c57%3(Preferred) 
   Default Gateway . . . . . . . . . : 
   DHCPv6 IAID . . . . . . . . . . . : 134217728
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2C-47-5F-53-9A-E9-2D-4B-32-11
   NetBIOS over Tcpip. . . . . . . . : Disabled

Tunnel adapter isatap.lab.eng.pek2.redhat.com:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : lab.eng.pek2.redhat.com
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

C:\Windows\system32>ping 192.168.53.6

Pinging 192.168.53.6 with 32 bytes of data:
Reply from 192.168.53.6: bytes=32 time<1ms TTL=64
Reply from 192.168.53.6: bytes=32 time<1ms TTL=64
Reply from 192.168.53.6: bytes=32 time<1ms TTL=64
Reply from 192.168.53.6: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.53.6:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\Windows\system32>ping 192.168.53.101

Pinging 192.168.53.101 with 32 bytes of data:
Reply from 192.168.53.101: bytes=32 time<1ms TTL=64
Reply from 192.168.53.101: bytes=32 time<1ms TTL=64
Reply from 192.168.53.101: bytes=32 time<1ms TTL=64
Reply from 192.168.53.101: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.53.101:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\Windows\system32>
Comment 6 Yanhui Ma 2023-07-24 05:34:05 UTC 
According to comment 5, first pre-verified the bug. Here is the new bug with 'ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'.
Bug 2224964 - after windows guest failover vf migration with non-intel adapters, the network can't work, unless adding '-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
Comment 7 Qianqian Zhu 2023-08-22 07:43:45 UTC 
Move to Verified per comment 6.
Comment 10 errata-xmlrpc 2023-11-07 08:29:32 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virtio-win bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:6388
Note You need to log in before you can comment on or make changes to this bug.