Bug 1262232
| Summary: | self announcement and ctrl offloads does not work after migration | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | jason wang <jasowang> | ||||||
| Component: | qemu-kvm-rhev | Assignee: | jason wang <jasowang> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 7.2 | CC: | ailan, huding, jasowang, juzhang, lmiksik, mrezanin, pezhang, qiguo, sherold, tlavigne, virt-maint, xfu, yvugenfi | ||||||
| Target Milestone: | rc | Keywords: | Regression | ||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | qemu-kvm-rhev-2.3.0-28.el7 | Doc Type: | Bug Fix | ||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2015-12-04 16:57:43 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Upstream posted at http://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03059.html Fix included in qemu-kvm-rhev-2.3.0-28.el7 Hi, Jason Test with unfixed and fixed build, got the following results: with unfixed build: qemu-kvm-rhev-2.3.0-24.el7.x86_64 After migration, from tcpdump the tap: 14:44:59.248334 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 14:44:59.281400 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) tell 54:52:01:2a:0b:01 (oui Unknown), length 46 14:44:59.431372 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) tell 54:52:01:2a:0b:01 (oui Unknown), length 46 14:44:59.474553 IP 192.168.1.11 > 192.168.1.1: ICMP echo request, id 3748, seq 31, length 64 14:44:59.474585 IP 192.168.1.1 > 192.168.1.11: ICMP echo reply, id 3748, seq 31, length 64 14:44:59.681413 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) tell 54:52:01:2a:0b:01 (oui Unknown), length 46 14:45:00.031383 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) tell 54:52:01:2a:0b:01 (oui Unknown), length 46 With fixed build qemu-kvm-rhev-2.3.0-29.el7.x86_64 After migration, got the packets: 14:20:58.807154 ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length 28 14:20:58.807173 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 14:20:58.833400 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) tell 54:52:01:2a:0b:01 (oui Unknown), length 46 14:20:58.856625 ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length 28 14:20:58.856655 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 14:20:58.983374 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) tell 54:52:01:2a:0b:01 (oui Unknown), length 46 14:20:59.006655 ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length 28 14:20:59.006674 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 14:20:59.233394 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) tell 54:52:01:2a:0b:01 (oui Unknown), length 46 14:20:59.256616 ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length 28 14:20:59.256639 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 14:20:59.583391 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) tell 54:52:01:2a:0b:01 (oui Unknown), length 46 14:20:59.606607 ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length 28 14:20:59.606626 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 From the result, with the fixed build, there're arp request "ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length 28" Is this the fixed expected result. Thanks, Qian (In reply to Qian Guo from comment #6) > Hi, Jason > > Test with unfixed and fixed build, got the following results: > > with unfixed build: > qemu-kvm-rhev-2.3.0-24.el7.x86_64 > > After migration, from tcpdump the tap: > > 14:44:59.248334 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor > advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 > 14:44:59.281400 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) > tell 54:52:01:2a:0b:01 (oui Unknown), length 46 > 14:44:59.431372 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) > tell 54:52:01:2a:0b:01 (oui Unknown), length 46 > 14:44:59.474553 IP 192.168.1.11 > 192.168.1.1: ICMP echo request, id 3748, > seq 31, length 64 > 14:44:59.474585 IP 192.168.1.1 > 192.168.1.11: ICMP echo reply, id 3748, seq > 31, length 64 > 14:44:59.681413 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) > tell 54:52:01:2a:0b:01 (oui Unknown), length 46 > 14:45:00.031383 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) > tell 54:52:01:2a:0b:01 (oui Unknown), length 46 > > > > > With fixed build qemu-kvm-rhev-2.3.0-29.el7.x86_64 > > After migration, got the packets: > > 14:20:58.807154 ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length > 28 > 14:20:58.807173 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor > advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 > 14:20:58.833400 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) > tell 54:52:01:2a:0b:01 (oui Unknown), length 46 > 14:20:58.856625 ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length > 28 > 14:20:58.856655 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor > advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 > 14:20:58.983374 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) > tell 54:52:01:2a:0b:01 (oui Unknown), length 46 > 14:20:59.006655 ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length > 28 > 14:20:59.006674 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor > advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 > 14:20:59.233394 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) > tell 54:52:01:2a:0b:01 (oui Unknown), length 46 > 14:20:59.256616 ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length > 28 > 14:20:59.256639 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor > advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 > 14:20:59.583391 ARP, Reverse Request who-is 54:52:01:2a:0b:01 (oui Unknown) > tell 54:52:01:2a:0b:01 (oui Unknown), length 46 > 14:20:59.606607 ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length > 28 > 14:20:59.606626 IP6 fe80::5652:1ff:fe2a:b01 > ff02::1: ICMP6, neighbor > advertisement, tgt is fe80::5652:1ff:fe2a:b01, length 32 > > > > From the result, with the fixed build, there're arp request > > "ARP, Request who-has 192.168.1.11 tell 192.168.1.11, length 28" > > Is this the fixed expected result. Yes. Thanks > > Thanks, > Qian (In reply to Qian Guo from comment #8) > According to comment 7, this bug is fixed. Talked with Jason, I will test the offloads part for windows guest, will update here once I got the result. Thanks, Qian Hi, Yan & Jaosn I followed Yan's suggestions, but I did not reproduce it with unfixed build(qemu-kvm-rhev-2.3.0-26.el7.x86_64). I disabled the GUEST_TSO in src qemu: -device virtio-net-pci,guest_tso4=off,guest_tso6=off,id=net0,netdev=hostnet0,mac=52:54:01:a2:0c:01 Then inside guest, disable/enable netkvm driver, then get followings from gdbview that GUST_TSOs are not loaded. ....... 00000079 5:18:42 AM [ParaNdis_InitializeContext] Message interrupt assigned 00000080 5:18:42 AM [ParaNdis_ResetVirtIONetDevice] Done 00000081 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_CSUM 00000082 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_GUEST_CSUM 00000083 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_MAC 00000084 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_GSO 00000085 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_GUEST_ECN 00000086 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_GUEST_UFO 00000087 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_HOST_TSO4 00000088 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_HOST_TSO6 00000089 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_HOST_ECN 00000090 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_HOST_UFO 00000091 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_MRG_RXBUF 00000092 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_STATUS 00000093 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_CTRL_VQ 00000094 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_CTRL_RX 00000095 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_CTRL_VLAN 00000096 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_CTRL_RX_EXTRA 00000097 5:18:42 AM VirtIO Host Feature VIRTIO_NET_F_CTRL_MAC_ADDR 00000098 5:18:42 AM VirtIO Host Feature VIRTIO_F_INDIRECT 00000099 5:18:42 AM VirtIO Host Feature VIRTIO_F_ANY_LAYOUT 00000100 5:18:42 AM VirtIO Host Feature VIRTIO_RING_F_EVENT_IDX 00000101 5:18:42 AM [ParaNdis_InitializeContext] Link status on driver startup: 1 00000102 5:18:42 AM Permanent device MAC: 52-54-01-a2-0c-01 00000103 5:18:42 AM No valid MAC configured 00000104 5:18:42 AM Actual MAC: 52-54-01-a2-0c-01 Then migration, from dst, disable/enable again, from the gdbview, the GUEST_TSOs are not loaded either. 00000079 5:20:15 AM [ParaNdis_InitializeContext] Message interrupt assigned 00000080 5:20:15 AM [ParaNdis_ResetVirtIONetDevice] Done 00000081 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_CSUM 00000082 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_GUEST_CSUM 00000083 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_MAC 00000084 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_GSO 00000085 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_GUEST_ECN 00000086 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_GUEST_UFO 00000087 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_HOST_TSO4 00000088 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_HOST_TSO6 00000089 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_HOST_ECN 00000090 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_HOST_UFO 00000091 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_MRG_RXBUF 00000092 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_STATUS 00000093 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_CTRL_VQ 00000094 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_CTRL_RX 00000095 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_CTRL_VLAN 00000096 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_CTRL_RX_EXTRA 00000097 5:20:15 AM VirtIO Host Feature VIRTIO_NET_F_CTRL_MAC_ADDR 00000098 5:20:15 AM VirtIO Host Feature VIRTIO_F_INDIRECT 00000099 5:20:15 AM VirtIO Host Feature VIRTIO_F_ANY_LAYOUT 00000100 5:20:15 AM VirtIO Host Feature VIRTIO_RING_F_EVENT_IDX 00000101 5:20:15 AM [ParaNdis_InitializeContext] Link status on driver startup: 1 00000102 5:20:15 AM Permanent device MAC: 52-54-01-a2-0c-01 00000103 5:20:15 AM No valid MAC configured 00000104 5:20:15 AM Actual MAC: 52-54-01-a2-0c-01 So, with unfixed build, the GUEST_TSOs works after before and after migration. Maybe I missed something, could you help check the log, I will attach them. Thanks, Qian Created attachment 1084600 [details]
The gdbview log that before the migration
Created attachment 1084601 [details]
The gdbview log that after the migration
Additional info: I also tested with qemu-kvm-rhev-2.3.0-31.el7.x86_64, same result as comment 10. Feel free to contact me if any further test. Thanks, Qian Tested with unfixed and fixed builds with following scenario: 1.enable the tso(by default, and I also enabled it in the cli). 2.disable LSO4/LSO6 inside windows guest, there's no switch to control the tso feature. 3.Test netperf inside guest as client and host as netserver *the throughput is 3032.04(10^6 bit/s) 4.migration, then test netperf again *the throughput is 3177.41(10^6 bit/s) Additionally, I also checked gdbview, no change before and after migration Hi, Jason Could you help have a check if we can set this bug as verified due to above tests and comments :) Thanks, Qian (In reply to Qian Guo from comment #14) > Tested with unfixed and fixed builds with following scenario: > > 1.enable the tso(by default, and I also enabled it in the cli). > 2.disable LSO4/LSO6 inside windows guest, there's no switch to control the > tso feature. > 3.Test netperf inside guest as client and host as netserver > *the throughput is 3032.04(10^6 bit/s) > 4.migration, then test netperf again > *the throughput is 3177.41(10^6 bit/s) > > Additionally, I also checked gdbview, no change before and after migration > > Hi, Jason > > Could you help have a check if we can set this bug as verified due to above > tests and comments :) > > Thanks, > Qian I think the bug could be verified first. Leave the windows guest control offload for future investigation. (In reply to jason wang from comment #15) > (In reply to Qian Guo from comment #14) > > Tested with unfixed and fixed builds with following scenario: > > > > 1.enable the tso(by default, and I also enabled it in the cli). > > 2.disable LSO4/LSO6 inside windows guest, there's no switch to control the > > tso feature. > > 3.Test netperf inside guest as client and host as netserver > > *the throughput is 3032.04(10^6 bit/s) > > 4.migration, then test netperf again > > *the throughput is 3177.41(10^6 bit/s) > > > > Additionally, I also checked gdbview, no change before and after migration > > > > Hi, Jason > > > > Could you help have a check if we can set this bug as verified due to above > > tests and comments :) > > > > Thanks, > > Qian > > I think the bug could be verified first. Leave the windows guest control > offload for future investigation. Got it. Hi Jason and Yan, Any further testing need to be tested with window guest later, free to let us know. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2546.html |
Description of problem: After commit 019a3edbb25f1571e876f8af1ce4c55412939e5d ("virtio: make features 64bit wide"). Device's guest_features was actually set after vdc->load(). This breaks the assumption that device specific load() function can check guest_features. For virtio-net, self announcement and guest offloads won't work after migration. Fixing this by defer them to virtio_net_load() where guest_features were guaranteed to be set. Other virtio devices looks fine Notes: qemu-kvm-rhev commit is : commit c9736b10ca3daefdb7284a9b39f8ec88afe7382e Author: Xiao Wang <jasowang> Date: Tue Jul 7 09:18:24 2015 +0200 virtio: make features 64bit wide Message-id: <1436260751-25015-22-git-send-email-jasowang> Patchwork-id: 66796 O-Subject: [RHEL7.2 qemu-kvm-rhev PATCH V2 21/68] virtio: make features 64bit wide Bugzilla: 1227343 RH-Acked-by: Michael S. Tsirkin <mst> RH-Acked-by: David Gibson <dgibson> RH-Acked-by: Laurent Vivier <lvivier> RH-Acked-by: Thomas Huth <thuth> Version-Release number of selected component (if applicable): How reproducible: 100% Steps to Reproduce: 1. tcpdump tap after migration 2. 3. Actual results: No GARP were found by tcpdump. Expected results: Guest should send GARP. Additional info: