Bug 1431561
| Summary: | [NetKVM] Implement VIRTIO_NET_F_GUEST_ANNOUNCE feature in Windows virtio-net driver - IPv6 | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Yvugenfi <yvugenfi> |
| Component: | virtio-win | Assignee: | Yvugenfi <yvugenfi> |
| virtio-win sub component: | virtio-win-prewhql | QA Contact: | Virtualization Bugs <virt-bugs> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | CC: | ailan, jherrman, lijin, lmiksik, michen, mtessun, phou, salmy, wyu, xiagao, ybendito, yvugenfi |
| Version: | 7.4 | Keywords: | RFE |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Windows | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Enhancement | |
| Doc Text: |
The virtio-win driver now supports the VIRTIO_NET_F_GUEST_ANNOUNCE feature, which, if configured, ensures that ARP packets are recognized by the network when performing live migration of Windows guests. As a result, packet loss during the migration is significantly decreased.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-08-01 12:58:08 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1411764 | ||
| Bug Blocks: | |||
|
Description
Yvugenfi@redhat.com
2017-03-13 09:44:56 UTC
Hi Yan, Could you suggest a method for QE to verify this bug? We tried to check the timeout counts after migration,but we didn't see big difference between 126 and 135,there was only 1 timeout found. (In reply to lijin from comment #2) > Hi Yan, > > Could you suggest a method for QE to verify this bug? > > We tried to check the timeout counts after migration,but we didn't see big > difference between 126 and 135,there was only 1 timeout found. I think the best way is to see the packet goes out of VM on tap or bridge. (In reply to Yan Vugenfirer from comment #3) > (In reply to lijin from comment #2) > > Hi Yan, > > > > Could you suggest a method for QE to verify this bug? > > > > We tried to check the timeout counts after migration,but we didn't see big > > difference between 126 and 135,there was only 1 timeout found. > > I think the best way is to see the packet goes out of VM on tap or bridge. I use wireshark to capture the packages in guests. I see following packages on both build 126 and 135(not every time,just randomly): 193 27.041628 00:52:4c:20:8d:44 Broadcast ARP 60 Gratuitous ARP for 10.66.10.63 (Request) 158 27.041565 00:52:4c:20:8d:44 Broadcast ARP 60 Gratuitous ARP for 169.254.168.92 (Request) 255 48.662695 :: ff02::1:fffb:aa39 ICMPv6 78 Neighbor Solicitation 268 50.250055 :: ff02::1:ff49:b25f ICMPv6 78 Neighbor Solicitation Yan, Could you help to check if my steps are correct? Thanks Hi, You will not be able to see those packets in the guest. The packets are generated in the driver itself. (In reply to Yan Vugenfirer from comment #5) > Hi, > > You will not be able to see those packets in the guest. The packets are > generated in the driver itself. So we should capture the packages of tap/bridge on the host? with tcpdump command?catch which kind of package message? I tried twice,I did not see any Gratuitous ARP nor Neighbor Solicitation. Could you help to provide a way more in details? Thanks a lot Hi Yuri, Can you please advise Lijin? Thanks. Packets shall be captured on the bridge, for example (for ipv4): "tcpdump -nntttt -i br0 arp or icmp" to see both ping and icmp Such a way I was able to see all arps including gratuitous. (In reply to ybendito from comment #8) > Packets shall be captured on the bridge, for example (for ipv4): > "tcpdump -nntttt -i br0 arp or icmp" to see both ping and icmp > Such a way I was able to see all arps including gratuitous. Similar for ipv6: "tcpdump -nntttt -i br0 icmp6" should do the job For ipv4: with build 135,I see following ARP packages after migration(there is no such package with build126): 2017-04-05 17:59:14.332539 ARP, Request who-has 10.66.10.226 tell 10.66.10.226, length 28 2017-04-05 17:59:14.382404 ARP, Request who-has 10.66.10.226 tell 10.66.10.226, length 28 2017-04-05 17:59:14.532398 ARP, Request who-has 10.66.10.226 tell 10.66.10.226, length 28 2017-04-05 17:59:14.782397 ARP, Request who-has 10.66.10.226 tell 10.66.10.226, length 28 2017-04-05 17:59:15.132381 ARP, Request who-has 10.66.10.226 tell 10.66.10.226, length 28 10.66.10.226 is the guest ipv4 address For ipv6: I can NOT see any neighbor solicitation package after migration with both 126 and 135,the neighbor solicitation only can be seen when src guest boot up: 2017-04-05 17:57:20.281457 IP6 :: > ff02::1:ffda:d7d2: ICMP6, neighbor solicitation, who has fe80::8c2:341e:1ada:d7d2, length 24 2017-04-05 17:57:20.281512 IP6 fe80::8c2:341e:1ada:d7d2 > ff02::2: ICMP6, router solicitation, length 8 2017-04-05 17:57:21.281486 IP6 fe80::8c2:341e:1ada:d7d2 > ff02::1: ICMP6, neighbor advertisement, tgt is fe80::8c2:341e:1ada:d7d2, length 32 2017-04-05 17:57:24.282294 IP6 fe80::8c2:341e:1ada:d7d2 > ff02::2: ICMP6, router solicitation, length 16 2017-04-05 17:57:28.281490 IP6 fe80::8c2:341e:1ada:d7d2 > ff02::2: ICMP6, router solicitation, length 16 fe80::8c2:341e:1ada:d7d2 is guest ipv6 address Hi Yuri, Could you check the result above,seems I did not get the expected ipv6 message? Thanks. (In reply to lijin from comment #10) > For ipv4: > with build 135,I see following ARP packages after migration(there is no such > package with build126): > 2017-04-05 17:59:14.332539 ARP, Request who-has 10.66.10.226 tell > 10.66.10.226, length 28 > 2017-04-05 17:59:14.382404 ARP, Request who-has 10.66.10.226 tell > 10.66.10.226, length 28 > 2017-04-05 17:59:14.532398 ARP, Request who-has 10.66.10.226 tell > 10.66.10.226, length 28 > 2017-04-05 17:59:14.782397 ARP, Request who-has 10.66.10.226 tell > 10.66.10.226, length 28 > 2017-04-05 17:59:15.132381 ARP, Request who-has 10.66.10.226 tell > 10.66.10.226, length 28 > 10.66.10.226 is the guest ipv4 address > > For ipv6: > I can NOT see any neighbor solicitation package after migration with both > 126 and 135,the neighbor solicitation only can be seen when src guest boot > up: > 2017-04-05 17:57:20.281457 IP6 :: > ff02::1:ffda:d7d2: ICMP6, neighbor > solicitation, who has fe80::8c2:341e:1ada:d7d2, length 24 > 2017-04-05 17:57:20.281512 IP6 fe80::8c2:341e:1ada:d7d2 > ff02::2: ICMP6, > router solicitation, length 8 > 2017-04-05 17:57:21.281486 IP6 fe80::8c2:341e:1ada:d7d2 > ff02::1: ICMP6, > neighbor advertisement, tgt is fe80::8c2:341e:1ada:d7d2, length 32 > 2017-04-05 17:57:24.282294 IP6 fe80::8c2:341e:1ada:d7d2 > ff02::2: ICMP6, > router solicitation, length 16 > 2017-04-05 17:57:28.281490 IP6 fe80::8c2:341e:1ada:d7d2 > ff02::2: ICMP6, > router solicitation, length 16 > fe80::8c2:341e:1ada:d7d2 is guest ipv6 address > > > Hi Yuri, > Could you check the result above,seems I did not get the expected ipv6 > message? > Thanks. We're checking it with 135 We checked it with 135 in our lab: sameeh@bark:~$ sudo /sbin/tcpdump -nli br0 icmp6 -w packets.pcap tcpdump: listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes ^C38 packets captured 40 packets received by filter 0 packets dropped by kernel NSM packets clearly seen in the record at migration time, source IP address = 0, desc IP address = address of adapter I still can not find the NSM packets,could you help to check if my steps are correct? 1.boot src guest on src host: /usr/libexec/qemu-kvm -name 122BLKWIN2016 -enable-kvm -m 2G -smp 2 -nodefconfig -nodefaults -rtc base=localtime,driftfix=slew -boot order=cd,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=win2012R2-lijin.raw,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,cache=none -device ide-drive,drive=drive-ide0-0-0,id=ide0-0-0 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0 -device usb-tablet,id=input0 -cdrom virtio-win-prewhql-126.iso -monitor stdio -qmp tcp:0:4449,server,nowait -fda virtio-win-prewhql-126.vfd -vnc 0.0.0.0:21 -vga std -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0,vhost=on,queues=4 -device virtio-net-pci,mq=on,vectors=10,netdev=hostnet0,id=net0,mac=00:52:4c:20:8d:88 2.boot dst guest on dst host: /usr/libexec/qemu-kvm -name 122BLKWIN2016 -enable-kvm -m 2G -smp 2 -nodefconfig -nodefaults -rtc base=localtime,driftfix=slew -boot order=cd,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=win2012R2-lijin.raw,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,cache=none -device ide-drive,drive=drive-ide0-0-0,id=ide0-0-0 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0 -device usb-tablet,id=input0 -cdrom virtio-win-prewhql-126.iso -monitor stdio -qmp tcp:0:4449,server,nowait -fda virtio-win-prewhql-126.vfd -vnc 0.0.0.0:21 -vga std -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0,vhost=on,queues=4 -device virtio-net-pci,mq=on,vectors=10,netdev=hostnet0,id=net0,mac=00:52:4c:20:8d:88 -incoming tcp::5888 3.#tcpdump -nntttt -i br0 icmp6 4.start migration from src guest: (qemu) migrate -d tcp:10.66.8.178(the dst host ip):5888 5.after migaration finished,check the tcpdump info Actual result: no NSM message found Do I need to migrate with ipv6 address?I try to boot dst guest with ipv6 listening port,but hit following error: -incoming tcp:[fe80::fe4d:d4ff:fedb:36b8]:5800: Failed to bind socket: Invalid argument What gives 'cat /etc/qemu-ifup'? (In reply to ybendito from comment #14) > What gives 'cat /etc/qemu-ifup'? #!/bin/sh switch=switch /sbin/ifconfig $1 0.0.0.0 up /usr/sbin/brctl addif ${switch} $1 /usr/sbin/brctl setfd ${switch} 0 /usr/sbin/brctl stp ${switch} off (In reply to lijin from comment #15) > (In reply to ybendito from comment #14) > > What gives 'cat /etc/qemu-ifup'? > > #!/bin/sh > switch=switch > /sbin/ifconfig $1 0.0.0.0 up > /usr/sbin/brctl addif ${switch} $1 > /usr/sbin/brctl setfd ${switch} 0 > /usr/sbin/brctl stp ${switch} off So to which bridge this actually connects the interface? What is the output of brctl show after migration? (In reply to ybendito from comment #16) > (In reply to lijin from comment #15) > > (In reply to ybendito from comment #14) > > > What gives 'cat /etc/qemu-ifup'? > > > > #!/bin/sh > > switch=switch > > /sbin/ifconfig $1 0.0.0.0 up > > /usr/sbin/brctl addif ${switch} $1 > > /usr/sbin/brctl setfd ${switch} 0 > > /usr/sbin/brctl stp ${switch} off > > So to which bridge this actually connects the interface? > What is the output of brctl show after migration? connect to bridge "switch" # brctl show bridge name bridge id STP enabled interfaces switch 8000.7ee4a6d47db0 no eno1 tap0 virbr0 8000.525400ab39ca yes virbr0-nic (In reply to lijin from comment #13) > I still can not find the NSM packets,could you help to check if my steps are > correct? > > 1.boot src guest on src host: > /usr/libexec/qemu-kvm -name 122BLKWIN2016 -enable-kvm -m 2G -smp 2 > -nodefconfig -nodefaults -rtc base=localtime,driftfix=slew -boot > order=cd,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive > file=win2012R2-lijin.raw,if=none,id=drive-ide0-0-0,format=raw, > serial=mike_cao,cache=none -device > ide-drive,drive=drive-ide0-0-0,id=ide0-0-0 -chardev pty,id=charserial0 > -device isa-serial,chardev=charserial0,id=isa_serial0 -device > usb-tablet,id=input0 -cdrom virtio-win-prewhql-126.iso -monitor stdio -qmp > tcp:0:4449,server,nowait -fda virtio-win-prewhql-126.vfd -vnc 0.0.0.0:21 > -vga std -netdev > tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0,vhost=on,queues=4 > -device > virtio-net-pci,mq=on,vectors=10,netdev=hostnet0,id=net0,mac=00:52:4c:20:8d:88 > > 2.boot dst guest on dst host: > /usr/libexec/qemu-kvm -name 122BLKWIN2016 -enable-kvm -m 2G -smp 2 > -nodefconfig -nodefaults -rtc base=localtime,driftfix=slew -boot > order=cd,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive > file=win2012R2-lijin.raw,if=none,id=drive-ide0-0-0,format=raw, > serial=mike_cao,cache=none -device > ide-drive,drive=drive-ide0-0-0,id=ide0-0-0 -chardev pty,id=charserial0 > -device isa-serial,chardev=charserial0,id=isa_serial0 -device > usb-tablet,id=input0 -cdrom virtio-win-prewhql-126.iso -monitor stdio -qmp > tcp:0:4449,server,nowait -fda virtio-win-prewhql-126.vfd -vnc 0.0.0.0:21 > -vga std -netdev > tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0,vhost=on,queues=4 > -device > virtio-net-pci,mq=on,vectors=10,netdev=hostnet0,id=net0,mac=00:52:4c:20:8d: > 88 -incoming tcp::5888 > > 3.#tcpdump -nntttt -i br0 icmp6 What is br0 here? According to what we see above this should be '-i switch'? > > 4.start migration from src guest: > (qemu) migrate -d tcp:10.66.8.178(the dst host ip):5888 > > 5.after migaration finished,check the tcpdump info > > Actual result: > no NSM message found > > > Do I need to migrate with ipv6 address?I try to boot dst guest with ipv6 > listening port,but hit following error: > -incoming tcp:[fe80::fe4d:d4ff:fedb:36b8]:5800: Failed to bind socket: > Invalid argument (In reply to ybendito from comment #18) > (In reply to lijin from comment #13) > > I still can not find the NSM packets,could you help to check if my steps are > > correct? > > > > 1.boot src guest on src host: > > /usr/libexec/qemu-kvm -name 122BLKWIN2016 -enable-kvm -m 2G -smp 2 > > -nodefconfig -nodefaults -rtc base=localtime,driftfix=slew -boot > > order=cd,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive > > file=win2012R2-lijin.raw,if=none,id=drive-ide0-0-0,format=raw, > > serial=mike_cao,cache=none -device > > ide-drive,drive=drive-ide0-0-0,id=ide0-0-0 -chardev pty,id=charserial0 > > -device isa-serial,chardev=charserial0,id=isa_serial0 -device > > usb-tablet,id=input0 -cdrom virtio-win-prewhql-126.iso -monitor stdio -qmp > > tcp:0:4449,server,nowait -fda virtio-win-prewhql-126.vfd -vnc 0.0.0.0:21 > > -vga std -netdev > > tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0,vhost=on,queues=4 > > -device > > virtio-net-pci,mq=on,vectors=10,netdev=hostnet0,id=net0,mac=00:52:4c:20:8d:88 > > > > 2.boot dst guest on dst host: > > /usr/libexec/qemu-kvm -name 122BLKWIN2016 -enable-kvm -m 2G -smp 2 > > -nodefconfig -nodefaults -rtc base=localtime,driftfix=slew -boot > > order=cd,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive > > file=win2012R2-lijin.raw,if=none,id=drive-ide0-0-0,format=raw, > > serial=mike_cao,cache=none -device > > ide-drive,drive=drive-ide0-0-0,id=ide0-0-0 -chardev pty,id=charserial0 > > -device isa-serial,chardev=charserial0,id=isa_serial0 -device > > usb-tablet,id=input0 -cdrom virtio-win-prewhql-126.iso -monitor stdio -qmp > > tcp:0:4449,server,nowait -fda virtio-win-prewhql-126.vfd -vnc 0.0.0.0:21 > > -vga std -netdev > > tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0,vhost=on,queues=4 > > -device > > virtio-net-pci,mq=on,vectors=10,netdev=hostnet0,id=net0,mac=00:52:4c:20:8d: > > 88 -incoming tcp::5888 > > > > 3.#tcpdump -nntttt -i br0 icmp6 > > What is br0 here? According to what we see above this should be '-i switch'? sorry,I did use switch here,not br0 > > > > 4.start migration from src guest: > > (qemu) migrate -d tcp:10.66.8.178(the dst host ip):5888 > > > > 5.after migaration finished,check the tcpdump info > > > > Actual result: > > no NSM message found > > > > > > Do I need to migrate with ipv6 address?I try to boot dst guest with ipv6 > > listening port,but hit following error: > > -incoming tcp:[fe80::fe4d:d4ff:fedb:36b8]:5800: Failed to bind socket: > > Invalid argument 1. What is output of "ipconfig /all" on the guest?
2. Are you able to ping the guest machine using IPv6 ping ("ping -6 machine")
I suspect the guest system does not have IPv6 address assigned.
Re-test with build 136 on win2016 between two hosts' migration can catch NSM message on dst and src hosts as below, please help to confirm . If it is right, i will change this bug to verified. [root@dhcp-9-217 qemu-29]# tcpdump -nntttt -i switch icmp6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on switch, link-type EN10MB (Ethernet), capture size 262144 bytes 2017-05-11 13:16:21.508078 IP6 :: > fe80::8972:b750:4707:1eaf: ICMP6, neighbor solicitation, who has fe80::8972:b750:4707:1eaf, length 24 2017-05-11 13:16:21.557898 IP6 :: > fe80::8972:b750:4707:1eaf: ICMP6, neighbor solicitation, who has fe80::8972:b750:4707:1eaf, length 24 2017-05-11 13:16:21.707854 IP6 :: > fe80::8972:b750:4707:1eaf: ICMP6, neighbor solicitation, who has fe80::8972:b750:4707:1eaf, length 24 2017-05-11 13:16:21.957880 IP6 :: > fe80::8972:b750:4707:1eaf: ICMP6, neighbor solicitation, who has fe80::8972:b750:4707:1eaf, length 24 2017-05-11 13:16:22.307868 IP6 :: > fe80::8972:b750:4707:1eaf: ICMP6, neighbor solicitation, who has fe80::8972:b750:4707:1eaf, length 24 Thanks Yu Wang (In reply to Yu Wang from comment #21) > Re-test with build 136 on win2016 between two hosts' migration > > > can catch NSM message on dst and src hosts as below, please help to confirm > . If it is right, i will change this bug to verified. > > [root@dhcp-9-217 qemu-29]# tcpdump -nntttt -i switch icmp6 > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on switch, link-type EN10MB (Ethernet), capture size 262144 bytes > 2017-05-11 13:16:21.508078 IP6 :: > fe80::8972:b750:4707:1eaf: ICMP6, > neighbor solicitation, who has fe80::8972:b750:4707:1eaf, length 24 > 2017-05-11 13:16:21.557898 IP6 :: > fe80::8972:b750:4707:1eaf: ICMP6, > neighbor solicitation, who has fe80::8972:b750:4707:1eaf, length 24 > 2017-05-11 13:16:21.707854 IP6 :: > fe80::8972:b750:4707:1eaf: ICMP6, > neighbor solicitation, who has fe80::8972:b750:4707:1eaf, length 24 > 2017-05-11 13:16:21.957880 IP6 :: > fe80::8972:b750:4707:1eaf: ICMP6, > neighbor solicitation, who has fe80::8972:b750:4707:1eaf, length 24 > 2017-05-11 13:16:22.307868 IP6 :: > fe80::8972:b750:4707:1eaf: ICMP6, > neighbor solicitation, who has fe80::8972:b750:4707:1eaf, length 24 > > > Thanks > Yu Wang Yes, this is very similar to what is expected, including timing of these messages (+50ms, 150ms, 250ms, 350ms). If this IPv6 address is the address of the virtio-net adapter of the vm that is migrating (you can see one in adapter properties or using ipconfig) - then all is as expected change status to verified according to commment#22 Hi Steve, Could you ack this bug? Thanks. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:2341 |