Bug 1465813

Summary: [virtio-win][whql][netkvm][macvtap] jobs failed with error "Received less than the expected percentage of packets sent"
Product: Red Hat Enterprise Linux 7 Reporter: Peixiu Hou <phou>
Component: virtio-winAssignee: Yvugenfi <yvugenfi>
virtio-win sub component: virtio-win-prewhql QA Contact: Virtualization Bugs <virt-bugs>
Status: CLOSED WONTFIX Docs Contact:
Severity: high    
Priority: unspecified CC: ailan, lijin, phou, xiagao
Version: 7.4   
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-07-10 07:24:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Peixiu Hou 2017-06-28 09:32:02 UTC
Description of problem:
Follows job failed with error "Received less than the expected percentage of packets sent":
NDISTest 6.5 - [2 Machine] - PacketFilters
NDISTest 6.5 - [2 Machine] - MultiCastAddress
NDISTest 6.5 - [2 Machine] - Checkconnection

OS:
Win2012-64/R2, Win8-32/64, Win8.1-32/64

Version-Release number of selected component (if applicable):
kernel-3.10.0-679.el7.x86_64
qemu-img-rhev-2.9.0-9.el7.x86_64
virtio-win-prewhql-139

How reproducible:
100%

Steps to Reproduce:
1. Set macvtap2 link to em2:
#ip link add link em2 name macvtap2 type macvtap mode bridge
#ip link set macvtap2 address 1a:46:0b:ca:bc:06 up
#ip link show macvtap2
78: macvtap2@em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 8192
    link/ether 1a:46:0b:ca:bc:06 brd ff:ff:ff:ff:ff:ff

2. Also set macvtap3 link to em2.
#ip link show macvtap3
79: macvtap3@em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 8192
    link/ether 1a:46:0b:ca:bc:07 brd ff:ff:ff:ff:ff:ff

3. Boot guest with macvtap device:
Client: 
/usr/libexec/qemu-kvm -name 139NICBLUE32C8P -enable-kvm -m 6G -smp 8,cores=8 -uuid 0be67a30-a247-4e5c-b504-12e8f1bbdfcb -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/tmp/139NICBLUE32C8P,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -boot order=cd,menu=on -device piix3-usb-uhci,id=usb -drive file=139NICBLUE32C8P,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=en_windows_8_1_enterprise_x86_dvd_2972289.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=139NICBLUE32C8P.vfd,if=floppy,id=drive-fdc0-0-0,format=raw,cache=none -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=00:52:53:3f:0d:df -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0 -device usb-tablet,id=input0 -spice id=on,disable-ticketing,port=5914 -vga qxl -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -M pc -netdev tap,vhost=on,id=macvtap_netdev,fd=78 78<>/dev/tap78 -device virtio-net-pci,netdev=macvtap_netdev,mac=1c:47:0b:ca:bc:06

Server:
/usr/libexec/qemu-kvm -name 139NICBLUE32S8P -enable-kvm -m 6G -smp 4 -uuid 4deda008-227c-4fe4-9ed5-c32f7e8d4539 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/tmp/139NICBLUE32S8P,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -boot order=cd,menu=on -device piix3-usb-uhci,id=usb -drive file=139NICBLUE32S8P,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=en_windows_8_1_enterprise_x86_dvd_2972289.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=139NICBLUE32S8P.vfd,if=floppy,id=drive-fdc0-0-0,format=raw,cache=none -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=00:52:1e:5d:ff:45 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:5 -vga std -M pc -netdev tap,vhost=on,id=macvtap_netdev,fd=79 79<>/dev/tap79 -device virtio-net-pci,netdev=macvtap_netdev,mac=1a:46:0b:ca:bc:07

2. Submit the job:
NDISTest 6.5 - [2 Machine] - PacketFilters
NDISTest 6.5 - [2 Machine] - MultiCastAddress
NDISTest 6.5 - [2 Machine] - Checkconnection

3.Check the result.

Actual results:
Failed

Expected results:
Passed

Additional info:
1. Tried with the default txqueuelen 500, reproduced this bug.
2. Tried with txqueuelen 8192, reproduced this bug.
3. Tried with public network(ip show 10.73.75.*), reproduced this bug.
4. Tried with private network(ip show 192.168.0.*), reproduced this bug.

Comment 3 Peixiu Hou 2017-06-28 10:01:11 UTC
1. The job NDISTest 6.5 - [2 Machine] - Address Change, hit similar error message, details as following:
--------------------------------------------------------------------------------
Error 6/26/2017 6:30:30.907 AM Test adapter Red Hat VirtIO Ethernet Adapter received less than the expected percentage of packets sent.
Test adapter is expected to receive 237 packets, (95 percent of 250 packets sent from Support adapter Red Hat VirtIO Ethernet Adapter)
Please check that your hardware has been programmed with the new network address it read from the registry. 


2. The job NDISTest 6.5 - [2 Machine] - Stats, hit similar error message, details as following:
--------------------------------------------------------------------------------
Error 6/14/2017 12:35:16.318 PM Open did not receive the minimum required percentage of packets transmitted by the sender.

Error 6/14/2017 12:35:20.318 PM Open did not receive the minimum required percentage of packets transmitted by the sender. On CommHelper SendSimpleCommHelper_0, received 0 packets. 

3. The job NDISTest 6.0 - [2 Machine] - 2c_Priority, failed with error "Unable to get priority test results on Test adapter " on win2012-64/R2, win8.1-32/64.
On win8-32/64, it can be filtered pass. The errata info as follows:
-------------------------------------------------------------------------------
NDISTest 6.0 - [2 Machine] - 2C Priority test is failing with the error message, Unable to get priority test results on Test adapter.
Errata detail
Filter id : 	1604
Expires : 	7/1/2017
Type : 	Errata
Issue description : 	The NDISTest 6.0 - [2 Machine] - 2C Priority test is failing with the error message, Unable to get priority test results on Test adapter.
Resolution description : 	10/100 Mbps nic adapters are not required to meet this requirement - Device.Network.LAN.Base.PriorityVLAN. If the Ethernet device does not implement link speeds of gigabit or greater than this requirement does not apply. This is a device specific errata. To add additional devices to this filter please open a support case and upload a hckx package.
Features : 	
Operating system :

Comment 4 Yvugenfi@redhat.com 2017-06-29 11:10:37 UTC
(In reply to Peixiu Hou from comment #3)
> 1. The job NDISTest 6.5 - [2 Machine] - Address Change, hit similar error
> message, details as following:
> -----------------------------------------------------------------------------
> ---
> Error 6/26/2017 6:30:30.907 AM Test adapter Red Hat VirtIO Ethernet Adapter
> received less than the expected percentage of packets sent.
> Test adapter is expected to receive 237 packets, (95 percent of 250 packets
> sent from Support adapter Red Hat VirtIO Ethernet Adapter)
> Please check that your hardware has been programmed with the new network
> address it read from the registry. 
> 
> 
> 2. The job NDISTest 6.5 - [2 Machine] - Stats, hit similar error message,
> details as following:
> -----------------------------------------------------------------------------
> ---
> Error 6/14/2017 12:35:16.318 PM Open did not receive the minimum required
> percentage of packets transmitted by the sender.
> 
> Error 6/14/2017 12:35:20.318 PM Open did not receive the minimum required
> percentage of packets transmitted by the sender. On CommHelper
> SendSimpleCommHelper_0, received 0 packets. 
> 
> 3. The job NDISTest 6.0 - [2 Machine] - 2c_Priority, failed with error
> "Unable to get priority test results on Test adapter " on win2012-64/R2,
> win8.1-32/64.
> On win8-32/64, it can be filtered pass. The errata info as follows:
> -----------------------------------------------------------------------------
> --
> NDISTest 6.0 - [2 Machine] - 2C Priority test is failing with the error
> message, Unable to get priority test results on Test adapter.
> Errata detail
> Filter id : 	1604
> Expires : 	7/1/2017
> Type : 	Errata
> Issue description : 	The NDISTest 6.0 - [2 Machine] - 2C Priority test is
> failing with the error message, Unable to get priority test results on Test
> adapter.
> Resolution description : 	10/100 Mbps nic adapters are not required to meet
> this requirement - Device.Network.LAN.Base.PriorityVLAN. If the Ethernet
> device does not implement link speeds of gigabit or greater than this
> requirement does not apply. This is a device specific errata. To add
> additional devices to this filter please open a support case and upload a
> hckx package.
> Features : 	
> Operating system :

ip link set macvtap2 address 1a:46:0b:ca:bc:06 up

Address change changes the mac address of the NIC in the guest. In such case I think macvtap will filter out the packets.

Also shouldn't we run for macvtap to pass multicast packets?
ifconfig macvtap<x> allmulti

Comment 5 Peixiu Hou 2017-06-30 03:22:36 UTC
(In reply to Yan Vugenfirer from comment #4)
> (In reply to Peixiu Hou from comment #3)
> > 1. The job NDISTest 6.5 - [2 Machine] - Address Change, hit similar error
> > message, details as following:
> > -----------------------------------------------------------------------------
> > ---
> > Error 6/26/2017 6:30:30.907 AM Test adapter Red Hat VirtIO Ethernet Adapter
> > received less than the expected percentage of packets sent.
> > Test adapter is expected to receive 237 packets, (95 percent of 250 packets
> > sent from Support adapter Red Hat VirtIO Ethernet Adapter)
> > Please check that your hardware has been programmed with the new network
> > address it read from the registry. 
> > 
> > 
> > 2. The job NDISTest 6.5 - [2 Machine] - Stats, hit similar error message,
> > details as following:
> > -----------------------------------------------------------------------------
> > ---
> > Error 6/14/2017 12:35:16.318 PM Open did not receive the minimum required
> > percentage of packets transmitted by the sender.
> > 
> > Error 6/14/2017 12:35:20.318 PM Open did not receive the minimum required
> > percentage of packets transmitted by the sender. On CommHelper
> > SendSimpleCommHelper_0, received 0 packets. 
> > 
> > 3. The job NDISTest 6.0 - [2 Machine] - 2c_Priority, failed with error
> > "Unable to get priority test results on Test adapter " on win2012-64/R2,
> > win8.1-32/64.
> > On win8-32/64, it can be filtered pass. The errata info as follows:
> > -----------------------------------------------------------------------------
> > --
> > NDISTest 6.0 - [2 Machine] - 2C Priority test is failing with the error
> > message, Unable to get priority test results on Test adapter.
> > Errata detail
> > Filter id : 	1604
> > Expires : 	7/1/2017
> > Type : 	Errata
> > Issue description : 	The NDISTest 6.0 - [2 Machine] - 2C Priority test is
> > failing with the error message, Unable to get priority test results on Test
> > adapter.
> > Resolution description : 	10/100 Mbps nic adapters are not required to meet
> > this requirement - Device.Network.LAN.Base.PriorityVLAN. If the Ethernet
> > device does not implement link speeds of gigabit or greater than this
> > requirement does not apply. This is a device specific errata. To add
> > additional devices to this filter please open a support case and upload a
> > hckx package.
> > Features : 	
> > Operating system :
> 
> ip link set macvtap2 address 1a:46:0b:ca:bc:06 up
> 
> Address change changes the mac address of the NIC in the guest. In such case
> I think macvtap will filter out the packets.
> 
> Also shouldn't we run for macvtap to pass multicast packets?
> ifconfig macvtap<x> allmulti

Thanks Yan, after set ifconfig macvtap<x> allmulti, the job "MuiltiCast Address", "Checkconnection" and "2c_priority" can be passed, thanks so much~

Comment 6 Yvugenfi@redhat.com 2017-07-02 12:12:39 UTC
What about "NDISTest 6.5 - [2 Machine] - PacketFilters", stats and address change? Are they still failing?

Comment 7 Peixiu Hou 2017-07-03 02:22:57 UTC
(In reply to Yan Vugenfirer from comment #6)
> What about "NDISTest 6.5 - [2 Machine] - PacketFilters", stats and address
> change? Are they still failing?

Hi Yan, 

Tried them again, after set ifconfig macvtap<x> allmulti, the job stats can be passed, but the job "PacketFilters" and "Address Change" also failed.


Best Regards~
Peixiu

Comment 8 xiagao 2017-07-03 03:09:28 UTC
After set "ifconfig macvtap<x> allmulti" , 

The job NDISTest 6.0 - [2 Machine] - 2c_Priority, still failed with error "Unable to get priority test results on Test adapter" on 2008-32 and 2008-64, and it can NOT be filtered pass.

Comment 9 Yvugenfi@redhat.com 2017-07-03 11:38:38 UTC
(In reply to xiagao from comment #8)
> After set "ifconfig macvtap<x> allmulti" , 
> 
> The job NDISTest 6.0 - [2 Machine] - 2c_Priority, still failed with error
> "Unable to get priority test results on Test adapter" on 2008-32 and
> 2008-64, and it can NOT be filtered pass.

Please double check the setup. It looks that the test suit cannot gather logs, rather then some test error.

Comment 10 xiagao 2017-07-05 01:14:52 UTC
(In reply to Yan Vugenfirer from comment #9)
> (In reply to xiagao from comment #8)
> > After set "ifconfig macvtap<x> allmulti" , 
> > 
> > The job NDISTest 6.0 - [2 Machine] - 2c_Priority, still failed with error
> > "Unable to get priority test results on Test adapter" on 2008-32 and
> > 2008-64, and it can NOT be filtered pass.
> 
> Please double check the setup. It looks that the test suit cannot gather
> logs, rather then some test error.

Yes, you are right. Reset the env, this job passed.
Thanks.

Comment 11 Yvugenfi@redhat.com 2017-07-05 09:19:14 UTC
So now we have two failing tests for the same reason.

Tests: "PacketFilters" and "Address Change"

The reason: After MAC address change in the guest, macvtap drops the packets.