Bug 1577708
| Summary: | [virtio-win][netkvm][whql] Job "E2EPerf" fail with build151 | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Yu Wang <wyu> | ||||
| Component: | virtio-win | Assignee: | Sameeh Jubran <sjubran> | ||||
| virtio-win sub component: | virtio-win-prewhql | QA Contact: | Virtualization Bugs <virt-bugs> | ||||
| Status: | CLOSED ERRATA | Docs Contact: | |||||
| Severity: | high | ||||||
| Priority: | high | CC: | ddepaula, lijin, phou, sjubran, virt-bugs, vrozenfe, wyu, xiagao, yvugenfi | ||||
| Version: | 7.6 | Keywords: | Reopened | ||||
| Target Milestone: | rc | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: |
NO_DOCS
|
Story Points: | --- | ||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2018-10-30 16:21:51 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Hit this issue on all win8 and win8+ guests. Did you enable and configure RSC on the host? For more info on RSC please check this BZ: https://bugzilla.redhat.com/show_bug.cgi?id=950611 I have cherry-picked the useful comments on enabling the RSC: * To enable a vm with rsc feature on, put 'guest_rsc=on' in the virtio device option, this feature is turned off by default. * Before running the RSC test case, please make sure have turned 'tso,gso' of the tap devices off in host. ethtool -K tap$n tso off ethtool -K tap$n gso off ethtool -K tap$n gro off * When on _TAP_ the setting is "tx-checksumming: off, tcp-segmentation-offload: off" this actually disables the RSC (coalescing) on guest receiving path. (In reply to Sameeh Jubran from comment #4) > For more info on RSC please check this BZ: > https://bugzilla.redhat.com/show_bug.cgi?id=950611 > > I have cherry-picked the useful comments on enabling the RSC: > > * To enable a vm with rsc feature on, put 'guest_rsc=on' in the virtio > device option, this feature is turned off by default. > > * Before running the RSC test case, please make sure have turned 'tso,gso' > of the tap devices off in host. > > ethtool -K tap$n tso off > ethtool -K tap$n gso off > ethtool -K tap$n gro off It prompt "Property '.guest_rsc' not found", I remember that rsc is on by default before, and we can pass job "Run RSC Test" . > > * When on _TAP_ the setting is "tx-checksumming: off, > tcp-segmentation-offload: off" this actually disables the RSC (coalescing) > on guest receiving path. "tx-checksumming" and "tcp-segmentation-offload" if off on my host, and cannot turn on by "ethtool -K tap1 tcp-segmentation-offload on" details: [root@dell-per440-02 whql_autotest]# ethtool -k tap1 Features for tap1: rx-checksumming: off [fixed] tx-checksumming: on tx-checksum-ipv4: off [fixed] tx-checksum-ip-generic: on tx-checksum-ipv6: off [fixed] tx-checksum-fcoe-crc: off [fixed] tx-checksum-sctp: off [fixed] scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: on tcp-segmentation-offload: off tx-tcp-segmentation: off tx-tcp-ecn-segmentation: off tx-tcp6-segmentation: off tx-tcp-mangleid-segmentation: off udp-fragmentation-offload: off [requested on] generic-segmentation-offload: off generic-receive-offload: off large-receive-offload: off [fixed] rx-vlan-offload: off [fixed] tx-vlan-offload: on ntuple-filters: off [fixed] receive-hashing: off [fixed] highdma: off [fixed] rx-vlan-filter: off [fixed] vlan-challenged: off [fixed] tx-lockless: on [fixed] netns-local: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: off [fixed] tx-gre-segmentation: off [fixed] tx-ipip-segmentation: off [fixed] tx-sit-segmentation: off [fixed] tx-udp_tnl-segmentation: off [fixed] fcoe-mtu: off [fixed] tx-nocache-copy: off loopback: off [fixed] rx-fcs: off [fixed] rx-all: off [fixed] tx-vlan-stag-hw-insert: on rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: off [fixed] busy-poll: off [fixed] tx-gre-csum-segmentation: off [fixed] tx-udp_tnl-csum-segmentation: off [fixed] tx-gso-partial: off [fixed] tx-sctp-segmentation: off [fixed] l2-fwd-offload: off [fixed] hw-tc-offload: off [fixed] rx-udp_tunnel-port-offload: off [fixed] Does the old build of netkvm pass on this setup that you are using? How until today did you pass this test with RSC, wei's patches for RSC are not upstream, are they downstream on some branch/build of qemu? (In reply to Sameeh Jubran from comment #6) > Does the old build of netkvm pass on this setup that you are using? No, it failed with this setup. This is a new setup env., but we setup it the same as before, no new added configuration. > > How until today did you pass this test with RSC, wei's patches for RSC are > not upstream, are they downstream on some branch/build of qemu? We just use the package of breweb, no specified packages and setup. And as comment#0, there are different errors for this job. 1 using qemu-ifup1 (our private network) for virtio-net Adapter connected receive link speed must equal the maximum receive link speed. 2 using qemu-ifup-private (libvirt virbr0) for virtio-net Receiver failed to coalesce the minimum percentage of received packets (70%). Only 0% of receives were coalesced. Thanks Yu Wang (In reply to Yu Wang from comment #7) > (In reply to Sameeh Jubran from comment #6) > > Does the old build of netkvm pass on this setup that you are using? > > No, it failed with this setup. > This is a new setup env., but we setup it the same as before, no new added > configuration. > > > > > How until today did you pass this test with RSC, wei's patches for RSC are > > not upstream, are they downstream on some branch/build of qemu? > > We just use the package of breweb, no specified packages and setup. > > And as comment#0, there are different errors for this job. > > 1 using qemu-ifup1 (our private network) for virtio-net > Adapter connected receive link speed must equal the maximum receive link > speed. > > 2 using qemu-ifup-private (libvirt virbr0) for virtio-net > Receiver failed to coalesce the minimum percentage of received packets > (70%). Only 0% of receives were coalesced. > > Thanks > Yu Wang Then the RSC needs to be configured correctly, you need to apply the following patches to the suitable components: host's kernel: https://patchwork.kernel.org/patch/9139473/ qemu: http://qemu-devel.nongnu.narkive.com/BE04yt47/rfc-patch-v7-0-2-support-receive-segment-offload-rsc-for-whql (In reply to Sameeh Jubran from comment #8) > (In reply to Yu Wang from comment #7) > > (In reply to Sameeh Jubran from comment #6) > > > Does the old build of netkvm pass on this setup that you are using? > > > > No, it failed with this setup. > > This is a new setup env., but we setup it the same as before, no new added > > configuration. > > > > > > > > How until today did you pass this test with RSC, wei's patches for RSC are > > > not upstream, are they downstream on some branch/build of qemu? > > > > We just use the package of breweb, no specified packages and setup. > > > > And as comment#0, there are different errors for this job. > > > > 1 using qemu-ifup1 (our private network) for virtio-net > > Adapter connected receive link speed must equal the maximum receive link > > speed. > > > > 2 using qemu-ifup-private (libvirt virbr0) for virtio-net > > Receiver failed to coalesce the minimum percentage of received packets > > (70%). Only 0% of receives were coalesced. > > > > Thanks > > Yu Wang > > Then the RSC needs to be configured correctly, you need to apply the > following patches to the suitable components: > > host's kernel: https://patchwork.kernel.org/patch/9139473/ > qemu: > http://qemu-devel.nongnu.narkive.com/BE04yt47/rfc-patch-v7-0-2-support- > receive-segment-offload-rsc-for-whql We never applied these patches during rhel7.5 but jobs passed correctly. And during 151 testing, job "Run RSC Tests" passes while "E2EPerf" fails. If rsc is not configured correctly,shouldn't job "Run RSC Tests" fail as well? (In reply to lijin from comment #9) > (In reply to Sameeh Jubran from comment #8) > > (In reply to Yu Wang from comment #7) > > > (In reply to Sameeh Jubran from comment #6) > > > > Does the old build of netkvm pass on this setup that you are using? > > > > > > No, it failed with this setup. > > > This is a new setup env., but we setup it the same as before, no new added > > > configuration. > > > > > > > > > > > How until today did you pass this test with RSC, wei's patches for RSC are > > > > not upstream, are they downstream on some branch/build of qemu? > > > > > > We just use the package of breweb, no specified packages and setup. > > > > > > And as comment#0, there are different errors for this job. > > > > > > 1 using qemu-ifup1 (our private network) for virtio-net > > > Adapter connected receive link speed must equal the maximum receive link > > > speed. > > > > > > 2 using qemu-ifup-private (libvirt virbr0) for virtio-net > > > Receiver failed to coalesce the minimum percentage of received packets > > > (70%). Only 0% of receives were coalesced. > > > > > > Thanks > > > Yu Wang > > > > Then the RSC needs to be configured correctly, you need to apply the > > following patches to the suitable components: > > > > host's kernel: https://patchwork.kernel.org/patch/9139473/ > > qemu: > > http://qemu-devel.nongnu.narkive.com/BE04yt47/rfc-patch-v7-0-2-support- > > receive-segment-offload-rsc-for-whql > > We never applied these patches during rhel7.5 but jobs passed correctly. > > And during 151 testing, job "Run RSC Tests" passes while "E2EPerf" fails. If > rsc is not configured correctly,shouldn't job "Run RSC Tests" fail as well? Not necessarily, but the old driver should pass E2EPerf and it doesn't which indicates it's an issue with the setup. Do you still have the old setup of the testing?? Can you try it there? (In reply to Sameeh Jubran from comment #10) > (In reply to lijin from comment #9) > > (In reply to Sameeh Jubran from comment #8) > > > (In reply to Yu Wang from comment #7) > > > > (In reply to Sameeh Jubran from comment #6) > > > > > Does the old build of netkvm pass on this setup that you are using? > > > > > > > > No, it failed with this setup. > > > > This is a new setup env., but we setup it the same as before, no new added > > > > configuration. > > > > > > > > > > > > > > How until today did you pass this test with RSC, wei's patches for RSC are > > > > > not upstream, are they downstream on some branch/build of qemu? > > > > > > > > We just use the package of breweb, no specified packages and setup. > > > > > > > > And as comment#0, there are different errors for this job. > > > > > > > > 1 using qemu-ifup1 (our private network) for virtio-net > > > > Adapter connected receive link speed must equal the maximum receive link > > > > speed. > > > > > > > > 2 using qemu-ifup-private (libvirt virbr0) for virtio-net > > > > Receiver failed to coalesce the minimum percentage of received packets > > > > (70%). Only 0% of receives were coalesced. > > > > > > > > Thanks > > > > Yu Wang > > > > > > Then the RSC needs to be configured correctly, you need to apply the > > > following patches to the suitable components: > > > > > > host's kernel: https://patchwork.kernel.org/patch/9139473/ > > > qemu: > > > http://qemu-devel.nongnu.narkive.com/BE04yt47/rfc-patch-v7-0-2-support- > > > receive-segment-offload-rsc-for-whql > > > > We never applied these patches during rhel7.5 but jobs passed correctly. > > > > And during 151 testing, job "Run RSC Tests" passes while "E2EPerf" fails. If > > rsc is not configured correctly,shouldn't job "Run RSC Tests" fail as well? > > Not necessarily, but the old driver should pass E2EPerf and it doesn't which > indicates it's an issue with the setup. QE is also suspect the new env.'s issue, but still no clue. > Do you still have the old setup of the testing?? Can you try it there? No, old env. is lost as hardware issue. QE will continue to debug our setup,thanks a lot. Hi Sameeh, I try on windows 10 and get following result: For the subtest "Verifying Test Environment": It will failed with below error if use default value with virtio-net-pci: - Error 5/20/2018 9:56:25.729 PM Adapter connected receive link speed must equal the maximum receive link speed. - Error 5/20/2018 9:56:25.729 PM Adapter connected transmit link speed must equal the maximum transmit link speed. I check inside guest, the speed is 10.0 Gbps. And I check the logs below: Link Properties for Red Hat VirtIO Ethernet Adapter Current Media State Connected Current Transmit Link Speed 10,000,000,000 bps Current Receive Link Speed 10,000,000,000 bps Max Transmit Link Speed 100,000,000,000 bps Max Receive Link Speed 100,000,000,000 bps MediaDuplexState 0x00000002 PauseFunctions 0x00000000 AutoNegotiationFlags 0x00000000 Failure: __COUNTER__ Adapter connected receive link speed must equal the maximum receive link speed. Failure: __COUNTER__ Adapter connected transmit link speed must equal the maximum transmit link speed. After I set speed with qemu cli: "speed=100000", this error **disappear**. Do you know why Max Transmit/Receive Link Speed is 100,000,000,000 bps here? And if boot virtio-net-pci with libvirt bridge(virbr0):
-netdev tap,script=/etc/qemu-ifup-private,downscript=no,id=hostnet1,vhost=on -device virtio-net-pci,netdev=hostnet1,bus=pci.0,id=net1,mac=00:52:1a:62:8f:dc
]# cat /etc/qemu-ifup-private
#!/bin/sh
switch=virbr0
/sbin/ifconfig $1 promisc 0.0.0.0 &&
/usr/sbin/brctl addif ${switch} $1 &&
ethtool -K $1 tx off
Will hit following errors:
[Receives 128K TCP] Analyzing Traffic results
- Error 5/20/2018 10:00:03.114 PM Receiver failed to coalesce the minimum percentage of received packets (70%). Only 0% of receives were coalesced.
[Receives 64K TCP] Analyzing Traffic results
- Error 5/20/2018 10:00:50.233 PM Receiver failed to coalesce the minimum percentage of received packets (70%). Only 0% of receives were coalesced.
[Receives 4K TCP] Analyzing Traffic results
- Error 5/20/2018 10:01:37.298 PM Receiver failed to coalesce the minimum percentage of received packets (70%). Only 0% of receives were coalesced.
Change virtio-net-pci to another bridge, this error **disappear**
Do you have any clue of this result?
I will downgrade libvirt as well to check if it's libvirt issue at the same time
Combined comment#12 and comment#13, this job can pass only when set netkvm with non-virbr0 bridge and set speed=100000 for it. (In reply to lijin from comment #13) > And if boot virtio-net-pci with libvirt bridge(virbr0): > -netdev tap,script=/etc/qemu-ifup-private,downscript=no,id=hostnet1,vhost=on > -device > virtio-net-pci,netdev=hostnet1,bus=pci.0,id=net1,mac=00:52:1a:62:8f:dc > > ]# cat /etc/qemu-ifup-private > #!/bin/sh > switch=virbr0 > /sbin/ifconfig $1 promisc 0.0.0.0 && > /usr/sbin/brctl addif ${switch} $1 && > ethtool -K $1 tx off Do you run "ethtool -K $1 tx off" command as well when using another bridge? I think it is causing the problem. > > > Will hit following errors: > [Receives 128K TCP] Analyzing Traffic results > - Error 5/20/2018 10:00:03.114 PM Receiver failed to coalesce the minimum > percentage of received packets (70%). Only 0% of receives were coalesced. > > [Receives 64K TCP] Analyzing Traffic results > - Error 5/20/2018 10:00:50.233 PM Receiver failed to coalesce the minimum > percentage of received packets (70%). Only 0% of receives were coalesced. > > [Receives 4K TCP] Analyzing Traffic results > - Error 5/20/2018 10:01:37.298 PM Receiver failed to coalesce the minimum > percentage of received packets (70%). Only 0% of receives were coalesced. > > > Change virtio-net-pci to another bridge, this error **disappear** > > Do you have any clue of this result? > > I will downgrade libvirt as well to check if it's libvirt issue at the same > time (In reply to Yan Vugenfirer from comment #15) > (In reply to lijin from comment #13) > > And if boot virtio-net-pci with libvirt bridge(virbr0): > > -netdev tap,script=/etc/qemu-ifup-private,downscript=no,id=hostnet1,vhost=on > > -device > > virtio-net-pci,netdev=hostnet1,bus=pci.0,id=net1,mac=00:52:1a:62:8f:dc > > > > ]# cat /etc/qemu-ifup-private > > #!/bin/sh > > switch=virbr0 > > /sbin/ifconfig $1 promisc 0.0.0.0 && > > /usr/sbin/brctl addif ${switch} $1 && > > ethtool -K $1 tx off > > Do you run "ethtool -K $1 tx off" command as well when using another bridge? > I think it is causing the problem. Yes, you are right, after delete this line, this error disappear. Thanks a lot. Do you have any thought about comment#12? (In reply to lijin from comment #12) > Hi Sameeh, > > I try on windows 10 and get following result: > > For the subtest "Verifying Test Environment": > It will failed with below error if use default value with virtio-net-pci: > - Error 5/20/2018 9:56:25.729 PM Adapter connected receive link speed must > equal the maximum receive link speed. > - Error 5/20/2018 9:56:25.729 PM Adapter connected transmit link speed must > equal the maximum transmit link speed. > I check inside guest, the speed is 10.0 Gbps. > > And I check the logs below: > Link Properties for Red Hat VirtIO Ethernet Adapter > Current Media State Connected > Current Transmit Link Speed 10,000,000,000 bps > Current Receive Link Speed 10,000,000,000 bps > Max Transmit Link Speed 100,000,000,000 bps > Max Receive Link Speed 100,000,000,000 bps > MediaDuplexState 0x00000002 > PauseFunctions 0x00000000 > AutoNegotiationFlags 0x00000000 > > Failure: __COUNTER__ Adapter connected receive link speed must equal the > maximum receive link speed. > Failure: __COUNTER__ Adapter connected transmit link speed must equal the > maximum transmit link speed. > > After I set speed with qemu cli: "speed=100000", this error **disappear**. > > Do you know why Max Transmit/Receive Link Speed is 100,000,000,000 bps here? This is because of a new feature (BZ#1537430): https://github.com/virtio-win/kvm-guest-drivers-windows/commit/23d536f6183f342cae5caf87861d65b4e1992498 https://github.com/virtio-win/kvm-guest-drivers-windows/commit/aecbd2dfb388ae27644093a409fa8afebf88e617 https://github.com/virtio-win/kvm-guest-drivers-windows/commit/1adaf6ffb21d37aad8e2c2f8a55491d49cd668a5 Now we can set the link speed from QEMU command line. The previous MAX speed and default current speed was 10G. Now the current stayed the same, but we increased MAX speed to allow to set it to a higher value from QEMU. That probably causes the problem with perf test. (In reply to Yan Vugenfirer from comment #17) > (In reply to lijin from comment #12) > > Hi Sameeh, > > > > I try on windows 10 and get following result: > > > > For the subtest "Verifying Test Environment": > > It will failed with below error if use default value with virtio-net-pci: > > - Error 5/20/2018 9:56:25.729 PM Adapter connected receive link speed must > > equal the maximum receive link speed. > > - Error 5/20/2018 9:56:25.729 PM Adapter connected transmit link speed must > > equal the maximum transmit link speed. > > I check inside guest, the speed is 10.0 Gbps. > > > > And I check the logs below: > > Link Properties for Red Hat VirtIO Ethernet Adapter > > Current Media State Connected > > Current Transmit Link Speed 10,000,000,000 bps > > Current Receive Link Speed 10,000,000,000 bps > > Max Transmit Link Speed 100,000,000,000 bps > > Max Receive Link Speed 100,000,000,000 bps > > MediaDuplexState 0x00000002 > > PauseFunctions 0x00000000 > > AutoNegotiationFlags 0x00000000 > > > > Failure: __COUNTER__ Adapter connected receive link speed must equal the > > maximum receive link speed. > > Failure: __COUNTER__ Adapter connected transmit link speed must equal the > > maximum transmit link speed. > > > > After I set speed with qemu cli: "speed=100000", this error **disappear**. > > > > Do you know why Max Transmit/Receive Link Speed is 100,000,000,000 bps here? > > This is because of a new feature (BZ#1537430): > https://github.com/virtio-win/kvm-guest-drivers-windows/commit/ > 23d536f6183f342cae5caf87861d65b4e1992498 > > https://github.com/virtio-win/kvm-guest-drivers-windows/commit/ > aecbd2dfb388ae27644093a409fa8afebf88e617 > > https://github.com/virtio-win/kvm-guest-drivers-windows/commit/ > 1adaf6ffb21d37aad8e2c2f8a55491d49cd668a5 > > Now we can set the link speed from QEMU command line. The previous MAX speed > and default current speed was 10G. Now the current stayed the same, but we > increased MAX speed to allow to set it to a higher value from QEMU. That > probably causes the problem with perf test. Do you have any plan to set current speed to MAX as well? If not, QE will add speed parameter to whql scripts to make netkvm whql pass. (In reply to lijin from comment #18) > (In reply to Yan Vugenfirer from comment #17) > > (In reply to lijin from comment #12) .... > > Now we can set the link speed from QEMU command line. The previous MAX speed > > and default current speed was 10G. Now the current stayed the same, but we > > increased MAX speed to allow to set it to a higher value from QEMU. That > > probably causes the problem with perf test. > > Do you have any plan to set current speed to MAX as well? > If not, QE will add speed parameter to whql scripts to make netkvm whql pass. Not for now. So better add speed parameter to the scripts. Thanks. (In reply to Yan Vugenfirer from comment #19) > (In reply to lijin from comment #18) > > (In reply to Yan Vugenfirer from comment #17) > > > (In reply to lijin from comment #12) > .... > > > Now we can set the link speed from QEMU command line. The previous MAX speed > > > and default current speed was 10G. Now the current stayed the same, but we > > > increased MAX speed to allow to set it to a higher value from QEMU. That > > > probably causes the problem with perf test. > > > > Do you have any plan to set current speed to MAX as well? > > If not, QE will add speed parameter to whql scripts to make netkvm whql pass. > > Not for now. So better add speed parameter to the scripts. > > Thanks. Got it, thanks Closing this bug. Re-open the bug as qe are running whql testing in rhv env., there is no way to workaround this issue(qemu cli is generated automatically). whql jobs should pass as well in the upper layers. Maybe we can set current speed to max speed. Reproduce this bug with build 154 Verified this bug with build 156 Steps as comment#0 Above all, this bug has been fixed and change status to verified. Thanks Yu Wang Can we have QA, PM and Release flags for this, please? Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:3413 |
Created attachment 1435987 [details] E2Eperf-failed-logs Description of problem: job log refer to attachment Version-Release number of selected component (if applicable): virtio-win-prewhql-151 How reproducible: 100% Steps to Reproduce: 1. Boot guest with net device /usr/libexec/qemu-kvm -name 151NICBLUE64SGL -enable-kvm -m 6G -smp 4 -uuid d4dc34ff-0a9d-4cc0-8529-fe5278faf363 -nodefconfig -nodefaults -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff -chardev socket,id=charmonitor,path=/tmp/151NICBLUE64SGL,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -boot order=cd,menu=on -device piix3-usb-uhci,id=usb -drive file=151NICBLUE64SGL,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=en_windows_8.1_enterprise_with_update_x64_dvd_6054382.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=151NICBLUE64SGL.vfd,if=floppy,id=drive-fdc0-0-0,format=raw,cache=none -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=00:52:34:74:5c:0c -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:1 -vga std -M pc -netdev tap,script=/etc/qemu-ifup1,downscript=no,id=hostnet1,vhost=on,queues=4 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=00:52:6c:57:ac:c6,bus=pci.0,mq=on,vectors=10 2. submit job E2Eperf 3. Actual results: Failed Expected results: Pass Additional info: 1 using qemu-ifup1 (our private network) Adapter connected receive link speed must equal the maximum receive link speed. 2 using qemu-ifup-private (libvirt virbr0) Receiver failed to coalesce the minimum percentage of received packets (70%). Only 0% of receives were coalesced. 3 if use qemu-ifup-private (libvirt virbr0), both failed with rhel7.5 release build and build151, so it is a regression.