1223426 – [NetKVM] Performance degradation with multi-queue

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1223426 - [NetKVM] Performance degradation with multi-queue

Summary: [NetKVM] Performance degradation with multi-queue

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	virtio-win
Sub Component:
Version:	7.3
Hardware:	Unspecified
OS:	Windows
Priority:	unspecified
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	Yvugenfi@redhat.com
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2015-05-20 13:34 UTC by Yvugenfi@redhat.com
Modified:	2016-11-04 08:46 UTC (History)
CC List:	6 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:	NO_DOCS
Clone Of:
Environment:
Last Closed:	2016-11-04 08:46:28 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2016:2609	0	normal	SHIPPED_LIVE	virtio-win bug fix and enhancement update	2016-11-03 15:27:12 UTC

Description Yvugenfi@redhat.com 2015-05-20 13:34:51 UTC

Description of problem:

Reported for upstream code by Google:

-->
We pulled the latest netkvm changes last week and the performance droped from around 7 Gbps to 2.8. It is a huge drop. Just wanted to let you know.... we will probably revert the February changes if we cant' figure out what is causing this.
-->

On our side, perf was still good at 592fc9e7c428237cebd34baae5231aea5bcc96d8, and it went back on 630ae942aaa7b7a3f3a76bc0a46041f2873e3893. Given the fact that the perf keep normal with vhost+mq on your side, I will also look at our device side which is a virtio-net compatible implementation to cross-check any possible cause. (The perf degradation was measure with exact same device code though).

-->

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Yu Wang 2016-03-23 08:57:02 UTC

Hi Yan,


Could you tell me what is the detail steps and test tool for reproducing this bug? And what is the virtio-win-prewhql version that cause Performance degradation ?


Thanks
wyu

Comment 2 Yvugenfi@redhat.com 2016-03-23 09:15:05 UTC

Hi Wyu,

You can use netperf or iperf to test the performance. Run the tools with 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536 packet sizes.
In NetKVM code there is even Tests directory with wrapper for netperf, but you don't have to use it - you can use your own script if it is more convenient.

https://github.com/YanVugenfirer/kvm-guest-drivers-windows/blob/master/NetKVM/Tests/netperf_wrapper.rb


Builds to compare - latest released driver https://brewweb.devel.redhat.com/buildinfo?buildID=467728 (we still don't have MQ in the release) with 
latest virtio-win-prewhql driver.


Best regards,
Yan.

Comment 7 Yu Wang 2016-05-17 02:16:58 UTC

Hi Yan,

compared above two versions on comment#3, there is no obvious degradation and increase with multiqueue, is that OK? Or the performance should be increased with multiqueue?

Thanks
wyu

Comment 8 Yvugenfi@redhat.com 2016-05-22 09:03:29 UTC

(In reply to wangyu from comment #7)
> Hi Yan,
> 
> compared above two versions on comment#3, there is no obvious degradation
> and increase with multiqueue, is that OK? Or the performance should be
> increased with multiqueue?
> 
> Thanks
> wyu

Hi Wyu,

With multi-queue when using several streams we should see some performance improvement. With one stream, it should be the same.

Best regards,
Yan.

Comment 9 Yu Wang 2016-05-27 03:21:20 UTC

change status to "VERIFIED"  according to comment#3 and comment#8

Comment 12 errata-xmlrpc 2016-11-04 08:46:28 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2609.html

Note You need to log in before you can comment on or make changes to this bug.