Bug 467845 - RHEL 5.3: allow tcp socket buffers grow to larger than a page size
RHEL 5.3: allow tcp socket buffers grow to larger than a page size
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel (Show other bugs)
All Linux
medium Severity medium
: rc
: ---
Assigned To: Mark McLoughlin
Martin Jenner
Depends On:
  Show dependency treegraph
Reported: 2008-10-21 04:59 EDT by Mark McLoughlin
Modified: 2011-03-24 07:54 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-01-20 15:16:01 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
tcp-Let-skbs-grow-over-a-page-on-fast-peers.patch (2.68 KB, patch)
2008-10-21 06:16 EDT, Mark McLoughlin
no flags Details | Diff

  None (edit)
Description Mark McLoughlin 2008-10-21 04:59:16 EDT
Testing RHEL 5.3 virtio_net drivers with RHEL5 KVM, I notice that the guest rarely sends packets greater than a page to the host.

Given that we have GSO support, sending 64k GSO packets would be much more efficient.

Herbert points out this fix in 2.6.26:

  [TCP]: Let skbs grow over a page on fast peers

  While testing the virtio-net driver on KVM with TSO I noticed
  that TSO performance with a 1500 MTU is significantly worse
  compared to the performance of non-TSO with a 16436 MTU.  The
  packet dump shows that most of the packets sent are smaller
  than a page.

  Looking at the code this actually is quite obvious as it always
  stop extending the packet if it's the first packet yet to be
  sent and if it's larger than the MSS.  Since each extension is
  bound by the page size, this means that (given a 1500 MTU) we're
  very unlikely to construct packets greater than a page, provided
  that the receiver and the path is fast enough so that packets can
  always be sent immediately.

  The fix is also quite obvious.  The push calls inside the loop
  is just an optimisation so that we don't end up doing all the
  sending at the end of the loop.  Therefore there is no specific
  reason why it has to do so at MSS boundaries.  For TSO, the
  most natural extension of this optimisation is to do the pushing
  once the skb exceeds the TSO size goal.

  This is what the patch does and testing with KVM shows that the
  TSO performance with a 1500 MTU easily surpasses that of a 16436
  MTU and indeed the packet sizes sent are generally larger than

  I don't see any obvious downsides for slower peers or connections,
  but it would be prudent to test this extensively to ensure that
  those cases don't regress.

I've tested this with RHEL5 virtio_net and it gives a >200% increase in the results of a guest->host netperf micro benchmark with 64k send buffers.

The patch is very straightforward and required no backporting.
Comment 2 Mark McLoughlin 2008-10-21 06:16:37 EDT
Created attachment 320995 [details]

Posted to rhkernel-list
Comment 4 RHEL Product and Program Management 2008-10-21 13:27:49 EDT
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
Comment 5 Don Zickus 2008-10-29 12:18:58 EDT
in kernel-2.6.18-121.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5
Comment 8 errata-xmlrpc 2009-01-20 15:16:01 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.