Bug 768114

Summary: Enable TCP_NODELAY by default
Product: Red Hat Enterprise MRG Reporter: Justin Ross <jross>
Component: qpid-cppAssignee: Andrew Stitcher <astitcher>
Status: CLOSED ERRATA QA Contact: Ernie <eallen>
Severity: low Docs Contact:
Priority: medium    
Version: 2.3CC: astitcher, eallen, iboverma, jmorgan, jross, kim.vdriet, lzhaldyb, pmoravec
Target Milestone: 3.0Keywords: Improvement, Performance
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qpid-qmf-0.22-7 Doc Type: Enhancement
Doc Text:
The qpid C++ clients and broker now deactivate the Nagle algorithm by default. The Nagle algorithm improves the connection bandwidth by sacrificing some latency. Under most circumstances, using qpid will achieve identical or improved performance with the Nagle algorithm deactivated. Where this is not the case, the algorithm can be enabled by specifying the `--tcp-nodelay=no` option on the command line for the broker, or `tcp_nodelay=no` in the configuration file. This enhancement should result in a performance improvement for most user configurations. This feature is documented in the "General Broker Options" section of the Messaging Installation and Configuration Guide.
Story Points: ---
Clone Of:
: 961008 (view as bug list) Environment:
Last Closed: 2014-09-24 15:03:39 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 894375, 961009    
Bug Blocks: 794748, 961008    

Description Justin Ross 2011-12-15 19:30:17 UTC
See https://issues.apache.org/jira/browse/QPID-3689

Comment 1 Pavel Moravec 2013-01-09 12:31:36 UTC
When implementing this improvement, please make --tcp-nodelay option similar to --auth (i.e. one can specify if the option is "on" or "off"). Current behavior is quite misleading:

$ qpidd --trace --tcp-nodelay=no
2013-01-09 12:24:37 [System] debug Exception constructed: Error in command line options: extra parameter in 'tcp-nodelay'

$ qpidd --trace --tcp-nodelay no
..
(tcp_nodelay is switched _ON_)

$ qpidd --trace --tcp-nodelay false
..
(tcp_nodelay is switched _ON_)

$ qpidd --trace --tcp-nodelay=false
2013-01-09 12:29:37 [System] debug Exception constructed: Error in command line options: extra parameter in 'tcp-nodelay'

$ qpidd --trace
..
(tcp_nodelay is switched off)


Isn't it enough to apply this "patch" for this?

--- ./src/qpid/broker/Broker.cpp_orig	2013-01-09 12:31:22.599030546 +0100
+++ ./src/qpid/broker/Broker.cpp_new	2013-01-09 12:31:44.028904499 +0100
@@ -158,7 +158,7 @@ Broker::Options::Options(const std::stri
         ("auth", optValue(auth, "yes|no"), "Enable authentication, if disabled all incoming connections will be trusted")
         ("realm", optValue(realm, "REALM"), "Use the given realm when performing authentication")
         ("default-queue-limit", optValue(queueLimit, "BYTES"), "Default maximum size for queues (in bytes)")
-        ("tcp-nodelay", optValue(tcpNoDelay), "Set TCP_NODELAY on TCP connections")
+        ("tcp-nodelay", optValue(tcpNoDelay, "yes|no"), "Set TCP_NODELAY on TCP connections")
         ("require-encryption", optValue(requireEncrypted), "Only accept connections that are encrypted")
         ("known-hosts-url", optValue(knownHosts, "URL or 'none'"), "URL to send as 'known-hosts' to clients ('none' implies empty list)")
         ("sasl-config", optValue(saslConfigPath, "DIR"), "gets sasl config info from nonstandard location")

Comment 2 Andrew Stitcher 2013-01-10 15:21:46 UTC
(In reply to comment #1)
> ...
> Isn't it enough to apply this "patch" for this?
> ...
> -        ("tcp-nodelay", optValue(tcpNoDelay), "Set TCP_NODELAY on TCP
> connections")
> +        ("tcp-nodelay", optValue(tcpNoDelay, "yes|no"), "Set TCP_NODELAY on
> TCP connections")

What you changed in there is only the documentation string that appears in the usage message - sorry, if only it was that simple.

I think this request would better be tracked as a separate feature request as it is related but separate, Ie it may well be a good idea with a different priority from the original.

I say this because I think this is a usability request whereas the original is a performance related request.

However once the default is changed we will need a way to turn tcp nodelay off so this is probably a prerequisite.

Comment 3 Pavel Moravec 2013-01-11 15:37:49 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > ...
> > Isn't it enough to apply this "patch" for this?
> > ...
> > -        ("tcp-nodelay", optValue(tcpNoDelay), "Set TCP_NODELAY on TCP
> > connections")
> > +        ("tcp-nodelay", optValue(tcpNoDelay, "yes|no"), "Set TCP_NODELAY on
> > TCP connections")
> 
> What you changed in there is only the documentation string that appears in
> the usage message - sorry, if only it was that simple.
--auth option is managed the same, I already tested it

> 
> I think this request would better be tracked as a separate feature request
> as it is related but separate, Ie it may well be a good idea with a
> different priority from the original.
See bz894375.

Comment 4 Andrew Stitcher 2013-04-17 22:51:36 UTC
This is now fixed on the trunk of qpid (for 0.23) in r1469088.

Comment 5 Ernie 2013-07-22 18:56:58 UTC
Verified. 
rhel 6.4  x32 and x64
qpid-cpp-server-0.22-7.el6.x86_64
cyrus-sasl-devel-2.1.23-13.el6_3.1.x86_64
saslwrapper-devel-0.22-3.el6.x86_64
qpid-qmf-devel-0.22-6.el6.x86_64
cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64
qpid-proton-c-devel-0.4-2.2.el6.x86_64
python-saslwrapper-0.22-3.el6.x86_64
qpid-cpp-client-0.22-7.el6.x86_64
qpid-cpp-client-devel-0.22-7.el6.x86_64
cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64
qpid-tools-0.22-3.el6.noarch
qpid-proton-c-0.4-2.2.el6.x86_64
saslwrapper-0.22-3.el6.x86_64
cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64
qpid-snmpd-1.0.0-12.el6.x86_64
qpid-qmf-0.22-6.el6.x86_64
cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64
python-qpid-0.22-4.el6.noarch
cyrus-sasl-2.1.23-13.el6_3.1.x86_64
python-qpid-qmf-0.22-6.el6.x86_64

qpidd --trace 2>&1 | grep TCP_NODELAY
outputs "2013-07-22 14:36:37 [Network] info Set TCP_NODELAY on connection to [::1]:34284"
when a client attempts a connection.

Comment 10 errata-xmlrpc 2014-09-24 15:03:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-1296.html