Bug 840907 - qpidd connection counting algorithm not strictly counting not-negotiated connections.
Summary: qpidd connection counting algorithm not strictly counting not-negotiated conn...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp
Version: Development
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: 2.2
: ---
Assignee: messaging-bugs
QA Contact: MRG Quality Engineering
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-07-17 14:47 UTC by Frantisek Reznicek
Modified: 2015-11-16 01:14 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-07-23 15:01:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 683711 1 None None None 2021-01-20 06:05:38 UTC

Internal Links: 683711

Description Frantisek Reznicek 2012-07-17 14:47:47 UTC
Description of problem:

Qpidd is (and always was) counting only connections with complete handshake.
This defect tries to revise that bahavior.

Bug 683711 and bug 609685 improve qpidd behavior to be more restrictive and minimize possible DoS situations.
There is still possible to block qpidd operation within bug 609685 max-negotiate-time.

Later experiment shows that connection counter increases when handshake is complete. This may lead to situations when attacker wastes resources (mostly file handles).

Current connection restriction features:
  --max-connections N (500)         The maximum combined number of connections 
  --max-connections-per-user N (0)  The maximum number of connections allowed 
  --max-connections-per-ip N (0)    The maximum number of connections allowed 

are able to limit number of connections, but only those which are already established.

The only mechanism which discards connections with incomplete handshake is:
  --max-negotiate-time MilliSeconds (2000) Maximum time a protocol negotiation


I propose to add another switch or change behavior of --max-connections the way that every plain socket (telnet or so) increase counter.

Currently it is possible to establish more not-negotiated connections (telnet) than --max-connections N limit. See details below...


If this view is not supported by developers, then I propose to explicitly change documentation of the --max-connections* switches to include more explicit description what connections are counted... ('established connections' or so)


Version-Release number of selected component (if applicable):
  python-qpid-0.14-9.el6_3.noarch
  python-qpid-qmf-0.14-10.el6_3.i686
  qpid-cpp-client-0.14-18.el6_3.i686
  qpid-cpp-client-devel-0.14-18.el6_3.i686
  qpid-cpp-client-devel-docs-0.14-16.el6.noarch
  qpid-cpp-client-rdma-0.14-18.el6_3.i686
  qpid-cpp-client-ssl-0.14-18.el6_3.i686
  qpid-cpp-server-0.14-18.el6_3.i686
  qpid-cpp-server-cluster-0.14-18.el6_3.i686
  qpid-cpp-server-devel-0.14-18.el6_3.i686
  qpid-cpp-server-rdma-0.14-18.el6_3.i686
  qpid-cpp-server-ssl-0.14-18.el6_3.i686
  qpid-cpp-server-store-0.14-18.el6_3.i686
  qpid-cpp-server-xml-0.14-18.el6_3.i686
  qpid-java-client-0.14-3.el6.noarch
  qpid-java-common-0.14-3.el6.noarch
  qpid-java-example-0.14-3.el6.noarch
  qpid-jca-0.14-9.el6.noarch
  qpid-jca-xarecovery-0.14-9.el6.noarch
  qpid-jca-zip-0.14-9.el6.noarch
  qpid-qmf-0.14-10.el6_3.i686
  qpid-qmf-devel-0.14-10.el6_3.i686
  qpid-tests-0.14-1.el6_2.noarch
  qpid-tools-0.14-3.el6_3.noarch
  rh-qpid-cpp-tests-0.14-18.el6_3.i686
  ruby-qpid-qmf-0.14-10.el6_3.i686


How reproducible:
100%

Steps to Reproduce:
  See Additional info.

Actual results:
  A user can short term (defined by --max-negotiate-time) violate the --max-connections N limit.

Expected results:
  No user should be able to violate the --max-connections N limit at all.

Additional info:

Demonstration of ability to create more than 25 (max-connections) connections (with incomplete hanshake).

  [root@dhcp-37-178 bz609685]# echo 'acl allow all all' > qpidd.acl
  [root@dhcp-37-178 bz609685]# qpidd --data-dir dd -p 5672 --log-enable=info+ --log-to-file qpidd.log --max-connections 25 -d --max-negotiate-time 10000 --acl-file ${PWD}/qpidd.acl
  [root@dhcp-37-178 bz609685]# !netst
  netstat -nlp | grep qpidd
  tcp        0      0 0.0.0.0:5672                0.0.0.0:*                   LISTEN      11079/qpidd
  tcp        0      0 :::5672                     :::*                        LISTEN      11079/qpidd
  [root@dhcp-37-178 bz609685]# ./connector2.py -b 10.34.37.178:5672 -c 50
  10
  20
  connection-forced: User connection denied by configured limit(320)
  Traceback (most recent call last):
    File "./connector2.py", line 80, in <module>
      conn.open()
    File "<string>", line 6, in open
    File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 244, in open
      self.attach()
    File "<string>", line 6, in attach
    File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 262, in attach
      self._ewait(lambda: self._transport_connected and not self._unlinked())
    File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 197, in _ewait
      self.check_error()
    File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 190, in check_error
      raise self.error
  ConnectionError: connection-forced: User connection denied by configured limit(320)
  Opened connections: 25
  [root@dhcp-37-178 bz609685]# 
  [root@dhcp-37-178 bz609685]# ls -la /proc/`pidof qpidd`/fd | grep -c socket
  2
  [root@dhcp-37-178 bz609685]# for i in `seq 30`; do ls -la /proc/`pidof qpidd`/fd | grep -c socket; expect -f ./telnet.exp telnet 10.34.37.178 5672 &>/dev/null & done
  3
  [1] 11101
  4
  [2] 11108
  5
  ...
  [29] 11297
  32
  [30] 11304
  [root@dhcp-37-178 bz609685]# ls -la /proc/`pidof qpidd`/fd | grep -c socket33
  [root@dhcp-37-178 bz609685]# 
  [1]   Exit 1                  expect -f ./telnet.exp telnet 10.34.37.178 5672 &>/dev/null
  [2]   Exit 1                  expect -f ./telnet.exp telnet 10.34.37.178 5672 &>/dev/null
  ...
  [28]   Exit 1                  expect -f ./telnet.exp telnet 10.34.37.178 5672 &>/dev/null
  [29]-  Exit 1                  expect -f ./telnet.exp telnet 10.34.37.178 5672 &>/dev/null
  [30]+  Exit 1                  expect -f ./telnet.exp telnet 10.34.37.178 5672 &>/dev/null
  [root@dhcp-37-178 bz609685]# ls -la /proc/`pidof qpidd`/fd | grep -c socket
  2

Comment 1 Andrew Stitcher 2012-07-17 17:30:45 UTC
My view here is that the lifecycle of the Connection object is wrong:

Really it should start as soon as the connection is accepted and note that it is in an "UNNEGOTIATED/UNAUTHENTICATED" state. Then when it has finished the protocal negotaion/user authentication it should change states appropriately until it is in the "WORKING" state.

This working state would correspond to the current lifecycle of the object.

[The actual names of states were made up and are probably not the best, probably there would be other states too]

The we should apply the limits to Connection creation in the new lifecycle which would help rather better with DoS situations.

Also it is not clear to me that the max-connections setting is the most useful thing (except for backwards configuration compatibility). Now that we have more granular settings it may not be so useful. Perhaps a maximum unauthenticated connections would be more useful - we'd have to put anonymous connections inthere somewhere too.


Note You need to log in before you can comment on or make changes to this bug.