Bug 595914

Summary: QMF python console will occasionally crash if a get request times out.
Product: Red Hat Enterprise MRG Reporter: Ken Giusti <kgiusti>
Component: qpid-qmfAssignee: Ken Giusti <kgiusti>
Status: CLOSED ERRATA QA Contact: Jan Sarenik <jsarenik>
Severity: high Docs Contact:
Priority: high    
Version: betaCC: freznice, jross, jsarenik
Target Milestone: 1.3   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-10-20 11:29:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
tool to create many queues and bindings to repro the bug. none

Description Ken Giusti 2010-05-25 22:02:25 UTC
Created attachment 416572 [details]
tool to create many queues and bindings to repro the bug.

Description of problem:

On a broker configured with a very large # of queues bound to amq.direct, if the python qmf getObject() method times out, occasionally a crash occurs during the exception:

# qpid-config exchanges -b amq.direct
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib64/python2.4/threading.py", line 442, in __bootstrap
    self.run()
  File "/usr/lib64/python2.4/threading.py", line 422, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.4/site-packages/qpid/queue.py", line 84, in run
    self.listener(o)
  File "/usr/lib/python2.4/site-packages/qmf/console.py", line 2366, in _v2Cb
   
self.amqpSession.channel.session_completed(self.amqpSession.receiver._completed)
AttributeError: 'NoneType' object has no attribute 'session_completed'

Failed: Exception: Timed out after 60 seconds    


Version-Release number of selected component (if applicable):


How reproducible:

Occasional


Steps to Reproduce:
1. Configure 50K+ queues
2. use qpid-config to dump the bindings
3. If a timeout exception is hit, the crash may occur.
  
Actual results:


Expected results:

Timeout, but no crash.


Additional info:

Comment 1 Frantisek Reznicek 2010-06-08 12:51:28 UTC
Above mentioned AttribError:
AttributeError: 'NoneType' object has no attribute 'session_completed'

is very slowing the test and MRG/M verification down and makes evaluation of correct run more difficult.


I can see this behavior in my virtual machines even when simple receiver / sender tries to access non existent qpid address (high level API)

So I believe there should be more fundamental problem there.
# service qpidd start
# qpid-stat -b
Brokers
  broker          cluster       uptime  conn  sess  exch  queue
  ===============================================================
  localhost:5672  <standalone>  7m 10s     1     1     8     3
# qpid-stat -b
Brokers
  broker          cluster       uptime  conn  sess  exch  queue
  ===============================================================
  localhost:5672  <standalone>  7m 10s     1     1     8     3
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python2.3/threading.py", line 436, in __bootstrap
    self.run()
  File "/usr/lib/python2.3/threading.py", line 416, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.3/site-packages/qpid/queue.py", line 84, in run
    self.listener(o)
  File "/usr/lib/python2.3/site-packages/qmf/console.py", line 2437, in _v2Cb
    self.amqpSession.channel.session_completed(self.amqpSession.receiver._completed)
AttributeError: 'NoneType' object has no attribute 'session_completed'

# qpid-config
Total Exchanges: 8
          topic: 3
        headers: 1
         fanout: 1
         direct: 3

   Total Queues: 3
        durable: 0
    non-durable: 3

Ken,
can you rise the priority of the bug to at least medium, please?

Comment 2 Frantisek Reznicek 2010-06-08 12:52:34 UTC
raising NEEDINFO

Comment 3 Ken Giusti 2010-06-08 14:24:25 UTC
Bumping priority - affects others (grid, Justin, etc).

Comment 4 Ken Giusti 2010-06-08 16:43:58 UTC
Up stream fix: Committed revision 952731.

Comment 6 Jan Sarenik 2010-06-23 10:55:25 UTC
Current behavior is that only the message is printed if the timeout
is reached:

  # qpid-config
  Failed: Exception: Timed out after 60 seconds

Verified on qmf-0.7.946106-4.el5