Bug 872696

Summary: QMF queue events do not propagate to HA backups
Product: Red Hat Enterprise MRG Reporter: Jason Dillaman <jdillama>
Component: qpid-cppAssignee: Alan Conway <aconway>
Status: CLOSED CURRENTRELEASE QA Contact: MRG Quality Engineering <mrgqe-bugs>
Severity: high Docs Contact:
Priority: high    
Version: DevelopmentCC: aconway, esammons, jross, lzhaldyb, mcressma
Target Milestone: 2.3Keywords: OtherQA, Patch
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qpid-cpp-0.18-9 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-03-19 16:39:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 698367    
Attachments:
Description Flags
Quick patch to prevent duplicate output tasks
none
Quick patch to prevent duplicate output tasks none

Description Jason Dillaman 2012-11-02 18:45:43 UTC 
Description of problem:
After unsubscribing the last consumer from an auto-delete queue, the delete event was not propagated to the backup nodes.  Additionally, after re-creating a queue with the same name, the the backup did not receive the declare event.  The original queue declare event was correctly propagated to the backup.

Version-Release number of selected component (if applicable):
qpid-cpp-0.18-6

How reproducible:
100%

Steps to Reproduce:
1. Subscribe to an auto-delete queue
2. Unsubscribe from the auto-delete queue
  
Actual results:
The backup node did not receive the delete event.

Expected results:
The backup node should receive the delete event.
Comment 1 Jason Dillaman 2012-11-05 15:48:53 UTC 
After additional investigation, the issue appears to have been exposed due to channel errors that resulted in the link reconnecting (see BZ873347).  The QMF channel did not have any errors in the log but somehow the backup missed a queue creation event and thus failed to replicate a queue.
Comment 2 Jason Dillaman 2012-11-07 17:55:55 UTC 
Issue is not 100% reproducible, but has happened often across numerous machines.
Comment 3 Jason Dillaman 2012-11-08 19:47:04 UTC 
Created attachment 641055 [details]
Quick patch to prevent duplicate output tasks
Comment 4 Jason Dillaman 2012-11-08 19:49:00 UTC 
Discovered that the inter-broker link had hundreds of thousands of enqueued OutputTasks -- representing only a few unique consumers.  There should only be only a single output task for a given consumer.  This appears to have stalled the delivery of QMF messages to the backup broker.
Comment 5 Jason Dillaman 2012-11-08 21:11:33 UTC 
Created attachment 641080 [details]
Quick patch to prevent duplicate output tasks
Comment 6 Alan Conway 2012-11-09 17:06:35 UTC 
Fix committed: http://mrg1.lab.bos.redhat.com/cgit/qpid.git/commit/?h=0.18-mrg-aconway-ha-1&id=dc05b45d4582373d69ef5d378073ed1957b29307

On branch: http://mrg1.lab.bos.redhat.com/cgit/qpid.git/log/?h=0.18-mrg-aconway-ha-1