Bug 1225178 - Not capturing events properly from RHOS (RabbitMQ)
Summary: Not capturing events properly from RHOS (RabbitMQ)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers
Version: 5.4.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.4.0
Assignee: Greg Blomquist
QA Contact: Pete Savage
URL:
Whiteboard:
Depends On: 1223976
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-05-26 18:38 UTC by Greg Blomquist
Modified: 2015-06-24 20:39 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 1223976
Environment:
Last Closed: 2015-06-16 13:02:52 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:1100 0 normal SHIPPED_LIVE CFME 5.4.0 bug fixes, and enhancement update 2015-06-16 16:28:42 UTC

Comment 1 CFME Bot 2015-05-26 19:36:22 UTC
New commit detected on cfme/5.4.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=ab30c1e644b21e48ab75264fd84d10bc432e53bf

commit ab30c1e644b21e48ab75264fd84d10bc432e53bf
Author:     Greg Blomquist <gblomqui>
AuthorDate: Tue May 26 12:38:30 2015 -0400
Commit:     Greg Blomquist <gblomqui>
CommitDate: Tue May 26 15:21:35 2015 -0400

    Create unique binding queue names for RabbitMQ
    
    When CFME connects to RabbitMQ to collect OpenStack events, it creates queues to
    bind to the OpenStack services' exchanges.  The queues were named after the
    services to which they were bound.  For example, binding to the "nova" service
    would result in a binding queue called "nova".
    
    If more than one appliance attempted to connect to RabbitMQ to collect OpenStack
    events, only the first appliance to create the binding queue would receive any
    events.
    
    Now, the binding queue is named after the appliance connecting to the RabbitMQ
    service.  The new binding queue name will look like "miq-<host|ip>-<exchange>"
    
     * e.g.: "miq-10.10.10.10-nova"
    
    This allows for two things:  individual appliances will get their own binding
    queue per service, and administrators will be able to tell which binding queues
    belong to which appliances.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1225178

 lib/openstack/amqp/openstack_rabbit_event_monitor.rb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comment 3 Pete Savage 2015-06-03 06:17:47 UTC
Verified in 5.4.0.4

Comment 4 CFME Bot 2015-06-04 18:01:50 UTC
New commit detected on cfme/5.4.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=0b6cb3a145a4bc2fa994f2a8a9f2d91a2b9d8a24

commit 0b6cb3a145a4bc2fa994f2a8a9f2d91a2b9d8a24
Author:     Greg Blomquist <gblomqui>
AuthorDate: Wed Jun 3 16:16:05 2015 -0400
Commit:     Greg Blomquist <gblomqui>
CommitDate: Thu Jun 4 13:52:33 2015 -0400

    Fix custom naming for AMQP binding queues
    
    The original fixes for the bugs linked below used the OpenStack server's IP
    address as the hostname information in the binding queue name.  This meant that
    any two appliances that attempted to connect to a single OpenStack env would
    create the same named binding queues.
    
    This fix uses the appliance's ip address in the binding queue name, making the
    name of the binding queue unique per appliance.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1225178
    https://bugzilla.redhat.com/show_bug.cgi?id=1225173
    
    Upstream PR: https://github.com/ManageIQ/manageiq/pull/3050

 lib/openstack/amqp/openstack_qpid_event_monitor.rb             | 10 ++++++----
 lib/openstack/amqp/openstack_qpid_receiver.rb                  |  5 +++--
 lib/openstack/amqp/openstack_rabbit_event_monitor.rb           |  6 ++++--
 lib/spec/openstack/amqp/openstack_qpid_event_monitor_spec.rb   |  2 +-
 lib/spec/openstack/amqp/openstack_qpid_receiver_spec.rb        |  2 +-
 lib/spec/openstack/amqp/openstack_rabbit_event_monitor_spec.rb |  2 +-
 vmdb/lib/workers/mixins/event_catcher_openstack_mixin.rb       |  4 +++-
 7 files changed, 19 insertions(+), 12 deletions(-)

Comment 6 errata-xmlrpc 2015-06-16 13:02:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1100.html

Comment 7 CFME Bot 2015-06-24 20:39:26 UTC
New commit detected on cfme/5.4.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=bdc878afed77d8dcb539191dadb1d05dbb875421

commit bdc878afed77d8dcb539191dadb1d05dbb875421
Author:     Greg Blomquist <gblomqui>
AuthorDate: Tue Jun 23 13:30:07 2015 -0400
Commit:     Greg Blomquist <gblomqui>
CommitDate: Wed Jun 24 16:05:37 2015 -0400

    Include miq_server when retrieving worker
    
    To try to make the way the OpenStack event catcher creates binding queues
    work a little better, the appliance's IP address was looked up and used as part
    of the binding queue's name.
    
    However, there were a couple of things working against this fix.  First, the
    appliance's IP address was not readily available to the worker process.  Second,
    ManageIQ has a DB connection pool with only one connection.  And, threads (i.e.,
    where event catcher workers do all their work) that attempt to run queries are
    opening a new DB connection.
    
    The original fix never actually tried opening the a new connection.  Instead, it
    was perfectly happy to get back a nil value for the appliance and try to lookup
    Nil#ipaddress.
    
    This fix gets around this problem by throwing the appliance record (miq_server,
    actually) into an ivar and making that available to the thread.  This keeps the
    thread from having to query for the miq_server, while still giving it access to
    the MiqServer#ipaddress.
    
    Original PR:
    https://github.com/ManageIQ/manageiq/pull/3050
    
    Fixes:
    https://bugzilla.redhat.com/show_bug.cgi?id=1233798
    
    References:
    https://bugzilla.redhat.com/show_bug.cgi?id=1225173
    https://bugzilla.redhat.com/show_bug.cgi?id=1225178

 vmdb/lib/workers/mixins/event_catcher_openstack_mixin.rb |  2 +-
 vmdb/lib/workers/worker_base.rb                          | 13 +++++++------
 2 files changed, 8 insertions(+), 7 deletions(-)

Comment 8 CFME Bot 2015-06-24 20:39:39 UTC
New commit detected on cfme/5.4.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=36efb1c837f179d701422ea3a4aae77e84c8d6bd

commit 36efb1c837f179d701422ea3a4aae77e84c8d6bd
Merge: 3ae711d bdc878a
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Wed Jun 24 16:37:18 2015 -0400
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Wed Jun 24 16:37:18 2015 -0400

    Merge branch 'bz1233798-backport_worker_base_for_bz1232484' into '5.4.z'
    
    Include miq_server when retrieving worker
    
    To try to make the way the OpenStack event catcher creates binding queues work a little better, the appliance's IP address was looked up and used as part
    of the binding queue's name.
    
    However, there were a couple of things working against this fix.  First, the appliance's IP address was not readily available to the worker process.  Second,
    ManageIQ has a DB connection pool with only one connection.  And, threads (i.e., where event catcher workers do all their work) that attempt to run queries are
    opening a new DB connection.
    
    The original fix never actually tried opening the a new connection.  Instead, it was perfectly happy to get back a nil value for the appliance and try to lookup
    Nil#ipaddress.
    
    This fix gets around this problem by throwing the appliance record (miq_server, actually) into an ivar and making that available to the thread.  This keeps the
    thread from having to query for the miq_server, while still giving it access to the MiqServer#ipaddress.
    
    Upstream PR:
    https://github.com/ManageIQ/manageiq/pull/3244
    
    No cherry-pick conflicts
    
    Fixes:
    https://bugzilla.redhat.com/show_bug.cgi?id=1233798
    
    References:
    https://bugzilla.redhat.com/show_bug.cgi?id=1225173
    https://bugzilla.redhat.com/show_bug.cgi?id=1225178
    
    See merge request !145

 vmdb/lib/workers/mixins/event_catcher_openstack_mixin.rb |  2 +-
 vmdb/lib/workers/worker_base.rb                          | 13 +++++++------
 2 files changed, 8 insertions(+), 7 deletions(-)


Note You need to log in before you can comment on or make changes to this bug.