Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1054249 - Thread consuming qpid messages can die silently
Thread consuming qpid messages can die silently
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron (Show other bugs)
3.0
Unspecified Unspecified
urgent Severity urgent
: z1
: 4.0
Assigned To: Alan Pevec
Roey Dekel
: TestBlocker, ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2014-01-16 09:03 EST by Russell Bryant
Modified: 2016-04-26 15:30 EDT (History)
10 users (show)

See Also:
Fixed In Version: openstack-neutron-2013.2.1-4.el6ost
Doc Type: Bug Fix
Doc Text:
Previously, thread consuming QPID messages would be killed silently by unhandled errors, thus resulting in isolating the component from the rest of the system. With this update, consuming threads are made more resilient to errors by ensuring they do not die on an unhandled error. It will now log the error and retry the consuming.
Story Points: ---
Clone Of: 1050214
Environment:
Last Closed: 2014-01-22 13:33:00 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1189711 None None None Never
Red Hat Product Errata RHSA-2014:0091 normal SHIPPED_LIVE Moderate: openstack-neutron security, bug fix, and enhancement update 2014-01-22 18:31:15 EST

  None (edit)
Description Russell Bryant 2014-01-16 09:03:20 EST
+++ This bug was initially created as a clone of Bug #1050214 +++

+++ This bug was initially created as a clone of Bug #1050213 +++

The code for receiving and processing qpid messages runs in its own greenthread.  Unfortunately, there is a code path that if it raises an exception, the greenthread will die silently, without any entry in the log file.

In particular, the code in question is:

https://git.openstack.org/cgit/openstack/nova/tree/nova/openstack/common/rpc/impl_qpid.py?h=stable/grizzly#n468

if self.session.next_receiver() raises an exception other than qpid's Empty or ConnectionError exceptions, the thread will die and no more messages will be received.

The fix is to backport the portion of the following change that applies to impl_qpid.  That includes the new decorator in excutils and the change to impl_qpid.py.

--- Additional comment from Russell Bryant on 2014-01-08 16:12:17 EST ---

The change that needs to be backported is: https://review.openstack.org/#/c/32235/13

I also attached a version of the backport for nova to bug 1050213



This is also needed in RHOS 4.0 for Neutron.
Comment 3 Roey Dekel 2014-01-19 10:25:38 EST
Verified on Havana with :

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
puddle: 2014-01-16.1
python-qpid-0.14-11.el6_3.noarch
qpid-cpp-client-0.14-22.el6_3.x86_64
qpid-cpp-server-0.14-22.el6_3.x86_64
openstack-neutron-openvswitch-2013.2.1-4.el6ost.noarch
python-neutronclient-2.3.1-2.el6ost.noarch
python-neutron-2013.2.1-4.el6ost.noarch
openstack-neutron-2013.2.1-4.el6ost.noarch

Results:
--------
Code is implemented. (Not being verified functionality)
Comment 5 errata-xmlrpc 2014-01-22 13:33:00 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2014-0091.html

Note You need to log in before you can comment on or make changes to this bug.