Bug 1323834 - backport "Use expiring queues instead of auto-delete ones in RabbitMQ driver" to oslo.messaging in OSP8
Summary: backport "Use expiring queues instead of auto-delete ones in RabbitMQ driver"...
Keywords:
Status: CLOSED DUPLICATE of bug 1310157
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-oslo-messaging
Version: 8.0 (Liberty)
Hardware: Unspecified
OS: Linux
unspecified
high
Target Milestone: ---
: 8.0 (Liberty)
Assignee: Victor Stinner
QA Contact: Udi Shkalim
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-04 21:46 UTC by Zhen Qin
Modified: 2019-02-17 12:14 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-04-06 14:41:17 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 279561 0 None MERGED Kombu: make reply and fanout queues expire instead of auto-delete 2020-06-22 07:22:07 UTC

Description Zhen Qin 2016-04-04 21:46:11 UTC
Description of problem:
If reconnection occurs between openstack components and rabbitmq, for example, triggered by the server reboot, there are some queues in rabbitmq become orphaned (a.k.a., no consumer consume them), but they still accumulate RPC or fanout messages, which cause possible memory pressure.

There are upstream fix to address this issue by setting TTL for direct and fanout queues instead of relying on auto_delete flag:
https://review.openstack.org/#/c/243845/
(it's has been backported to liberty: https://review.openstack.org/#/c/279561/)
https://review.openstack.org/#/c/292313/

The last patch is the latest, which changes the default value of rabbit_transient_queues_ttl to 30mins.

Version-Release number of selected component (if applicable):
python-oslo-messaging-2.5.0-4.el7ost.src.rpm

How reproducible:
Give rabbitmq node A has most master queues running, power off or restart the server where rabbitmq node A runs. Or following the reproduce steps in following bug reports:
https://bugs.launchpad.net/oslo.messaging/+bug/1515278
or
https://bugs.launchpad.net/oslo.messaging/+bug/1495568

Steps to Reproduce:
1.
2.
3.

Actual results:
Orphaned queues generated in rabbitmq, which can be verified by running 'rabbitmqctl list_queues name messages consumers' command.

Expected results:
no orphaned queues after a short period.

Additional info:
Related bug report in upstream:
https://bugs.launchpad.net/oslo.messaging/+bug/1515278
https://bugs.launchpad.net/oslo.messaging/+bug/1495568
https://bugs.launchpad.net/oslo.messaging/+bug/1545002

Comment 2 John Eckersberg 2016-04-06 14:41:17 UTC

*** This bug has been marked as a duplicate of bug 1310157 ***


Note You need to log in before you can comment on or make changes to this bug.