Rubygem-staypuft: HA: amqp is still set to A/P hot-standby mode. Should be A/A. its timeout must be a lot higher. (900m or so) due to rabbitmq oslo driver missing a rabbitmq_heartbeat feature. This is a known upstream limitation/bug (already reported). Or you will end up with services reconnecting a gazillion times to rabbit.
(In reply to Leonid Natapov from comment #0) > Rubygem-staypuft: HA: amqp is still set to A/P hot-standby mode. Should be > A/A. A/P hot standby mode is absolutely valid for when the user has selected qpid as the messaging layer. Did you mean for $subject of the bug to read: HA: When using RabbitMQ, amqp is still set to A/P hot-standby mode and its timout must be a lot higher.
(In reply to Leonid Natapov from comment #0) > Rubygem-staypuft: HA: amqp is still set to A/P hot-standby mode. Should be > A/A. > its timeout must be a lot higher. (900m or so) due to rabbitmq oslo driver > missing a rabbitmq_heartbeat feature. This is a known upstream > limitation/bug (already reported). Or you will end up with services > reconnecting a gazillion times to rabbit. Are you referring to the haproxy timeout here, or rabbit itself (or both)? If both, I would think we want haproxy to not have exactly the same timeout value as the service it is in front of, so what are the values we want to set here?
(In reply to Jason Guiditta from comment #2) > (In reply to Leonid Natapov from comment #0) > > Rubygem-staypuft: HA: amqp is still set to A/P hot-standby mode. Should be > > A/A. > > its timeout must be a lot higher. (900m or so) due to rabbitmq oslo driver > > missing a rabbitmq_heartbeat feature. This is a known upstream > > limitation/bug (already reported). Or you will end up with services > > reconnecting a gazillion times to rabbit. > > Are you referring to the haproxy timeout here, or rabbit itself (or both)? > If both, I would think we want haproxy to not have exactly the same timeout > value as the service it is in front of, so what are the values we want to > set here? haproxy timeout.
Eck, any updates?
The A/A bit has been fixed, see bug 1121185. The timeout bit is more involved. The short of it is to read through this upstream bug: https://bugs.launchpad.net/oslo.messaging/+bug/856764/ Setting the haproxy timeout higher isn't necessarily going to help things, and may hurt under some circumstances. I need to do some more legwork to examine all the different versions of components we have for RHOS5 (amqplib, python-kombu, rabbitmq-server) and evaluate the state of oslo.messaging, which components are using it, and which are off doing their own thing. It's a tangled mess.
Any updates on this?
I am going to update the backend timeout to 900m to match: https://github.com/fabbione/rhos-ha-deploy/blob/master/rhos5-rhel7/mrgcloud-setup/RHOS-RHEL-HA-how-to-mrgcloud-rhos5-on-rhel7-lb-latest.txt#L41 The oslo stuff is being tracked elsewhere. Pretend I didn't mention it here.
I should have said, update both the backend and client timeouts to 900m.
https://github.com/redhat-openstack/astapor/pull/365
*** Bug 1142915 has been marked as a duplicate of this bug. ***
listen amqp bind 192.168.0.36:5672 mode tcp option tcplog timeout client 900m timeout server 900m
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2014-1350.html