Bug 1378324 - Nova api logging "Too many heartbeats missed" message in logs [NEEDINFO]
Summary: Nova api logging "Too many heartbeats missed" message in logs
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 7.0 (Kilo)
Assignee: Eoghan Glynn
QA Contact: Prasanth Anbalagan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-22 07:02 UTC by PURANDHAR SAIRAM MANNIDI
Modified: 2019-12-16 06:51 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-23 09:50:37 UTC
Target Upstream Version:
pmannidi: needinfo? (eglynn)


Attachments (Terms of Use)

Description PURANDHAR SAIRAM MANNIDI 2016-09-22 07:02:04 UTC
Description of problem:
Nova api logging "Too many heartbeats missed" message in logs. Could not access the resources for sometime

Customer wanted to check whether the configuration parameter 'heartbeat_timeout_threshold' should be enabled or disabled in a production setup.
From http://docs.openstack.org/kilo/config-reference/content/nova-conf-changes-kilo.html it has been marked EXPERIMENTAL

[oslo_messaging_rabbit] heartbeat_timeout_threshold = 0	(IntOpt) Number of seconds after which the Rabbit broker is considered down if heartbeat's keep-alive fails (0 disables the heartbeat, >0 enables it. Enabling heartbeats requires kombu>=3.0.7 and amqp>=1.4.0). EXPERIMENTAL

Version-Release number of selected component (if applicable):
RHOSP 7.0

How reproducible:
Frequently

Comment 3 Stephen Finucane 2016-09-23 09:50:37 UTC
Having looked at this issue upstream [1], it appears the feature 'heartbeat_timeout_threshold' configuration option in 'oslo.messaging' was set to "disabled-by-default" shortly after being introduced [2] due to issues with various libraries [3]. However, all versions of 'oslo.messaging' used by Kilo [4] were released before this bugfix was included.

Backporting support for newer versions of 'oslo.messaging' simply to remove this message is not practical, thus, the easiest fix is to simply disable this feature manually. You can do this safe in the knowledge that timeouts will still occur when services go down thanks to TCP timeouts.

[1] https://bugs.launchpad.net/oslo.messaging/+bug/1436769
[2] https://bugs.launchpad.net/oslo.messaging/+bug/1436769/comments/12
[3] https://bugs.launchpad.net/oslo.messaging/+bug/1436769/comments/15
[4] https://github.com/openstack/nova/blob/2015.1.4/requirements.txt#L39

Comment 4 awaugama 2017-08-30 17:52:20 UTC
WONTFIX/NOTABUG therefore QE Won't automate


Note You need to log in before you can comment on or make changes to this bug.