In Kilo (OSP7) prior, oslo.messaging logs a verbose (and scary looking) backtrace when a reply exchange is missing. Moreso, for each message received with missing reply exchange, the code will retry sending for 60 seconds with 0.25s sleep intervals. This means that when the original client really went away, 240 backtraces (60s * 4 tries/s) will be logged per message. This (1) creates a lot of unnecessary I/O, (2) uses a lot of disk space, and (3) misleads (and terrifies) operators into believing severe degredation of service is in progress (maybe, maybe not). This was corrected as part of Liberty (OSP8) here: https://review.openstack.org/#/c/193037/ But that change is unable to be backported to Kilo because it introduces new dependencies (cachetools). Here's the rejected Kilo backport: https://review.openstack.org/#/c/193484/ We should backport just the logging portion of that change.
https://code.engineering.redhat.com/gerrit/#/c/85299/2
python-oslo-messaging-1.8.3-7.el7ost - https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=528968
Code verified on rhos-7. [stack@undercloud-0 ~]$ rpm -qa |grep python-oslo-messaging python-oslo-messaging-1.8.3-7.el7ost.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2017-0158.html