Bug 1322609

Summary: Log handler repeatedly crashes
Product: Red Hat OpenStack Reporter: John Eckersberg <jeckersb>
Component: erlangAssignee: John Eckersberg <jeckersb>
Status: CLOSED ERRATA QA Contact: Asaf Hirshberg <ahirshbe>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 8.0 (Liberty)CC: apevec, binarin, draganHR, ealcaniz, fdinitto, jeckersb, jschluet, lhh, oblaut, sclewis, ushkalim, xiaoxwan
Target Milestone: gaKeywords: Regression, ZStream
Target Release: 8.0 (Liberty)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: erlang-R16B-03.10min.9.el7ost Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1324185 1324922 (view as bug list) Environment:
Last Closed: 2016-04-15 13:48:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1324185, 1324922    

Description John Eckersberg 2016-03-30 21:26:23 UTC
Starting with erlang-erts-R16B-03.10min.6.el7ost.x86_64, the log handler repeatedly crashes and fills up the rabbitmq startup_log with entries like:

Event crashed log handler:
{info_msg,<0.1719.0>,
          {<0.1832.0>,"Mirrored ~s: Adding mirror on node ~p: ~p~n",
           ["queue 'l3_agent_fanout_0f6bc20f4c54484f9de482cd6d83a15a' in vhost '/'",
            'rabbit@overcloud-controller-1',<6192.10668.1>]}}
function_clause

Meanwhile the rabbitmq log is empty.

Looks like a regression introduced in the "Enable error_logger depth fine tuning" patch.

Comment 3 Alexey Lebedeff 2016-04-07 13:17:10 UTC
R16B-03.16.el7 is also affected.

Comment 4 Asaf Hirshberg 2016-04-10 14:53:42 UTC
I update my environment to 08/04 puddle but couldn't find erlang-R16B-03.10min.9.el7ost, where can I get it or it should be in the latest puddle?

[root@overcloud-controller-0 ~]# rpm -qa|grep erlang-
erlang-stdlib-R16B-03.10min.8.el7ost.x86_64
erlang-os_mon-R16B-03.10min.8.el7ost.x86_64
erlang-hipe-R16B-03.10min.8.el7ost.x86_64
erlang-snmp-R16B-03.10min.8.el7ost.x86_64
erlang-ssl-R16B-03.10min.8.el7ost.x86_64
erlang-erts-R16B-03.10min.8.el7ost.x86_64
erlang-sasl-R16B-03.10min.8.el7ost.x86_64
erlang-xmerl-R16B-03.10min.8.el7ost.x86_64
erlang-inets-R16B-03.10min.8.el7ost.x86_64
erlang-tools-R16B-03.10min.8.el7ost.x86_64
erlang-crypto-R16B-03.10min.8.el7ost.x86_64
erlang-mnesia-R16B-03.10min.8.el7ost.x86_64
erlang-otp_mibs-R16B-03.10min.8.el7ost.x86_64
erlang-asn1-R16B-03.10min.8.el7ost.x86_64
erlang-syntax_tools-R16B-03.10min.8.el7ost.x86_64
erlang-runtime_tools-R16B-03.10min.8.el7ost.x86_64
erlang-public_key-R16B-03.10min.8.el7ost.x86_64
erlang-kernel-R16B-03.10min.8.el7ost.x86_64
erlang-sd_notify-0.1-2.el7ost.x86_64
erlang-compiler-R16B-03.10min.8.el7ost.x86_64

Comment 5 Jon Schlueter 2016-04-11 12:53:20 UTC
moved back to modified as not part of a puddle yet.

Comment 7 John Eckersberg 2016-04-11 20:11:05 UTC
Looks like it's in puddle 2016-04-11.1 now

Comment 8 Asaf Hirshberg 2016-04-12 12:38:56 UTC
Verified using 2016-04-11 puddle.

[root@overcloud-controller-0 ~]# rpm -qa|grep erlang
erlang-mnesia-R16B-03.10min.9.el7ost.x86_64
erlang-os_mon-R16B-03.10min.9.el7ost.x86_64
erlang-syntax_tools-R16B-03.10min.9.el7ost.x86_64
erlang-xmerl-R16B-03.10min.9.el7ost.x86_64
erlang-stdlib-R16B-03.10min.9.el7ost.x86_64
erlang-runtime_tools-R16B-03.10min.9.el7ost.x86_64
erlang-asn1-R16B-03.10min.9.el7ost.x86_64
erlang-erts-R16B-03.10min.9.el7ost.x86_64
erlang-hipe-R16B-03.10min.9.el7ost.x86_64
erlang-inets-R16B-03.10min.9.el7ost.x86_64
erlang-crypto-R16B-03.10min.9.el7ost.x86_64
erlang-snmp-R16B-03.10min.9.el7ost.x86_64
erlang-compiler-R16B-03.10min.9.el7ost.x86_64
erlang-ssl-R16B-03.10min.9.el7ost.x86_64
erlang-sd_notify-0.1-2.el7ost.x86_64
erlang-kernel-R16B-03.10min.9.el7ost.x86_64
erlang-sasl-R16B-03.10min.9.el7ost.x86_64
erlang-public_key-R16B-03.10min.9.el7ost.x86_64
erlang-otp_mibs-R16B-03.10min.9.el7ost.x86_64
erlang-tools-R16B-03.10min.9.el7ost.x86_64
[root@overcloud-controller-0 ~]# tail -f /var/log/rabbitmq/rabbit\@overcloud-controller-0.log 
accepting AMQP connection <0.3601.0> (172.17.0.15:58391 -> 172.17.0.14:5672)

=INFO REPORT==== 12-Apr-2016::12:34:27 ===
accepting AMQP connection <0.3612.0> (172.17.0.14:58577 -> 172.17.0.14:5672)

=INFO REPORT==== 12-Apr-2016::12:34:28 ===
accepting AMQP connection <0.3623.0> (172.17.0.14:58616 -> 172.17.0.14:5672)

=INFO REPORT==== 12-Apr-2016::12:34:29 ===
accepting AMQP connection <0.3634.0> (172.17.0.14:58659 -> 172.17.0.14:5672)

Comment 10 errata-xmlrpc 2016-04-15 13:48:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0636.html

Comment 11 xiaoxwan 2016-08-31 02:54:31 UTC
Hi, I'm here to ask a question. Well it might be stupid.
This bug starting with:
    erlang-erts-R16B-03.10min.6.el7ost.x86_64 (by Description)
Fixed In Version:
    erlang-R16B-03.10min.9.el7ost
And by checking the changelog, if this "Looks like a regression introduced in the "Enable error_logger depth fine tuning" patch."(by Description) is the reason, then both "erlang-erts-R16B-03.10min.9.el7ost.x86_64" and "erlang-R16B-03.10min.9.el7ost" got this bug fixed.
So I am wondering that what's the connection between erlang and erlang-erts?
Why not just say "fixed in version:erlang-erts-R16B-03.10min.9.el7ost"?
Could anyone give me a hand?

Comment 12 John Eckersberg 2016-08-31 13:44:46 UTC
(In reply to xiaoxwan from comment #11)
> Hi, I'm here to ask a question. Well it might be stupid.
> This bug starting with:
>     erlang-erts-R16B-03.10min.6.el7ost.x86_64 (by Description)
> Fixed In Version:
>     erlang-R16B-03.10min.9.el7ost
> And by checking the changelog, if this "Looks like a regression introduced
> in the "Enable error_logger depth fine tuning" patch."(by Description) is
> the reason, then both "erlang-erts-R16B-03.10min.9.el7ost.x86_64" and
> "erlang-R16B-03.10min.9.el7ost" got this bug fixed.
> So I am wondering that what's the connection between erlang and erlang-erts?
> Why not just say "fixed in version:erlang-erts-R16B-03.10min.9.el7ost"?
> Could anyone give me a hand?

erlang-erts is just a subpackage of the main erlang package.  Typically one only refers to the main package when referencing builds in the build system and errata tooling.