Bug 1742842
| Summary: | Rabbitmq stuck threads even after re-initializing the rabbitmq cluster | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Mircea Vutcovici <mircea.vutcovici> |
| Component: | rabbitmq-server | Assignee: | Peter Lemenkov <plemenko> |
| Status: | CLOSED DUPLICATE | QA Contact: | pkomarov |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 10.0 (Newton) | CC: | apevec, jeckersb, lhh, lmiccini, rosingh |
| Target Milestone: | --- | Keywords: | Triaged, ZStream |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | rabbitmq-server-3.6.3-7.el7ost | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-08-19 14:33:17 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Yes, this looks like a bug 1434593. Please update at least to rabbitmq-server-3.6.3-7.el7ost and retest. *** This bug has been marked as a duplicate of bug 1434593 *** |
Description of problem: rabbitmq has threads that are stuck even after restarting it after the /var/lib/rabbitmq/mnesia/ folder was emptied. Version-Release number of selected component (if applicable): rabbitmq-server-3.6.3-6.el7ost.noarch How reproducible: All the time on this environment. Steps to Reproduce: 1. stop rabbitmq 2. empty /var/lib/rabbitmq/mnesia/ folder 3. start rabbitmq 4. run: rabbitmqctl eval 'rabbit_diagnostics:maybe_stuck().' Actual results: [root@controller1 ~]# rabbitmqctl eval 'rabbit_diagnostics:maybe_stuck().' 2019-08-16 18:38:33 There are 11728 processes. 2019-08-16 18:38:33 Investigated 4 processes this round, 5000ms to go. 2019-08-16 18:38:34 Investigated 4 processes this round, 4500ms to go. 2019-08-16 18:38:34 Investigated 4 processes this round, 4000ms to go. 2019-08-16 18:38:35 Investigated 4 processes this round, 3500ms to go. 2019-08-16 18:38:35 Investigated 4 processes this round, 3000ms to go. 2019-08-16 18:38:36 Investigated 4 processes this round, 2500ms to go. 2019-08-16 18:38:36 Investigated 4 processes this round, 2000ms to go. 2019-08-16 18:38:37 Investigated 4 processes this round, 1500ms to go. 2019-08-16 18:38:37 Investigated 4 processes this round, 1000ms to go. 2019-08-16 18:38:38 Investigated 4 processes this round, 500ms to go. 2019-08-16 18:38:38 Found 4 suspicious processes. 2019-08-16 18:38:38 [{pid,<10374.16909.97>}, {registered_name,[]}, {current_stacktrace, [{rabbit_control_misc,collect_monitors,1, [{file,"src/rabbit_control_misc.erl"}, {line,124}]}, {rpc,'-handle_call_call/6-fun-0-',5, [{file,"rpc.erl"},{line,206}]}]}, {initial_call,{erlang,apply,2}}, {message_queue_len,3}, {links,[]}, {monitors,[]}, {monitored_by,[<10374.14.0>]}, {heap_size,233}] 2019-08-16 18:38:38 [{pid,<10374.4211.98>}, {registered_name,[]}, {current_stacktrace, [{rabbit_control_misc,collect_monitors,1, [{file,"src/rabbit_control_misc.erl"}, {line,124}]}, {rpc,'-handle_call_call/6-fun-0-',5, [{file,"rpc.erl"},{line,206}]}]}, {initial_call,{erlang,apply,2}}, {message_queue_len,3}, {links,[]}, {monitors,[]}, {monitored_by,[<10374.14.0>]}, {heap_size,233}] 2019-08-16 18:38:38 [{pid,<10374.9131.98>}, {registered_name,[]}, {current_stacktrace, [{rabbit_control_misc,collect_monitors,1, [{file,"src/rabbit_control_misc.erl"}, {line,124}]}, {rpc,'-handle_call_call/6-fun-0-',5, [{file,"rpc.erl"},{line,206}]}]}, {initial_call,{erlang,apply,2}}, {message_queue_len,3}, {links,[]}, {monitors,[]}, {monitored_by,[<10374.14.0>]}, {heap_size,233}] [...] Expected results: no threads stuck Additional info: this affects all openstack services that depends on rabbitmq for inter-communication