Bug 1336861
| Summary: | Connecting AMQP to port 5000 creates infinite reconnect loop that blocks workers | ||
|---|---|---|---|
| Product: | Red Hat CloudForms Management Engine | Reporter: | Pete Savage <psavage> |
| Component: | Providers | Assignee: | Ladislav Smola <lsmola> |
| Status: | CLOSED ERRATA | QA Contact: | Shveta <sshveta> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 5.6.0 | CC: | jfrey, jhardy, lsmola, obarenbo, simaishi, sshveta |
| Target Milestone: | GA | ||
| Target Release: | 5.6.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | openstack | ||
| Fixed In Version: | 5.6.0.8 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-06-29 16:03:11 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/be3ca3d9aee6c65d859c3110b1acc28a2fd9f974 commit be3ca3d9aee6c65d859c3110b1acc28a2fd9f974 Author: Ladislav Smola <lsmola> AuthorDate: Wed May 18 11:08:37 2016 +0200 Commit: Ladislav Smola <lsmola> CommitDate: Wed May 18 11:08:37 2016 +0200 Fix infinite reconnect loop caused by new bunny gem If trying to connect to existing port, bunny is set to try to reconnect infinite times, which will end up blocking the worker. We need to set reconnect limit and we need to disable reconnect for validation. So reconnect is set only for connection created by worker, that should be consuming events. Fixes BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1336861 .../manageiq/providers/openstack/event_catcher_mixin.rb | 15 +++++++++------ app/models/manageiq/providers/openstack/manager_mixin.rb | 2 +- config/settings.yml | 3 +++ .../openstack/events/openstack_rabbit_event_monitor.rb | 11 +++++++++-- 4 files changed, 22 insertions(+), 9 deletions(-) Fixed. Verified in 5.6.0.8-rc1.20160524155303_f2a5a50 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1348 |
Description of problem: Connecting AMQP to port 5000 creates infinite reconnect loop that blocks workers. This is the port that is default in 5.6 and is used mainly for keystone. Version-Release number of selected component (if applicable): 5.6.0.6 How reproducible: 100% Steps to Reproduce: 1. Add an Openstack provider adding the AMQP hostname as the same as the Default hostname and leaving the port as 5000 2. 3. Actual results: Inventory never completes, provider cannot be deleted, new providers cannot be created. Expected results: Connection error is handled, logged and warning shown in UI. Additional info: E, [2016-05-17T07:42:34.597712 #4898] ERROR -- #<Bunny::Session:0x5ea41b0 guest.xx.xx:5000, vhost=/, hosts=[1xx.xx.xx.xx]>: Got an exception when receiving data: IO timeout when reading 7 bytes (Timeout::Error) W, [2016-05-17T07:42:34.598236 #4898] WARN -- #<Bunny::Session:0x5ea41b0 guest.xx.xx:5000, vhost=/, hosts=[xx.xx.xx.xx]>: TCP connection failed, reconnecting in 5.0 seconds These log messages continually show up