| Summary: | Clustered broker fails to start on host reboot; logs critical cluster error re: exclusive queue | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise MRG | Reporter: | Ken Giusti <kgiusti> | ||||
| Component: | qpid-cpp | Assignee: | messaging-bugs <messaging-bugs> | ||||
| Status: | NEW --- | QA Contact: | MRG Quality Engineering <mrgqe-bugs> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 2.3 | CC: | jross, pgsousa | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | Type: | Bug | |||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Attachments: |
|
||||||
I have 3 brokers, whenever I kill one of them, rebooting the server, It fails to start: v 28 12:52:27 Controller03 qpidd[13540]: 2013-11-28 12:52:27 [System] error Connection 192.168.30.202:5672-192.168.30.200:60498 No protocol received closing Nov 28 12:52:27 Controller03 qpidd[13540]: 2013-11-28 12:52:27 [System] error Connection 192.168.30.202:5672-192.168.30.200:60500 No protocol received closing Nov 28 12:52:33 Controller03 qpidd[13540]: 2013-11-28 12:52:33 [System] error Connection 192.168.30.202:5672-192.168.30.200:60616 No protocol received closing Nov 28 12:52:33 Controller03 qpidd[13540]: 2013-11-28 12:52:33 [System] error Connection 192.168.30.202:5672-192.168.30.200:60617 No protocol received closing Another note: to start the node that's not working, I have to stop the remaining nodes and start them again. MEssage after starting the node I rebooted: qpidd[3334]: 2013-11-28 17:32:00 [Protocol] error Execution exception: resource-locked: Cannot subscribe to exclusive queue reply_d0f7bd5f86c8451ab219e23d242cb9f2 (qpid/broker/SessionAdapter.cpp:414) Nov 28 17:32:00 Controller03 qpidd[3334]: 2013-11-28 17:32:00 [System] error not-found: Unknown destination 0 session=anonymous.45b850d8-2042-4d5d-b155-804f9a038ce4:0 (qpid/broker/SemanticState.cpp:580) Nov 28 17:32:00 Controller03 qpidd[3334]: 2013-11-28 17:32:00 [HA] critical cluster(192.168.30.202:3334 UPDATEE) catch-up connection closed prematurely 192.168.30.200:5672-192.168.30.200:41163(192.168.30.202:3334-30 local,catchup) Nov 28 17:32:00 Controller03 qpidd[3334]: 2013-11-28 17:32:00 [HA] notice cluster(192.168.30.202:3334 LEFT) leaving cluster openstack |
Created attachment 829390 [details] logs obtained by user Description of problem: qpidd is being used as part of a cluster, used with openstack, deployed with an haproxy. One of the hosts running a qpidd cluster member is rebooted - qpidd fails to restart. Version-Release number of selected component (if applicable): [root@Controller01 .qpidd]# rpm -qa | grep qpid qpid-qmf-0.18-18.el6_4.x86_64 qpid-cpp-client-0.18-17.el6_4.x86_64 qpid-cpp-server-0.18-17.el6_4.x86_64 python-qpid-qmf-0.18-18.el6_4.x86_64 qpid-cpp-server-cluster-0.18-17.el6_4.x86_64 python-qpid-0.18-5.el6_4.noarch qpid-tools-0.18-8.el6.noarch How reproducible: Occasional. Steps to Reproduce: 1. 2. 3. Actual results: qpidd fails to start. Expected results: qpidd should start and join the cluster. Additional info: See attached cluster logs - cluster is issuing a critical log message regarding an exclusive queue. Openstack uses these exclusive queues as temporary reply queues. [root@Controller01 .qpidd]# cat /etc/qpidd.conf # GENERATED BY PUPPET # # Configuration file for qpidd. Entries are of the form: # name=value # # (Note: no spaces on either side of '='). Using default settings: # "qpidd --help" or "man qpidd" for more details. cluster-name="openstack" #cluster-size=3 port=5672 max-connections=500 worker-threads=17 connection-backlog=10 auth=no realm=QPID log-enable=debug+:cluster log-to-file=/tmp/qpidd.log