Bug 1121109

Summary: [GSS](6.4.z) Message starvation when a consumer hangs in a domain mode cluster
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Tyronne Wickramarathne <tywickra>
Component: HornetQAssignee: Yong Hao Gao <hgao>
Status: CLOSED NOTABUG QA Contact: Miroslav Novak <mnovak>
Severity: high Docs Contact: Russell Dickenson <rdickens>
Priority: urgent    
Version: 6.2.0CC: bmaxwell, csuconic, istudens, kkhan, msvehla, okalman, tywickra
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-08 15:37:14 UTC Type: Feature Request
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
domain.xml
none
host.xml none

Description Tyronne Wickramarathne 2014-07-18 11:30:26 UTC 
Description of problem:
A message starvation was observed when a consumer hangs on a two node HornetQ cluster in domain mode

Version-Release number of selected component (if applicable):
JBoss-EAP-6.2_CP4 aka JBoss-EAP-6.2.4
HornetQ-2.3.14.Final (2.3.14.1 (Branch 2.3.eap6_2), 123)

How reproducible:


Steps to Reproduce:
1. Please configure two cluster aware instances in a domain.xml to host two node cluster; NodeA and NodeB
2. Configure a destination at both server instances, queue/A
3. Start both server instances
4. Create two MDBeans with having maxSession at "1", FirstMDB and SecondMDB. Please trigger a latency in FirstMDB by making the consumer thread to sleep 5 seconds
5. Deploy both MDBs at each server instance
6. Stop the NodeB
7. Send 250+ messages to queue/A in NodeA
8. Please let the MDB in NodeA to consume a few messages
9. Start the Node2


Actual results:

The consumer in NodeB didn't consume any messages although the consumer-window-size was configured to "0" at both server instances
You could observe the consumer in NodeB starving messages 

Expected results:
Both consumer should be able to consume messages. 

Additional info:
Configuration files and test MDB would follow in a while.
Comment 1 Clebert Suconic 2014-07-18 13:15:36 UTC 
I don't think this is a bug... you can configure redistribution delay and the messages would be redistributed.
Comment 2 Tyronne Wickramarathne 2014-07-20 22:22:37 UTC 
Hi Clebert,

I have configured redistribution delay to 1 second during my tests. This issue occurs when the first consumer, FirstMDB pauses for 5 seconds, during message consumption. 

This works fine as expected when I remove the 5 second pause in the consumer. I have introduced a pause in the consumer to emulate the problem my customer was seeing.

PS:
I assigned this case to Howard, since I have discussed this issue in detail with him. :-)
 
Thanks
Comment 7 Tyronne Wickramarathne 2014-08-04 23:45:52 UTC 
Created attachment 924041 [details]
domain.xml
Comment 8 Tyronne Wickramarathne 2014-08-04 23:46:21 UTC 
Created attachment 924042 [details]
host.xml
Comment 10 Clebert Suconic 2014-08-05 00:26:50 UTC 
If you don't set the consumer-window-size to 0 the test case is invalid for us
Comment 21 Tyronne Wickramarathne 2014-10-09 21:44:09 UTC 
I thought of updating this bugzilla with a comment to avoid any confusions with its title. The same behaviour can be seen in standalone mode. 

Thanks
Tyronne
Comment 22 Clebert Suconic 2015-06-18 15:22:27 UTC 
I wouldn't fix this issue. You can just set consumerWindowSize=0 for slow consumers, or set slow consumer detection to kill the consumer.

I don't see any issues that could be fixed here.
Comment 29 Brad Maxwell 2016-07-08 15:37:14 UTC 
Closing as this is more of a feature that would need to be implemented in Artemis