Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 901124 (JBPAPP6-1155)

Summary: Duplicate message detection causes message loss in cluster
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Miroslav Novak <mnovak>
Component: HornetQAssignee: Clebert Suconic <clebert.suconic>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.0.0CC: atangrin, clebert.suconic, fnasser, mnovak, pslavice
Target Milestone: ---   
Target Release: EAP 6.0.1   
Hardware: Unspecified   
OS: Unspecified   
URL: http://jira.jboss.org/jira/browse/JBPAPP6-1155
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-07 09:56:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
reproducer.zip
none
client-maven-project.zip none

Description Miroslav Novak 2012-10-30 11:35:26 UTC
project_key: JBPAPP6

Test scenario:
1. Start two EAP 6.0.1.ER3 servers in cluster
2. Deploy queue InQueue on both of the nodes
3. Sent 50 messages with set "_HQ_DUPL_ID" message property to first node to InQueue
4. Receive them from first node

Result:
Only 25 messages is received. The rest is discarded by first node 1:
{code}
...
12:18:47,192 WARN  [org.hornetq.core.postoffice.impl.PostOfficeImpl] (Old I/O server worker (parentId: 954216388, [id: 0x38e02fc4, /192.168.40.1:5445])) Duplicate message detected - message will not be routed. Message information:ServerMessage[messageID=247,priority=4, bodySize=2413,expiration=0, durable=true, address=jms.queue.InQueue,properties=TypedProperties[{_HQ_ROUTE_TO=[B@70c7c57c, count=36, _HQ_DUPL_ID=47f14971-0337-439f-a157-977b6547e3051351595922925}]]@1266059708
12:18:47,193 WARN  [org.hornetq.core.postoffice.impl.PostOfficeImpl] (Old I/O server worker (parentId: 954216388, [id: 0x38e02fc4, /192.168.40.1:5445])) Duplicate message detected - message will not be routed. Message information:ServerMessage[messageID=250,priority=4, bodySize=2413,expiration=0, durable=true, address=jms.queue.InQueue,properties=TypedProperties[{_HQ_ROUTE_TO=[B@534cbc58, count=38, _HQ_DUPL_ID=1b329abf-7691-461c-872c-89fa48ad2d491351595922941}]]@1390501921
12:18:47,193 WARN  [org.hornetq.core.postoffice.impl.PostOfficeImpl] (Old I/O server worker (parentId: 954216388, [id: 0x38e02fc4, /192.168.40.1:5445])) Duplicate message detected - message will not be routed. Message information:ServerMessage[messageID=253,priority=4, bodySize=2413,expiration=0, durable=true, address=jms.queue.InQueue,properties=TypedProperties[{_HQ_ROUTE_TO=[B@4a6e4109, count=40, _HQ_DUPL_ID=a37ba36b-3a89-41a5-8a51-f59a847ac48c1351595923002}]]@208657420
...
{code}

Attached reproducer:
1. Download and unzip reproducer.zip
2. Prepare servers - "sh prepare.sh" - this creates two servers with deployed jms/queue/InQueue
3. Start first and second server:
sh start-server1.sh <first_ip>
sh start-server2.sh <second_ip>
4. Start producer:
sh start-producer.sh <first_ip>
5. Start consumer:
sh start-consumer.sh <first_ip>

Comment 1 Miroslav Novak 2012-10-30 12:12:58 UTC
Attachment: Added: reproducer.zip


Comment 2 Miroslav Novak 2012-10-30 12:18:28 UTC
Link: Added: This issue is related to JBPAPP-10067


Comment 3 Miroslav Novak 2012-10-30 16:59:17 UTC
Attached maven project with TestProducer.java which is used in reproducer.

Comment 4 Miroslav Novak 2012-10-30 16:59:17 UTC
Attachment: Added: client-maven-project.zip


Comment 5 Clebert Suconic 2012-10-30 17:15:07 UTC
This is an edge case that affects all the branches....

If you use duplicate-detection over cluster, you will also use duplicate detection over the bridge when transferring over.

The target should ignore the regular duplicate detection if the Bridge duplicate detection is present.

Having said that this is not a regression.. it's an edge case that you just discovered.. so no reason to reject the patch.

Comment 6 Clebert Suconic 2012-10-30 22:18:55 UTC
Link: Added: This issue Cloned to JBPAPP-10309


Comment 7 Clebert Suconic 2012-10-30 22:47:17 UTC
I have committed a fix on all the branches for HornetQ.

Should I resolve this now or only after a release?

Comment 8 Miroslav Novak 2012-10-31 08:29:29 UTC
Thanks Clebert. You can set this jira as resolved.

Comment 9 Miroslav Novak 2012-11-01 12:16:56 UTC
Loosing messages in cluster is a problem. It'd be good to see this fix in EAP 6.0.1.ER4

Comment 10 Clebert Suconic 2012-11-01 14:09:05 UTC
I will mark this as blocker, and will resolve it as soon as we make a release

Comment 11 Anne-Louise Tangring 2012-11-01 17:41:27 UTC
This issue is listed as Major or below and as such is not targetted for the EAP 6.0.1 release, now that we are in Blocker or Critical issue only mode. Should this be reconsidered, please contact the EAP PM team.

Comment 12 Miroslav Novak 2012-11-05 14:17:26 UTC
[~clebertsuconic]
Is it possible to reconsider the priority of this issue and describe the impact if left unfixed in EAP 6.0.1.ER4?


Comment 13 Clebert Suconic 2012-11-05 14:25:20 UTC
[~mnovak] I talked to Fernando, and we agreed we would do it if there was another CR...

Otherwise we would do it as a customer patch.


If you negotiate another release, We can do it

Comment 14 Miroslav Novak 2012-11-05 15:42:31 UTC
Just got info that tomorrow should be triage regarding new build. If there will be more jiras like this there will be new build. I'll raise the priority to get this to "Blocker or Critical issue only mode" list.

Comment 15 Fernando Nasser 2012-11-05 15:47:38 UTC
In this case moving to ER4

Comment 16 Pavel Slavicek 2012-11-07 12:35:51 UTC
Hi Clebert, if fix is available, could you resolve jira, please?

Comment 17 Fernando Nasser 2012-11-07 14:38:59 UTC
Clebert, we need a patch (source code diff) against the HornetQ 2.2.22.Final tag.
We are not having a new ER we are just patching very localized bis of ER3.
Please attach it to this JIRA.

Note that Anne-Louise still wants to discuss each patch among us for absolute necessity of inclusion as we 
want to minimize the changes.

Comment 18 Fernando Nasser 2012-11-07 14:39:29 UTC
Oh, instead of resolving, once the patch is available flip the JIRA to me for inclusion.

Comment 19 Miroslav Novak 2012-11-09 14:37:43 UTC
Link: Added: This issue Cloned to JBPAPP-10387


Comment 20 Anne-Louise Tangring 2012-11-13 20:43:44 UTC
Docs QE Status: Removed: NEW 


Comment 21 Miroslav Novak 2012-12-07 09:56:35 UTC
Issue verified in EAP 6.0.1.ER4.1. Closing.