Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1228026

Summary: Invalid message id under high load
Product: [JBoss] JBoss Data Grid 6 Reporter: Osamu Nagano <onagano>
Component: CPP ClientAssignee: Pedro Zapata <pzapataf>
Status: CLOSED UPSTREAM QA Contact: Alan Field <afield>
Severity: high Docs Contact:
Priority: high    
Version: 6.4.0CC: afield, chuffman, mgencur, zzhou
Target Milestone: ER1   
Target Release: 6.5.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
The C++ HotRod client was occasionally logging the following exception when there were many concurrent Put and Get operations: <programlisting>"Invalid message id. Expected $OLD_MSGID and received $NEW_MSGID"</programlisting> This issue was caused by the message id counter not being thread safe. This issue is resolved as of Red Hat JBoss Data Grid 6.5.1.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2025-02-10 03:47:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
hotrodwrapper.zip none

Description Osamu Nagano 2015-06-04 06:02:31 UTC 
A C++ Hot Rod client, which is executing about 560 get/sec and 50 put/sec concurrently, occasionally receives an exception with the following error message.

Invalid message id. Expected 5774 and received 4436

It seems the message id counter is not thread-safe.
Comment 1 Osamu Nagano 2015-06-04 06:09:51 UTC 
Here are snippets from the current master.
https://github.com/infinispan/cpp-client/blob/master/src/hotrod/impl/protocol/Codec10.cpp

Gloal variable:
     25 // TODO : multithread management
     26 long Codec10::msgId = 0;

In writeHeader:
     38     // TODO : multithread management
     39     transport.writeVLong(params.setMessageId(++msgId).messageId);

In readHeader:
     82     if (receivedOpCode != params.opRespCode) {
     83       if (receivedOpCode == HotRodConstants::ERROR_RESPONSE) {
     84         checkForErrorsInResponseStatus(transport, params, status);
     85         }
     86         std::ostringstream message;
     87         message << "Invalid response operation. Expected " << std::hex <<
     88             (int) params.opRespCode << " and received " << std::hex << (int) receivedOpCode;
     89         throw InvalidResponseException(message.str());
     90     }
Comment 11 Osamu Nagano 2015-07-29 07:33:05 UTC 
Created attachment 1057224 [details]
hotrodwrapper.zip

Minimum reproducing set of files for Nginx (OpenResty).  Find README.en in the archive.
Comment 15 Alan Field 2015-09-10 20:26:31 UTC 
Verified with JDG 6.5.1 CR1

I ran the siege test 10,000 times without seeing the error, so I am closing this BZ for now until there is a reproducer for this.
Comment 23 Red Hat Bugzilla 2025-02-10 03:47:41 UTC 
This product has been discontinued or is no longer tracked in Red Hat Bugzilla.