Bug 1001634 - Write command may be ignored during state transfer
Summary: Write command may be ignored during state transfer
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: JBoss Data Grid 6
Classification: JBoss
Component: Infinispan
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ER3
: 6.2.0
Assignee: Tristan Tarrant
QA Contact: Martin Gencur
URL:
Whiteboard:
: 1024918 (view as bug list)
Depends On:
Blocks: 1010419 1017190
TreeView+ depends on / blocked
 
Reported: 2013-08-27 12:34 UTC by Radim Vansa
Modified: 2025-02-10 03:28 UTC (History)
2 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2025-02-10 03:28:26 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker ISPN-3443 0 Blocker Resolved WriteCommand may be ignored during state transfer 2016-05-26 06:49:09 UTC

Description Radim Vansa 2013-08-27 12:34:16 UTC
Distributed sync non-tx cache.
Situation:

1) A node is joining the cluster, requesting some segment
2) RemoveCommand is sent to backup owner with ignorePreviousValue=true
3) It looks up the entry and finds null
4) State transfer invokes the PutKeyValueCommand and sets the value for removed entry (updateKeys has not the key yet)
5) RemoveCommand adds its key to updateKeys set, but it does not remove the value as it is already null (in its context)

Result: the value is removed on primary but on backup this is still present

Comment 2 JBoss JIRA Server 2013-08-28 10:52:23 UTC
Radim Vansa <rvansa> made a comment on jira ISPN-3443

Confirmed also for transactional cache.

Comment 3 JBoss JIRA Server 2013-08-28 10:55:06 UTC
Radim Vansa <rvansa> made a comment on jira ISPN-3443

The issue may happen for PutKeyValueCommand as well - the command commits the entry, then the ST can commit the entry and only after the command records this key into the updateKeys set.

Comment 4 JBoss JIRA Server 2013-10-07 09:14:03 UTC
Radim Vansa <rvansa> updated the status of jira ISPN-3443 to Reopened

Comment 5 JBoss JIRA Server 2013-10-07 09:14:03 UTC
Radim Vansa <rvansa> made a comment on jira ISPN-3443

I still have ignored commands - see comment I've posted 28/Aug/13 12:55 PM. Experienced with ReplaceCommand on backup owner in non-tx cache.
I believe the cause is race condition between checking the update keys in EntryWrappingInterceptor.commitEntryIfNeeded and actually committing it - if the ReplaceCommand is executed just between the check and committing the value, the value is overwritten by ST.

Comment 6 JBoss JIRA Server 2013-10-09 08:52:54 UTC
Dan Berindei <dberinde> made a comment on jira ISPN-3443

Radim, I missed your comment about the race condition in EntryWrappingInterceptor.commitEntryIfNeeded. I don't think it's a case of the WriteCommand being ignored, though: the command is committed, it's just that state transfer than overwrites the value.

Comment 7 JBoss JIRA Server 2013-10-09 09:01:31 UTC
Radim Vansa <rvansa> made a comment on jira ISPN-3443

You're right - should I rename this JIRA back to RemoveCommand may be ignored during state transfer and create different JIRA for overwriting by ST?

Comment 8 JBoss JIRA Server 2013-10-10 12:01:07 UTC
Radim Vansa <rvansa> updated the status of jira ISPN-3443 to Reopened

Comment 9 JBoss JIRA Server 2013-10-10 12:01:07 UTC
Radim Vansa <rvansa> made a comment on jira ISPN-3443

[~dan.berindei]: Regrettably, the fix is not correct. You have to atomically check and ADD the key to updated keys set. Otherwise, a situation when both transaction and state transfer commit the entry (in this order) is possible, resulting with outdated entry on the node.

Comment 10 JBoss JIRA Server 2013-10-11 14:28:14 UTC
Radim Vansa <rvansa> updated the status of jira ISPN-3443 to Resolved

Comment 11 JBoss JIRA Server 2013-10-11 14:28:14 UTC
Radim Vansa <rvansa> made a comment on jira ISPN-3443

My apologies, old version got into the test.

Comment 12 Radim Vansa 2013-12-10 10:10:15 UTC
*** Bug 1024918 has been marked as a duplicate of this bug. ***

Comment 14 Red Hat Bugzilla 2025-02-10 03:28:26 UTC
This product has been discontinued or is no longer tracked in Red Hat Bugzilla.


Note You need to log in before you can comment on or make changes to this bug.