Bug 915591

Summary: org.infinispan.transaction.WriteSkewException: Detected write skew (random failure)
Product: [JBoss] JBoss Data Grid 6 Reporter: Vitalii Chepeliuk <vchepeli>
Component: InfinispanAssignee: Tristan Tarrant <ttarrant>
Status: VERIFIED --- QA Contact: Martin Gencur <mgencur>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1.0CC: jdg-bugs, jkudrnac
Target Milestone: DR2   
Target Release: 6.2.0   
Hardware: sparc64   
OS: Solaris   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 921491 (view as bug list) Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 921491    
Attachments:
Description Flags
Added console output none

Description Vitalii Chepeliuk 2013-02-26 07:48:21 UTC 
2013-02-25 14:26:25,071 WARN [RepeatableReadEntry] (EntryWriter-9, WriteSkewTest) ISPN000005: Detected write skew on key [k]. Another process has changed the entry since we last read it! Unable to copy entry for update.
2013-02-25 14:26:25,073 ERROR [WriteSkewTest] (EntryWriter-9, WriteSkewTest) Unexpected
org.infinispan.transaction.WriteSkewException: Detected write skew.
at org.infinispan.container.entries.RepeatableReadEntry.performLocalWriteSkewCheck(RepeatableReadEntry.java:68)
at org.infinispan.container.entries.RepeatableReadEntry.copyForUpdate(RepeatableReadEntry.java:52)
at org.infinispan.container.EntryFactoryImpl.wrapEntryForPut(EntryFactoryImpl.java:170)
at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:164)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPutKeyValueCommand(OptimisticLockingInterceptor.java:142)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:251)
at org.infinispan.interceptors.TxInterceptor.visitPutKeyValueCommand(TxInterceptor.java:191)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1169)
at org.infinispan.CacheImpl.putInternal(CacheImpl.java:766)
at org.infinispan.CacheImpl.put(CacheImpl.java:760)
at org.infinispan.CacheImpl.put(CacheImpl.java:754)
at org.infinispan.CacheSupport.put(CacheSupport.java:53)
at org.infinispan.api.mvcc.repeatable_read.WriteSkewTest$EntryWriter.call(WriteSkewTest.java:433)
at org.infinispan.api.mvcc.repeatable_read.WriteSkewTest$EntryWriter.call(WriteSkewTest.java:418)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Comment 1 JBoss JIRA Server 2013-02-26 09:51:48 UTC 
Adrian Nistor <anistor> made a comment on jira ISPN-2858

So which test method failed? Could you also attach the trace log please?
Comment 2 Vitalii Chepeliuk 2013-02-26 12:28:19 UTC 
Created attachment 702832 [details]
Added console output
Comment 3 JBoss JIRA Server 2013-02-26 13:20:07 UTC 
Adrian Nistor <anistor> made a comment on jira ISPN-2858

From the attached output:
{noformat}
2013-02-25 12:55:30,993 ERROR [UnitTestTestNGListener] (testng-WriteSkewTest) Test testWriteSkewWithOnlyPut(org.infinispan.api.mvcc.repeatable_read.WriteSkewTest) failed.
{noformat}

This is a duplicate of ISPN-2771.
Comment 4 JBoss JIRA Server 2013-02-26 13:21:08 UTC 
Adrian Nistor <anistor> updated the status of jira ISPN-2858 to Resolved
Comment 7 JBoss JIRA Server 2013-07-10 10:37:39 UTC 
Pedro Ruivo <pedroruivo2> updated the status of jira ISPN-2771 to Coding In Progress
Comment 8 JBoss JIRA Server 2013-07-16 11:52:40 UTC 
Pedro Ruivo <pedroruivo2> updated the status of jira ISPN-2771 to Reopened
Comment 9 JBoss JIRA Server 2013-07-16 11:52:40 UTC 
Pedro Ruivo <pedroruivo2> made a comment on jira ISPN-2771

This test is still failing because it performs the local write skew check during the copyForUpdate(). Solutions:

easy: don't perform the local write skew check in the copyForUpdate because it will be performed in the prepare phase anyway.
complex: the correct behavior here is to perform the local write skew check when
 * the key has previous read || the command does not ignore the return value

try to do a test case for this race condition...
Comment 10 JBoss JIRA Server 2013-07-18 09:28:01 UTC 
Dan Berindei <dberinde> made a comment on jira ISPN-2771

[~pruivo] why is the second approach more correct than the first? Shouldn't one write skew check be enough?
Comment 11 JBoss JIRA Server 2013-07-18 09:42:22 UTC 
Pedro Ruivo <pedroruivo2> made a comment on jira ISPN-2771

[~dan.berindei] don't miss understand me. I think the most correct is only to perform the write skew check at prepare time (as it is done in clustered mode).