A ConcurrentModificationException in the CacheWriter interceptor during a commit. java.util.ConcurrentModificationException at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:953) at java.util.LinkedList$ListItr.next(LinkedList.java:886) at org.infinispan.interceptors.CacheWriterInterceptor.store(CacheWriterInterceptor.java:210) at org.infinispan.interceptors.CacheWriterInterceptor.commitCommand(CacheWriterInterceptor.java:115) The code is looping through the modifications associated with the transaction: List<WriteCommand> modifications = ctx.getCacheTransaction().getAllModifications(); ... 210: for (WriteCommand cacheCommand : modifications) { The transaction's modification list is stored as a synchronized list with a comment "we need to synchronize this collection to be able to get a valid snapshot from another thread during state transfer". But this thread is not doing state transfer, and I'd assume "get a valid snapshot" wouldn't include modification? Is it valid for this list to be modified from another thread (in which case all iterations should be synchronized), or is something modifying it incorrectly?
The stack trace includes LinkedList instead of SynchronizedList, so this is very likely a RemoteTransaction initialized from LockControlCommand.perform(). However, every code path that touches the modifications list in a RemoteTransaction seems to create a new list without modifying the old one, so I'm not sure how the CME could be triggered. I also ran the core test suite with a custom class replacing the LinkedList, and none of the mutating methods was ever called. Can you post the cache configuration and perhaps some logs? Cheers Dan Oops, you were right. LocalTransaction.addModification() also creates a LinkedList. It wraps it in a SynchronizedList, but SynchronizedList.iterator() doesn't wrap the iterator, so it doesn't show up in the stack trace. That doesn't change the situation however, I don't see any way the modifications list could be modified during the commit. At this time, the only thing I can think of is to "freeze" the modifications list at the beginning of the commit and throw an exception if anyone tries to modify it afterwards, then give the customer the code and wait for them to give us back a stack trace of the modifying thread. We could also synchronize the iteration over the modifications list. But adding a modification during commit seems like a bug anyway, and the added synchronization would just push the problem somewhere else. In theory we could also get a CME if the collection was modified on one thread and then iterated over in a different thread, because TransactionSynchronizationInterceptor is not in the interceptor chain to ensure visibility. But CMTTxInterceptor.invokeInOurTx executes the entire transaction in a single thread, so that's clearly not the case here. Cheers Dan
Pull request: https://github.com/infinispan/jdg/pull/705
William Burns <wburns> updated the status of jira ISPN-5379 to Resolved
We're not able to reproduce this issue. Closing
This product has been discontinued or is no longer tracked in Red Hat Bugzilla.