Bug 1152782

Summary: Exception in thread "Periodic Recovery" java.util.ConcurrentModificationException
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Ondrej Chaloupka <ochaloup>
Component: Transaction ManagerAssignee: Michael <mmusgrov>
Status: CLOSED CURRENTRELEASE QA Contact: Ondrej Chaloupka <ochaloup>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.4.0CC: hhovsepy, kkhan, tom.jenkinson
Target Milestone: DR11   
Target Release: EAP 6.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-08-19 12:43:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1165728    
Attachments:
Description Flags
server.log with ConcurrentModificationException none

Description Ondrej Chaloupka 2014-10-14 23:09:38 UTC 
Created attachment 947054 [details]
server.log with ConcurrentModificationException

This error is (probably) the same as it was reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1010242
but for Sybase it seems to be reproducible quite consistently.

We run test where Byteman rule is added to stop XA transaction in prepare phase (BasicAction#prepare).
Then we call 3 ejbs which starts 3 transactions. Those are stuck at prepare.
When all three are prepared to proceed then we unblock the byteman rule and check the results.

In case of Sybase database it happens that during checking of state - before the byteman rule is unblock - the app server came with exception [1] and stays deadlocked.

I expect that this is hardly to obtain such state in standard usage but the code should be clean from ConcurrentModificationException.


[1]
ERROR [stderr] (Periodic Recovery) Exception in thread "Periodic Recovery" java.util.ConcurrentModificationException
ERROR [stderr] (Periodic Recovery) 	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922)
ERROR [stderr] (Periodic Recovery) 	at java.util.HashMap$KeyIterator.next(HashMap.java:956)
ERROR [stderr] (Periodic Recovery) 	at com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule.periodicWorkFirstPass(CommitMarkableResourceRecordRecoveryModule.java:172)
ERROR [stderr] (Periodic Recovery) 	at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:747)
ERROR [stderr] (Periodic Recovery) 	at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:375)
Comment 1 Ondrej Chaloupka 2014-10-14 23:11:34 UTC 
I'm sorry incorrect link to 6.3.0 bugzilla. Correct is:
https://bugzilla.redhat.com/show_bug.cgi?id=1103780
Comment 3 Ondrej Chaloupka 2014-10-20 08:59:40 UTC 
Just checked the code and it seems that the cause is at line:
https://github.com/jbosstm/narayana/blob/4.17/ArjunaJTA/jta/classes/com/arjuna/ats/internal/jta/recovery/arjunacore/CommitMarkableResourceRecordRecoveryModule.java#L173

Item from collection is removed but without iterator would know about it. It's needed to use iterator itself for removing the item.
Comment 4 Michael 2014-10-20 09:18:40 UTC 
Good spot. I will raise a JIRA and link it to this case.
Comment 5 Ondrej Chaloupka 2014-11-26 12:25:02 UTC 
Codebase changed, concurrent locking for Sybase seems to go away.
Verified for EAP 6.4.0.DR11
Comment 6 JBoss JIRA Server 2014-11-28 10:32:35 UTC 
Tom Jenkinson <tom.jenkinson@redhat.com> updated the status of jira JBTM-2271 to Closed