Bug 779220 (SOA-1613)

Summary: Backport AtomicActionExpiryScanner to EAP4.3 [JBPAPP-2514]
Product: [JBoss] JBoss Enterprise SOA Platform 4 Reporter: Toshiya Kobayashi <tkobayas>
Component: EAP, Support Patch, Component UpgradeAssignee: trev <tkirby>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: 4.3 CP02CC: dlesage
Target Milestone: ---   
Target Release: 4.3 CP04 ER1   
Hardware: Unspecified   
OS: Unspecified   
URL: http://jira.jboss.org/jira/browse/SOA-1613
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-03-23 15:05:45 UTC Type: Feature Request
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Deadline: 2010-03-08   
Attachments:
Description Flags
manual-test.zip none

Description Toshiya Kobayashi 2009-11-18 01:58:12 UTC
Affects: Release Notes
Date of First Response: 2009-11-27 08:07:48
Help Desk Ticket Reference: https://enterprise.redhat.com/issue-tracker/311053
project_key: SOA

Current implementation of JBoss Transaction(4.2.3.SP5_CP05) for SOA-P 4.3.0_CP02 (EAP 4.3.0.GA_CP06) doesn't have a ExpiryScanner to clean up AtomicAction tx-logs. If there is an unrecoverable transaction (e.g. It might happen when JBoss instance is down before deleting tx-log after XAResource's commit is completed), periodic recovery will fail and WARN massages will be logged endlessly.

====
2009-06-24 23:59:02,886 WARN [com.arjuna.ats.jta.logging.loggerI18N][com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa][com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource to use for recovering non-serializable XAResource <131075, 31, 29, 1--53e1e740:1064:4a40cd21:2460b53e1e740:1064:4a40cd21:2460c

Comment 1 Toshiya Kobayashi 2009-11-18 01:59:11 UTC
Link: Added: This issue incorporates JBPAPP-2514


Comment 2 Toshiya Kobayashi 2009-11-18 02:05:59 UTC
EAP 4.3.0 CP07 is going to have AtomicActionExpiryScanner. But it's not complete. See my comment on JBTM-300

Comment 3 Darran Lofthouse 2009-11-27 13:07:48 UTC
JBPAPP-2514 is now complete.

Comment 4 Len DiMaggio 2009-12-16 04:06:56 UTC
Steps to recreate bug:


R+D added a unit test case (LogMoveTest.java)[1] which ensures the function.
But in addition, I'd like to do a manual integration test in order to confirm that this one-off solves the problem which NTT encountered in SOA-P environment.
The manual test procedure is like:

=========
1. Set up 2 SOA-P instances(ESB node and JMS node) and deploy a test ESB service. (I have a zip file which includes all stuff)
2. Start SOA-P instances.
3. Set a breakpoint on BasicAction.phase2Commit(boolean)[line2172:updateState()] using a debugger.
4. Invoke a test ESB client to start a transactional work.
5. After the ESB node stops at the breakpoint, kill(kill -9) the node.
6. Start the ESB node.
7. Priodic recovery (2min by default) kicks in. The following warning is logged because this is an unrecoverable transaction(Resources are already committed, but the tx-log remains)

Could not find new XAResource to use for recovering non-serializable XAResource < 131075, 30, 28, 1--3f57f59b:c51c:4a26721f:21b5-3f57f59b:c51c:4a26721f:21b6 >

8. [If the one-off works correctly] After the specified time (12hr by default), AtomicActionExpiryScanner should move the log to "Expired" directory. Then you can see the warning is not being logged any longer.
==========

Yes, it's reproducible. Is this acceptable to QE team? It's a bit complex so I think I should test it by myself during the QE phase.

[1] https://jira.jboss.org/jira/browse/JBTM-418

Regards,
Toshiya 

Comment 5 Toshiya Kobayashi 2009-12-18 09:03:08 UTC
Added manual test instructions and files

Comment 6 Toshiya Kobayashi 2009-12-18 09:03:08 UTC
Attachment: Added: manual-test.zip


Comment 7 Len DiMaggio 2010-01-14 02:04:37 UTC
Link: Added: This issue depends JBPAPP-3174


Comment 8 Len DiMaggio 2010-01-14 02:07:04 UTC
Link: Added: This issue is related to JBTM-418


Comment 9 Anne-Louise Tangring 2010-02-25 17:31:32 UTC
Approved for SOA 4.3 CP03, but is dependent on inclusion into EAP 4.3 CP08 and the ability to include EAP 4.3 CP08 into SOA 4.3 CP03 per schedule.


Comment 10 Toshiya Kobayashi 2010-02-26 00:23:54 UTC
JBPAPP-2514 has been resolved in EAP 4.3 CP07. So including EAP 4.3 CP07 will satisfy this JIRA.

Comment 11 trev 2010-03-12 10:33:09 UTC
ER1 contains EAP 4.3 CP07 so marking as resolved

Comment 12 David Le Sage 2010-03-14 23:53:07 UTC
The draft text for the Resolved Issues section of the Release Notes states:

https://jira.jboss.org/jira/browse/JBPAPP-3174

    The transaction functionality did not have an ExpiryScanner, (needed to "clean up"
    AtomicAction transaction logs). As a consequence, if a transaction could not be recovered,
    the periodic recovery mechanism would fail and an endless stream of WARN messages would be
    logged.

    An ExpiryScanner has now been backported. It cleans up the logs, meaning that the recovery
    mechanism is now much more reliable as a result.


Comment 13 Jiri Pechanec 2010-03-23 15:05:45 UTC
Verified in ER1 - EAP 4.3 CP07 is base for SOA-P