Bug 1383572 - [GSS](6.4.z) Deadlock in BasicAction when jboss remoting and JTA is used
Summary: [GSS](6.4.z) Deadlock in BasicAction when jboss remoting and JTA is used
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: EJB
Version: 6.4.10
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: CR1
: EAP 6.4.12
Assignee: Radovan Netuka
QA Contact: Jan Martiska
URL:
Whiteboard: eap6412-proposed
Depends On:
Blocks: 1238925 eap6412-payload 1383246 1399703
TreeView+ depends on / blocked
 
Reported: 2016-10-11 06:09 UTC by Flavia Rainone
Modified: 2020-04-15 14:43 UTC (History)
12 users (show)

Fixed In Version:
Clone Of:
: 1399703 (view as bug list)
Environment:
Last Closed: 2016-11-29 15:39:57 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
client application that reproduces the bug (requires db provisioning) (3.29 MB, application/zip)
2016-11-22 18:49 UTC, Flavia Rainone
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker JBEAP-6193 0 Major Verified [GSS](7.1.0) Deadlock in BasicAction when jboss remoting and JTA is used 2017-03-22 11:35:10 UTC
Red Hat Issue Tracker JBEAP-6394 0 Major Verified [GSS](7.0.z) Deadlock in BasicAction when jboss remoting and JTA is used 2017-03-22 11:35:10 UTC
Red Hat Issue Tracker JBTM-2749 0 Critical Closed Add an SPI method to lookup imported transactions by Xid 2017-03-22 11:35:10 UTC

Description Flavia Rainone 2016-10-11 06:09:38 UTC
At EJBRemoteTransactionPropagatingInterceptor, a transaction that has been already open at current server could be ignored, thus causing the transaction to be reimported, resulting in the double diamond problem.

To reproduce you need a reinvocation at a server that was already participating in the transaction, as the scenario below for example:
    server1 -> server2 -> server1

Comment 1 JBoss JIRA Server 2016-10-14 09:34:10 UTC
Flavia Rainone <frainone> updated the status of jira JBEAP-6193 to Coding In Progress

Comment 2 JBoss JIRA Server 2016-10-15 09:37:42 UTC
Flavia Rainone <frainone> updated the status of jira JBEAP-6394 to Coding In Progress

Comment 5 Peter Palaga 2016-10-19 08:19:57 UTC
Switching back to POST, because https://github.com/jbossas/jboss-eap/pull/2860 is not merged yet. I switched to MODIFIED because I saw the "Deadlock in the JTA remoting" commit merged in https://github.com/jbosstm/jboss-transaction-spi/commits/7.1.0.SPx . This is a reason why it is impractical to have one BZ to track changes in more than one git repo.

Comment 7 Jiří Bílek 2016-11-14 11:41:49 UTC
The fix works well on EAP 7.1.0 (our integration test for this issue does not work on 7.1.0.DR6 and works well on 7.1.0.DR7).
But for 6.4.12 it seems bad. EAP 6.4.12 gets the same issue as 6.4.11 and 7.1.0.DR6. The test looks right but for sure I would like to reproduce it with some reproducer.
Flavia could you provide some reproducer that is possible to run on EAP 6.4.x ?

Comment 8 Flavia Rainone 2016-11-22 12:42:17 UTC
Yes, the reproducer I used was provided by Tom Ross, based on the client's code. Woult that be enough?
This is the reproducer that I used to check if the fix was good.

Comment 9 Jiří Bílek 2016-11-22 13:04:17 UTC
Could you please attache the reproducer to this bz or send me it by en email?

Comment 10 Flavia Rainone 2016-11-22 14:49:40 UTC
I'll try to attach it to this bz, and I'll send an e-mail to Tom asking him to provision a database. The problem is that the reproducer is an app and requires a provisioned dataase, I'll send additional data by e-mail.

Comment 11 Flavia Rainone 2016-11-22 18:49:50 UTC
Created attachment 1222820 [details]
client application that reproduces the bug (requires db provisioning)

I'll explain in detail how to reproduce the bug once I have confirmed that we can have the db2 provisioned.

Comment 12 Jiří Bílek 2016-11-23 08:25:21 UTC
Thank you for reproducer. Provisioning of db2 is not necessary I can use our db2.

Comment 13 Flavia Rainone 2016-11-24 17:26:26 UTC
Okay, here are the instructions:
To reproduce  the issue follow the steps:
 - unzip the bundle.zip into, let’s say TEST_LOG directory 

 - build the test case project

      cd $TEST_LOC/bundle/test-deadlock
      mvn clean package


 - install JBoss EAP 6.4 CP9 
 - create two instance server-1 and server-2

      cd $JBOSS_HOME
      cp -rp standalone standalone-server-1
      cp -rp standalone standalone-server-2

      cp  $TEST_LOC/bundle/test-deadlock/eap-config/*.properties $JBOSS_HOME/standalone-server-1/configuration
      cp  $TEST_LOC/bundle/test-deadlock/eap-config/*.properties $JBOSS_HOME/standalone-server-2/configuration

      cp $TEST_LOC/bundle/test-deadlock/eap-config/server-1/standalone.xml to $JBOSS_HOME/standalone-server-1/configuration
      cp $TEST_LOC/bundle/test-deadlock/eap-config/server-2/standalone.xml to $JBOSS_HOME/standalone-server-2/configuration


 unzip the $TEST_LOC/bundle/db2.zip into $JBOSS_HOME/modules. db2.zip contains DB@ JDBC driver.

     cp $TEST_LOC/bundle/db2.zip $JBOSS_HOME/modules
     cd $JBOSS_HOME/module
     unzip db2.zip



start two JBoss instances

    cd $JBOSS_HOME/bin
     ./standalone.sh  -Djboss.server.base.dir=$JBOSS_HOME/standalone-server-1 -Djboss.node.name=server1


in another terminal session

    cd $JBOSS_HOME/bin
    ./standalone.sh  -Djboss.server.base.dir=$JBOSS_HOME/standalone-server-2 -Djboss.node.name=server2 -Djboss.socket.binding.port-offset=100


in another terminal

    cd $JBOSS_HOME/bin
    ./jboss-cli.sh —connect

    deploy $TEST_LOC/bundle/test-deadlock/project1-war/target/project1-war-0.0.1-SNAPSHOT.war


in yet another terminal session 

    cd $JBOSS_HOME/bin
    ./jboss-cli.sh —connect —controller=localhost:10099
    deploy $TEST_LOC/bundle/test-deadlock/project2-war/target/project2-war-0.0.1-SNAPSHOT.war


(I don’t think that jboss-cli accepts environment variables so you may have to substitute full patch for ${TEST_LOC} in jboss-cli)

Now run the client:

    cd $TEST_LOC/bundle/test-deadlock
    mvn exec:java


Then select “s” from the option list. 

If the bug is fixed, s will run, otherwise, there will be a deadlock and s will take a while to return, and will return with an error.

Comment 14 Jiří Bílek 2016-11-29 13:58:39 UTC
Thank you for reproducer.

A behavior of reproducer is the same for 6.4.10 as for 6.4.12. Insert into database is everytime succesfully finished but client is frozen.

Integration test is failing for both EAPs.

Reopened

Comment 16 Petr Penicka 2016-11-29 15:39:57 UTC
PM call: cloned into a new issue for 6.4.13 (or possibly 6.4.14 if Eng can't fix in time): https://bugzilla.redhat.com/show_bug.cgi?id=1399703

Closing this one as CLOSED NEXTRELEASE.


Note You need to log in before you can comment on or make changes to this bug.