Bug 1014612

Created attachment 806398 [details]
node two server log

First of all I think creation of the database schema should be performed right before its deployment. That way you avoid that kind of issues.  

In production environments, the schema creation is usually restricted to DBA staff, so to be honest I think we should not worry about of schema creation clashing because this is something that actually is not going to happen.

In fact, I think the BPMS installation tool will be responsible of the schema creation, if I'm not wrong. 

Therefore, I recommend try to create the schema first and then start up the two nodes.The schema creation files can be found here:

* https://github.com/droolsjbpm/dashboard-builder/tree/master/modules/dashboard-webapp/src/main/webapp/WEB-INF/etc/sql

David,

I agree that the scheme should be created by some DB admin and the installation tool could be also helpful. But I can see this deadlock even after creating the scheme before starting deployment. Take a look at the attached server logs - there are insert statements before deadlock occurred, so the scheme was already created and problem raised when tables were being initialized with data.

When the app starts for the first time, not only tries to initialize the schema but also creates some runtime data, i.e: the built-in showcase dashboard.

Is not possible to have two hosts trying to create the same runtime data at the same time. This is kinda like having two users trying to create the same document on the same folder at the same time, obviously, a concurrence issue is expècted to be raised in both cases.

The only solution I think can be applied here is to ensure only one server is started against a shared DB at a given time.

Hello David,

I understand there is a issue with concurrency - but if the application has an ability to initialize itself and it is supposed to be deployed to cluster, it should handle the initialization on several nodes and use some synchronization to ensure the deadlock in DB cannot occur.

To start only one server, deploy application and after that start another one is a workaround only for EAP in standalone mode. When using EAP domain, the application is deployed on whole group of nodes at one moment. Of course, we can find a similar workaround even for domain mode, but I doubt this is a way we want our customers to use EAP domain mode (if I simplify it, turning domain mode to standalone).

Hi,

We have implemented a "database locking" mechanism using a new table to register and handle the nodes connected to the database.

As one node starts the database population process, it sets a special status into its table row.. so the other nodes avoid initial modules DML execution.

I have tested this behavior with a EAP as a domain controller and 3 connected nodes, using a TCP shared H2 database.

NOTE: Due to only one node executes those DML sentences, the rest of the nodes startup quicker than the installer one. Until all sentences are not completely executes, navigation in other nodes is not possible (database is empty). After discussing with the team, this behaviour can be considered  negligible..

Commits for master
------------------

- https://github.com/droolsjbpm/dashboard-builder/commit/6f5ee9bc99ece6c2ec6c0496a1666f2e58b29c82

Commits for 6.0.x
------------------

- https://github.com/droolsjbpm/dashboard-builder/commit/b2dc14a269ded05e83e11bf94a2f5f83dd15b743

Another commit to master
------------------------
- https://github.com/droolsjbpm/dashboard-builder/commit/117d95007901c41b6553c999e75f8e01b70c76bf

6.0.x.
------
- https://github.com/droolsjbpm/dashboard-builder/commit/40a7e549dae42046a574239d3dc161feb1f1ac02

Verified on BPMS-6.0.0.ER5