Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1084686

Summary: separation between process/task engine and audit trail persistence
Product: [Retired] JBoss BPMS Platform 6 Reporter: jbride <jbride>
Component: Business CentralAssignee: Maciej Swiderski <mswiders>
Status: CLOSED WONTFIX QA Contact: Ivo Bek <ibek>
Severity: medium Docs Contact: Tomas 'Sheldon' Radej <tradej>
Priority: medium    
Version: 6.0.0CC: kverlaen, lpetrovi, rrajasek, tradej
Target Milestone: DR1   
Target Release: 6.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-13 11:28:30 UTC Type: Enhancement
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
persistence.xml with runtime and audit persistence units
none
persistence xml used with the fix none

Description jbride@redhat.com 2014-04-05 13:17:26 UTC
Hi.
  I'm requesting the ability to configure business-central to operate in a BPMS6 deployment architecture where the RDBMS used to support the process/human task engines is separate from the database used for BAM events.
  Via the optional system property of:  jbpm.audit.jms.enabled=true, its currently possible to send BAM events to a queue.  It's also currently possible to configure org.jbpm.process.audit.jms.AsyncAuditLogReceiver to persist BAM events to a database different from the database used to support the process/human task engines.  Through load testing, it has been demonstrated that this architecture allows the jbpm process engine to operate faster (and is more consistent) than the default configuration.
  In particular, I'm requesting that business-central allow for an optional configuration of it's persistence.xml where their could be two persistence units defined :
  1)  org.jbpm.domain
  2)  org.jbpm.audit

The mapping-file, Servicesorm.xml, along with the bam specific tables could optionally be moved to this secondary persistence unit, org.jbpm.audit.  Subsequently, jbpm/jbpm-services/jbpm-kie-services/src/main/java/org/jbpm/kie/services/impl/RuntimeDataServiceImpl.java could be optionally configured to inject a TransactionalCommandService whose EntityManagerFactory used:  org.jbpm.audit rather than the default:  org.jbpm.domain.

Probably the last consideration would be to ensure that the business-central UI gracefully handles scenarios where BAM data is not currently present in the org.jbpm.audit database.  When jbpm.audit.jms.enabled=true is set, there is no certainty as to when the BAM events would actually get persisted to the separated BAM database.

thank!  jeff

Comment 2 Mauricio Salatino 2014-04-06 17:55:49 UTC
Hi Jeff,
Thanks for creating this Bugzilla, we are currently trying to align the audit logs in the tasks, the processes and the BAM. So it is really good to get your input and this particular requirement.

Comment 4 Maciej Swiderski 2014-05-12 13:53:26 UTC
This should be possible based on the work done for deployment descriptors (https://bugzilla.redhat.com/show_bug.cgi?id=1017327) that allows to configure persistence unit name for audit and runtime separately.
So configuring the persistence unit names would be sufficient when using JPA mode of audit logs. Though for JMS based separate queues are needed to be created and then configuration file (jbpm.audit.jms.properties) would need to be added to kjar to instruct the runtime what queue it should use.

Would this be enough or we need more advanced support?

Comment 5 jbride@redhat.com 2014-05-12 14:05:09 UTC
Hi.  Seems reasonable.  One minor related BZ is this:  https://bugzilla.redhat.com/show_bug.cgi?id=1083807

jeff

Comment 6 Ivo Bek 2015-02-10 12:25:26 UTC
Verification failed in BPMS 6.1.0.ER4

1)

The audit persistence unit name in a deployment descriptor does not seem to be used because when a non-existing persistence unit is set, there is no exception and the audit data are persisted in the configured persistence unit name. However, when I set an incorrect persistence unit name, I get "No Persistence provider for EntityManager named org.jbpm.domain2" during Build&Deploy but nothing like that happens for the audit persistence unit.

Of couse, first, I tried a correct usage to persist audit data to a separate database jbpm-audit but in spite of that all data are persisted into the database defined by the persistence unit.

Therefore, the best way to reproduce the problem is to set an incorrect audit persistence unit in a deployment descriptor (does not produce an error).

2)

I also had the following issue during Dashbuilder deployment when I configured Dashbuilder to use only the jbpm-audit database and Business Central was using full jbpm database. Dashbuilder could not find the database tables which are created by Business Central. A workaround was to first let Business Central fully deploy (and create tables also for the second persistence unit) and then deploy Dashbuilder.

Comment 7 Ivo Bek 2015-02-10 12:27:48 UTC
Created attachment 990064 [details]
persistence.xml with runtime and audit persistence units

Comment 8 Maciej Swiderski 2015-02-11 17:28:42 UTC
initially this was done for process engine logs only but with following commits it was extended to support process, tasks and bam logs to be separated. Though this will work only for global configuration such as bia default deployment descriptor that will affect all projects on the runtime. It won't work properly for changed audit persistence unit done on project/kjar level. 

This is mainly because of CDI that initializes services on server startup and by that has no way to be altered once they are setup. Anyway that should still bring quite a lot of value to be able to split runtime and audit log on db level.

Here are the commits on master only, note that to be able to have proper separation one of the queries in servers has to be changed to remove use of TaskImpl entity and that enforced addition of one column (workitemid) on AuditTaskImpl

droolsjbpm-knowledge
master:
https://github.com/droolsjbpm/droolsjbpm-knowledge/commit/5764f2c016cbf9be24166e9d0e9ff0b8d4831fcd

jbpm
master:
https://github.com/droolsjbpm/jbpm/commit/999c5c693a268b4fe5769d8f1234e4b40addda0d

jbpm-console-ng
master:
https://github.com/droolsjbpm/jbpm-console-ng/commit/3fc8e4878f7f554d7a8ed1fd587b9ed47749c014

kie-wb-distributions
master:
https://github.com/droolsjbpm/kie-wb-distributions/commit/b9e2107dc4846a9a44613279668581763dd37f14

waiting for acks before back porting to 6.2.x so leaving in assigned state

Comment 9 Maciej Swiderski 2015-02-11 17:52:53 UTC
Created attachment 990556 [details]
persistence xml used with the fix

Comment 11 Maciej Swiderski 2015-12-07 16:44:21 UTC
moving to modified as it should be included in 6.2

Comment 12 Ivo Bek 2016-04-12 15:37:39 UTC
Currently, after some changes applied to have powerful services and queries, it's not possible to separate runtime and audit data because at least the 2 following queries require joining of tables across the 2 databases.

Error in named query: getAllGroupAuditTasksByUser: org.hibernate.hql.internal.ast.QuerySyntaxException: TaskImpl is not mapped [select distinct t from org.jbpm.services.task.audit.impl.model.AuditTaskImpl t, TaskImpl ti join ti.peopleAssignments.potentialOwners pos where (t.actualOwner = null or t.actualOwner = '') and t.status = 'Ready' and ti.id = t.taskId and pos.id in :potentialOwners order by t.taskId DESC]

Error in named query: getAllAdminAuditTasksByUser: org.hibernate.hql.internal.ast.QuerySyntaxException: TaskImpl is not mapped [select distinct t from org.jbpm.services.task.audit.impl.model.AuditTaskImpl t, TaskImpl ti join ti.peopleAssignments.businessAdministrators bas where ti.id = t.taskId and bas.id in :businessAdmins order by t.taskId DESC]

Comment 13 Lukáš Petrovický 2016-04-13 11:28:30 UTC
The situation here is actually quite complicated.

It seems that we have done this in 6.2, without telling QE. Which is why, in 6.3, we have changed it again (due to an RFE) and made this very thing absolutely impossible. :-)

At this point, I don't think there is anything we can do. We should perhaps document that this happened, and that the separation is no longer technically possible. (Even though it may have been possible in 6.2 and only 6.2.)

Comment 14 Tomas 'Sheldon' Radej 2016-04-25 12:44:44 UTC
Documented in BXMSDOC-276.