Bug 779628 (SOA-1998)

Summary: Inconsistent behavior in jBPM between CP02 and CP03 for action exception handling.
Product: [JBoss] JBoss Enterprise SOA Platform 4 Reporter: Marek Baluch <mbaluch>
Component: Documentation, JBPM - within SOA, ConfigurationAssignee: Julian Coleman <jcoleman>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: urgent    
Version: 4.3 CP04 ER1CC: dlesage
Target Milestone: ---Keywords: Regression
Target Release: 4.3 CP04 GA   
Hardware: Unspecified   
OS: Unspecified   
URL: http://jira.jboss.org/jira/browse/SOA-1998
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-05-20 11:25:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
mylyn-context.zip
none
diffs none

Description Marek Baluch 2010-03-18 09:27:45 UTC
Date of First Response: 2010-03-18 10:37:36
Workaround Description: Add <field name="retries"><int value="2" /></field> to "jbpm.job.executor" bean in jbpm.cfg.xml.
project_key: SOA

In CP02 both JobExecutor and JCA-inflow will retry a jBPM action when an Exception is thrown. In CP03, JCA-inflow will do the same but JobExecutor will not. Though it is possible to achieve the same result also for JobExecutor, but we must set it's "retries" field to "2".

IMHO this parameter should be documented. Value 1 (which is default in the code) does not do any attempts to retry the failed jBPM action though it might suggest one attempt will be made.

Comment 2 David Le Sage 2010-03-18 22:58:07 UTC
Documented as a Known Issue in the Release Notes:


https://jira.jboss.org/jira/browse/SOA-1998

    A code regression has led to a problem whereby the JobExecutor will not attempt to process a
    jBPM action action when an exception has been thrown.

    To work around this issue, set the jbpm.job.executor bean to

    <field name="retries"><int value="2" /></field>

    in the jbpm.cfg.xml file. (A value of "2" means that it will attempt to reprocess the failed action;
    the default value of "1" means it will not.)


Comment 4 David Le Sage 2010-03-19 03:09:14 UTC
That's okay. We can transfer it to resolved category if and when it is fixed.  This is only preliminary draft work.  

Comment 5 Alejandro Guizar 2010-03-24 15:06:23 UTC
Prior to the fix for SOA-1880, the job executor did not properly decrease the Job.retries field after an persistence exception occurred because the Hibernate session was left in an inconsistent state. Jobs practically had an unlimited number of retries regardless of the Job.retries value.

The regression noted here is actually the result of the job.retries field being correctly updated by the job executor for the first time.

However, I agree with what Marek states in the description: "value 1 (which is default in the code) does not do any attempt to retry the failed jBPM job though it might suggest one attempt will be made". I will change the queries or the code so that value 1 effectively means one more attempt will be made.

As for the workaround proposed, it should be optional after I do the change above, nevertheless it is still advisable to give jobs more chances to execute. I suggest a minimum value of 3, since smaller values sometimes cause concurrency tests not to complete against databases such as mssql and sybase.

Comment 6 Alejandro Guizar 2010-03-24 22:28:58 UTC
Changed job executor to interpret JobExecutor.retries property as the number of attempts to execute a failed job (e.g. value="2" means up to 3 executions).
Documented all job executor properties in "Asynchronous continuations" chapter (async.xml).
Suggested configuration:
  <bean name="jbpm.job.executor" class="org.jbpm.job.executor.JobExecutor">
    <property name="jbpmConfiguration"><ref bean="jbpmConfiguration" /></property>
    <property name="name"><string value="JbpmJobExecutor" /></property>
    <property name="nbrOfThreads"><int value="1" /></property>
    <property name="retries"><int value="2" /></property>
    <property name="idleInterval"><int value="5000" /></property>
    <property name="maxIdleInterval"><int value="3600000" /><!-- 1 hour --></property>
    <property name="maxLockTime"><int value="600000" /><!-- 10 minutes --></property>
    <property name="lockMonitorInterval"><int value="60000" /><!-- 1 minute --></property>
  </bean>

Comment 7 Alejandro Guizar 2010-03-24 22:29:24 UTC
Attachment: Added: mylyn-context.zip


Comment 8 David Le Sage 2010-03-25 04:51:26 UTC
Hello Alejandro, 

Thanks for that detailed information.  I have rewritten the Release Note so that it now states the following:


https://jira.jboss.org/jira/browse/SOA-1998

    A code regression led to a problem whereby the JobExecutor would not attempt to process a
    jBPM action action when an exception had been thrown. (This was because, previously, the job
    executor did not properly decrease the value in the Job.retries field after the occurrence of a
    persistence exception, leaving jobs with an unlimited number of retries. The fix to that meant that
    the job executor was updating the value correctly for the first time.)

    To fix this issue, queries have been modified so that the value of 1 now means that one more
    attempt to retry will be made. (Previously, it had meant that there would be no attempt to
    reprocess the failed action.)

    An optional further step one may wish to undertake is to raise this value still higher. To do so set
    jbpm.job.executor bean to

    <field name="retries"><int value="3" -/></field>

    in the jbpm.cfg.xml file. Red Hat recommends setting it to 3 because smaller values can cause
    concurrency test failures on some databases.


Comment 9 David Le Sage 2010-03-25 05:21:05 UTC
I have now also added the data you added to async.xml to section 10.4 of the product documents.  Cheers.  :-)

Comment 10 Martin Vecera 2010-03-25 08:37:08 UTC
Closed by mistake.

Comment 11 Alejandro Guizar 2010-03-25 16:51:22 UTC
Thank you, David. Can you still make one last change to the release note? 

<field name="retries"><int value="3" -/></field>
should be
<property name="retries"><int value="3" /></property>

While the former works, it causes direct field injection instead of setter method invocation. I want to discourage the former.

Comment 12 David Le Sage 2010-03-26 00:15:00 UTC
Hello Alejandro, 

As per the comment above and our IRC discussion, the Release Note has been rewritten to reflect the use of "property" instead of "field" and a default value of 3.  Here is the text:


https://jira.jboss.org/jira/browse/SOA-1998

    A code regression led to a problem whereby the JobExecutor would not attempt to process a
    jBPM action action when an exception had been thrown. (This was because, previously, the job
    executor did not properly decrease the value in the Job.retries field after the occurrence of a
    persistence exception, leaving jobs with an unlimited number of retries. The fix to that meant that
    the job executor was updating the value correctly for the first time.)

    The default value for the retries is now 3. (The fix means that queries have been modified so that
    the value of 1 now means that one more attempt to retry will be made. Previously, it had meant
    that there would be no attempt to reprocess the failed action.)
    To manually modify this value, set the jbpm.job.executor bean in the following way:

    <property name="retries"><int value="3" /></property>

    in the jbpm.cfg.xml file. (Red Hat recommends setting it to a value no lower than 3 because
    smaller values can cause concurrency test failures on some databases.)

I have also moved the new section in the jBPM Guide from Chapter 10 to Chapter 3 and fixed the retry value in the table there and removed the code sample.  Cheers.  :-)


Comment 13 Alejandro Guizar 2010-03-26 02:21:38 UTC
Set default value of job executor retries to 3.
Moved job executor properties table to configuration chapter.

Comment 15 Alejandro Guizar 2010-03-26 21:22:02 UTC
Len, it is not just documentation. I also changed the default value of job executor retries to 3 (was 1).

Comment 16 Marek Baluch 2010-05-11 10:41:59 UTC
This problem still exists. If the retries property is not set in jbpm.cfg.xml then the actual value of retries in JobExecutor during process execution is 0. No other attempt will be made to execute the failed action.

Comment 17 Marek Baluch 2010-05-11 11:11:08 UTC
The configuration in default.jbpm.cfg.xml is correct. This configuration is overwritten by jbpm.cfg.xml in jbpm.esb dir where the retries property is missing - this causes the retries value in JobExecutor to be zero.

Comment 18 Marek Baluch 2010-05-11 11:42:00 UTC
Text in Release Notes (David's comment from: 25/Mar/10 08:15 PM) does not reflect the current state. There's no default setting to value '3' in jbpm.cfg.xml nor in the source code at this time.

Comment 19 Kevin Conner 2010-05-12 12:58:12 UTC
Am I right in thinking that setting the retry count is new behaviour in the standard jBPM JobExecutor?  If so then this now differs from the JCA inflow.

Comment 21 Len DiMaggio 2010-05-12 13:20:42 UTC
Link: Added: This issue is related to SOA-2075


Comment 22 Kevin Conner 2010-05-12 13:31:18 UTC
For 1 - "retries" is missing because that doesn't exist in jBPM 3.2.7 which it is based on
For 2 - that was true of the jBPM executor but not JCA inflow
For 3 - The JCA executor does not initialise the retry count as it is an attribute of the job.

It would appear that JBPM-2691 has addressed added this by setting the retry count on error and having the job executor initialise it, rather than having this done when the Job is created.

As such, every job executor now has to have its own mechanism but the problem with the JCA inflow is that the jBPM job executor is invalid within a transactional environment.


Comment 24 Kevin Conner 2010-05-13 09:27:09 UTC
The following is a cleaner way of initialising the job retries, as this will be visible to every executor

Comment 25 Kevin Conner 2010-05-13 09:27:09 UTC
Attachment: Added: diffs


Comment 26 Kevin Conner 2010-05-13 10:04:57 UTC
Created JBPM-2861 to reflect changes.

Comment 27 Alejandro Guizar 2010-05-17 20:43:04 UTC
The new way to initialize the job retries that Kevin brings forward looks good to me, although it implies further changes to the documentation. I'll apply the patch and edit the project user guide under JBPM-2861. The docs team can incorporate the changes into the product manual afterwards.

Comment 28 Alejandro Guizar 2010-05-18 02:16:49 UTC
Link: Added: This issue incorporates JBPM-2861


Comment 29 Alejandro Guizar 2010-05-18 04:08:00 UTC
Resolved JBPM-2861. The patch is applied and the project user guide updated.

Comment 30 David Le Sage 2010-05-18 04:28:41 UTC
Thank you. I have updated the release notes for this.

Comment 31 Julian Coleman 2010-05-20 11:21:35 UTC
Fixed with CR3 build (containing later jBPM code base).

Comment 33 Alejandro Guizar 2010-09-14 21:25:48 UTC
Link: Added: This issue is related to JBPM-1992