Bug 1026836 - Job scheduled does not recover when postgres connection is terminated for a while
Job scheduled does not recover when postgres connection is terminated for a w...
Product: JBoss Operations Network
Classification: JBoss
Component: Core Server (Show other bugs)
JON 3.2
Unspecified Unspecified
unspecified Severity low
: ER02
: JON 3.3.0
Assigned To: RHQ Project Maintainer
Mike Foley
Depends On:
  Show dependency treegraph
Reported: 2013-11-05 09:05 EST by Libor Zoubek
Modified: 2015-11-01 19:43 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2014-12-11 09:00:13 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Libor Zoubek 2013-11-05 09:05:50 EST
Description of problem:

When JON server looses connection ton postgres database for a while (just by restarting it) it goes to state when everything seems to be working, but it cannot schedule quartz job.

This can be observed for example when you deploy a bundle. Having server in such broken state means that bundle overall deployment status will stay IN_PROGRESS forever, because JON was unable to schedule job fixing this state based on agent's bundle deployment results.

Version-Release number of selected component (if applicable):
JON 3.2.ER4

How reproducible: always

Steps to Reproduce:
1. run JON server
2. restart database backend

Actual results: quartz does not recover from closed db connection and keeps throwing (when there is an attempt to schedule someting):

releaseTriggerRetryLoop: RuntimeException null: java.lang.reflect.UndeclaredThrowableException
        at $Proxy149.rollback(Unknown Source)   at org.quartz.impl.jdbcjobstore.JobStoreSupport.rollbackConnection(JobStoreSupport.java:3601) [quartz-1.6.5.jar:1.6.5]
        at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3764) [quartz-1.6.5.jar:1.6.5]
        at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3722) [quartz-1.6.5.jar:1.6.5]
        at org.quartz.impl.jdbcjobstore.JobStoreSupport.releaseAcquiredTrigger(JobStoreSupport.java:2805) [quartz-1.6.5.jar:1.6.5]
        at org.quartz.core.QuartzSchedulerThread.releaseTriggerRetryLoop(QuartzSchedulerThread.java:534) [quartz-1.6.5.jar:1.6.5]
        at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:354) [quartz-1.6.5.jar:1.6.5]
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.GeneratedMethodAccessor277.invoke(Unknown Source) [:1.7.0_09-icedtea]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_09-icedtea]
        at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_09-icedtea]
        at org.quartz.impl.jdbcjobstore.AttributeRestoringConnectionInvocationHandler.invoke(AttributeRestoringConnectionInvocationHandler.java:71) [quartz-1.6.5.jar:1.6.5]
        ... 7 more
Caused by: org.postgresql.util.PSQLException: This connection has been closed.

Expected results: JON server should be fully-working when DB connection is restored.

Additional info:
Comment 2 Jay Shaughnessy 2014-09-04 16:43:35 EDT
This looks like a duplicate of Bug 1095016.  I'm setting to ON_QA in case you think it needs validation but you can likely just close as a duplicate.

Note You need to log in before you can comment on or make changes to this bug.