Bug 752110 - RHQ server OOM error
RHQ server OOM error
Status: NEW
Product: RHQ Project
Classification: Other
Component: No Component (Show other bugs)
4.2
Unspecified Unspecified
unspecified Severity unspecified (vote)
: ---
: ---
Assigned To: RHQ Project Maintainer
Mike Foley
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2011-11-08 10:27 EST by Alan Santos
Modified: 2015-02-05 20:19 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
latest server log (1.93 MB, text/x-log)
2011-11-08 10:30 EST, Alan Santos
no flags Details
agent log (4.31 MB, text/x-log)
2011-11-08 10:31 EST, Alan Santos
no flags Details

  None (edit)
Description Alan Santos 2011-11-08 10:27:38 EST
Description of problem:
Left RHQ server 4.2 running over night. Unresponsive in the AM, logs showed the server was out of heap space. 

Version-Release number of selected component (if applicable):


How reproducible:
Happened 2x.

Steps to Reproduce:
1.Started server
2.Imported agent running on same machine
3.Configured Apache, Postgres
4. Created drift template, made a change to monitored directory
5. Went away for a long time.
Comment 1 Alan Santos 2011-11-08 10:30:23 EST
Created attachment 532312 [details]
latest server log
Comment 2 Alan Santos 2011-11-08 10:31:31 EST
Created attachment 532313 [details]
agent log
Comment 3 Alan Santos 2011-11-08 10:32:43 EST
I have heap dump if helpful, but it's 1.3 GB
Comment 4 Charles Crouch 2011-11-09 13:52:34 EST
I note that you are running on H2. I presume you haven't seen this on PG or 
ORA? From the server logs there appear to be a lot of DB related errors prior 
to the memory issues, e.g. trying to redeliver a message to the JMS dead letter 
queue (dlq) nearly 900,000 times. Also exceptions such as "Caused by: 
java.lang.ClassCastException: org.h2.jdbc.JdbcBlob cannot be cast to 
org.jboss.mq.SpyMessage" that I'm very suspicious of. 

Given the secondary level support we have for H2, below PG and Oracle, I don't 
see this a release blocker. Mike confirmed that he frequently runs drift 
overnight using Oracle as the DB and has not observed similar issues. I will 
make this issue block the drift tracking bug, but set it as medium priority. 
Once we're done with major drift related issues for the release, we can see if 
we can reproduce this.
Comment 5 Alan Santos 2011-11-09 14:02:04 EST
that's fair. I'm only this build/database on the laptop that's acting like a desktop.  I'll swap out the database to pg and see if it reoccurs. 

I guess - secondary to this bz - is what caused the missing queue. Is it possible to simulate a similar JMS dead letter failure using another database?
Comment 6 Heiko W. Rupp 2011-11-09 15:10:39 EST
We used to have huge issues when jms was on hsqldb (the predecessor of h2). While they say h2 is much much better than hsqldb, it may be the same here in the end.

Note You need to log in before you can comment on or make changes to this bug.