Red Hat Bugzilla – Full Text Bug Listing
|Summary:||RHQ server OOM error|
|Product:||[Other] RHQ Project||Reporter:||Alan Santos <asantos>|
|Component:||No Component||Assignee:||RHQ Project Maintainer <rhq-maint>|
|Status:||NEW ---||QA Contact:||Mike Foley <mfoley>|
|Fixed In Version:||Doc Type:||Bug Fix|
|Doc Text:||Story Points:||---|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
Description Alan Santos 2011-11-08 10:27:38 EST
Description of problem: Left RHQ server 4.2 running over night. Unresponsive in the AM, logs showed the server was out of heap space. Version-Release number of selected component (if applicable): How reproducible: Happened 2x. Steps to Reproduce: 1.Started server 2.Imported agent running on same machine 3.Configured Apache, Postgres 4. Created drift template, made a change to monitored directory 5. Went away for a long time.
Comment 3 Alan Santos 2011-11-08 10:32:43 EST
I have heap dump if helpful, but it's 1.3 GB
Comment 4 Charles Crouch 2011-11-09 13:52:34 EST
I note that you are running on H2. I presume you haven't seen this on PG or ORA? From the server logs there appear to be a lot of DB related errors prior to the memory issues, e.g. trying to redeliver a message to the JMS dead letter queue (dlq) nearly 900,000 times. Also exceptions such as "Caused by: java.lang.ClassCastException: org.h2.jdbc.JdbcBlob cannot be cast to org.jboss.mq.SpyMessage" that I'm very suspicious of. Given the secondary level support we have for H2, below PG and Oracle, I don't see this a release blocker. Mike confirmed that he frequently runs drift overnight using Oracle as the DB and has not observed similar issues. I will make this issue block the drift tracking bug, but set it as medium priority. Once we're done with major drift related issues for the release, we can see if we can reproduce this.
Comment 5 Alan Santos 2011-11-09 14:02:04 EST
that's fair. I'm only this build/database on the laptop that's acting like a desktop. I'll swap out the database to pg and see if it reoccurs. I guess - secondary to this bz - is what caused the missing queue. Is it possible to simulate a similar JMS dead letter failure using another database?
Comment 6 Heiko W. Rupp 2011-11-09 15:10:39 EST
We used to have huge issues when jms was on hsqldb (the predecessor of h2). While they say h2 is much much better than hsqldb, it may be the same here in the end.