Bug 854598
| Summary: | [ovirt-engine-backend] - Flooding Logs - server.log & engine.log are flooded when postgresql is stopped in Remote DB Setup | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | David Botzer <dbotzer> | ||||
| Component: | ovirt-engine | Assignee: | Yair Zaslavsky <yzaslavs> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | David Botzer <dbotzer> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | 3.1.0 | CC: | bazulay, dyasny, hateya, iheim, lpeer, mkenneth, oourfali, pstehlik, Rhev-m-bugs, yeylon, ykaul | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | infra | ||||||
| Fixed In Version: | si19 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2012-12-04 20:01:32 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | Infra | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
If postgresql is down, an administrator never suspect the rhevm logs are getting flooded. So the admin can leave the postgres issue to be dealt in lower priority not realizing the impact it has on other components. Also realizing this issue, could bring him to resolution of setting up the system differently. Happens only on remote Setup Moved severity to medium - per comment #1 David - When you say flood, what do you mean? how many logs a minute are written? engine.log resolution of a second every second, 130 messages like this -> 2012-09-05 14:48:09,184 ERROR [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8009-6) Error in getting DB connection. The database is inaccessible. Original exception is: DataAccessResourceFailureException: Error retreiving database metadata; nested exception is org.springframework.jdbc.support.MetaDataAccessException: Could not get Connection for extracting meta data; nested exception is org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource ------------------------------------------ server.log resolution of a second every second, 700 messages like this -> 2012-09-05 15:01:46,993 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubject] (ajp-/127.0.0.1:8009-4) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: Could not create connection From checking the logs - Error in getting DB connection. The database is inaccessible. Original exception is: DataAccessResourceFailureException: Error retreiving Is logged when backend starts, but the db is inaccessible. What happens when backend is up, and then the server goes down? Sorry, I didnt understood the message, --- db is unaccessible because I did service postgresql stop while ovirt-engine is started when starting postgresql its ok, David, let me clarify my question/request - Can you please do the same after ovirt-engine is up for a while, and see how this behaves? Hi, Let me fix the bug steps: 1. I stopped postgresql on the remote DB server 2. I Restart the ovirt-engine service and only than the flooding started, I stand corrected: Same issue on local setup. 1. I stopped postgresql on the remote DB server 2. I Restart the ovirt-engine service and only than the flooding started, David explained me the issue is with service restart after db is down. The following is an upstream patch to relax the flooding: http://gerrit.ovirt.org/#/c/7928/ Upstream merge http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=commit;h=b5f6d5bd4887e9c3b0f90bbff7a52e90513bbfd9 Fixed, 3.1/si19.1 I repeated the same sequence and no flooding in the logs, Nor after I started sql server, Webadmin and reports are accessibly - ok, Fixed, 3.1/si19.1 |
Created attachment 610012 [details] remote-sql-stop Description of problem: High rate of logs in server.log & engine.log when postgresql is stopped in Remote DB Setup Version-Release number of selected component (if applicable): 3.1/si17 How reproducible: always Steps to Reproduce: 1.Install rhevm_reports+dwh in Remote DB setup 2.create 1 entity of a kind (DC,CL, Host,VM, Template) 3.Stop the postgresql service on the remote DB server Actual results: always Expected results: When the same issue is reported it should send messages in low rate at a longer period of time between messages Additional info: