Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 825729 Details for
Bug 1031717
Force removal of DataCenter fails to remove vm_pool from DB
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
engine.log
engine.log (text/x-log), 1.51 MB, created by
sefi litmanovich
on 2013-11-18 15:28:16 UTC
(
hide
)
Description:
engine.log
Filename:
MIME Type:
Creator:
sefi litmanovich
Created:
2013-11-18 15:28:16 UTC
Size:
1.51 MB
patch
obsolete
>2013-11-12 04:47:18,841 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-66) Attempting to update VMs/Templates Ovf. >2013-11-12 04:47:18,842 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-66) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 04:47:18,853 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-66) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 04:47:18,853 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-66) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 04:47:18,856 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-66) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 04:47:18,856 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-66) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 04:47:18,859 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-66) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 04:48:53,193 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-2) Start refreshing all users data >2013-11-12 04:48:53,224 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-2) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 05:47:18,859 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-2) Attempting to update VMs/Templates Ovf. >2013-11-12 05:47:18,860 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-2) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 05:47:18,869 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-2) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 05:47:18,869 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-2) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 05:47:18,872 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-2) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 05:47:18,872 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-2) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 05:47:18,874 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-2) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 05:48:53,334 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-57) Start refreshing all users data >2013-11-12 05:48:53,811 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-57) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 06:47:18,875 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to update VMs/Templates Ovf. >2013-11-12 06:47:18,876 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 06:47:18,885 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 06:47:18,885 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 06:47:18,888 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 06:47:18,888 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 06:47:18,889 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 06:48:53,847 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-23) Start refreshing all users data >2013-11-12 06:48:54,484 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-23) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 07:47:18,889 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2013-11-12 07:47:18,890 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 07:47:18,899 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 07:47:18,899 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 07:47:18,899 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 07:47:18,900 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 07:47:18,900 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 07:48:54,518 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-68) [53ee3835] Start refreshing all users data >2013-11-12 07:48:54,714 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-68) [53ee3835] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 08:47:18,901 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) Attempting to update VMs/Templates Ovf. >2013-11-12 08:47:18,902 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 08:47:18,903 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 08:47:18,903 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 08:47:18,906 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 08:47:18,906 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 08:47:18,907 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 08:48:54,745 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-53) Start refreshing all users data >2013-11-12 08:48:55,116 ERROR [org.ovirt.engine.core.bll.adbroker.LDAPTemplateWrapper] (DefaultQuartzScheduler_Worker-53) Error in running LDAP query. BaseDN is , filter is (|(objectGUID=9b9002d1-ec33-4083-8a7b-31f6b8931648)). Exception message is: Request: 4 cancelled; nested exception is javax.naming.CommunicationException: Request: 4 cancelled; remaining name '' >2013-11-12 08:48:55,116 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-53) Error in communicating with LDAP server Request: 4 cancelled; nested exception is javax.naming.CommunicationException: Request: 4 cancelled; remaining name '' >2013-11-12 08:48:55,117 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-53) Failed ldap search server LDAP://qa1.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to communication error. We should try the next server >2013-11-12 08:49:25,504 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-53) Error in communicating with LDAP server qa2-tlv.qa.lab.tlv.redhat.com:389; nested exception is javax.naming.CommunicationException: qa2-tlv.qa.lab.tlv.redhat.com:389 [Root exception is java.net.SocketTimeoutException: connect timed out] >2013-11-12 08:49:25,504 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-53) Failed ldap search server LDAP://qa2-tlv.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to connection timeout. We should try the next server >2013-11-12 08:49:25,504 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-53) User vdcadmin not found in directory server, its status switched to InActive >2013-11-12 09:47:18,907 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update VMs/Templates Ovf. >2013-11-12 09:47:18,909 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 09:47:18,909 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 09:47:18,910 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 09:47:18,913 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 09:47:18,913 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 09:47:18,914 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 09:49:25,524 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-81) [55890714] Start refreshing all users data >2013-11-12 09:49:26,295 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-81) [55890714] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 09:49:26,445 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-81) [55890714] Inactive User vdcadmin found in directory server, its status switched to Active >2013-11-12 10:47:18,915 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to update VMs/Templates Ovf. >2013-11-12 10:47:18,916 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 10:47:18,925 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 10:47:18,925 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 10:47:18,925 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 10:47:18,925 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 10:47:18,926 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 10:49:26,459 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-39) Start refreshing all users data >2013-11-12 10:49:27,507 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-39) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 11:47:18,926 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update VMs/Templates Ovf. >2013-11-12 11:47:18,928 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 11:47:18,928 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 11:47:18,929 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 11:47:18,932 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 11:47:18,932 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 11:47:18,933 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 11:49:27,598 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-43) Start refreshing all users data >2013-11-12 11:49:27,623 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-43) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 12:47:18,933 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2013-11-12 12:47:18,948 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 12:47:18,949 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 12:47:18,949 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 12:47:18,952 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 12:47:18,952 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 12:47:18,953 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 12:49:27,676 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-31) Start refreshing all users data >2013-11-12 12:49:27,846 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-31) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 13:47:18,953 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Attempting to update VMs/Templates Ovf. >2013-11-12 13:47:18,954 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 13:47:18,964 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 13:47:18,964 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 13:47:18,967 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 13:47:18,967 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 13:47:18,968 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 13:49:27,921 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-30) Start refreshing all users data >2013-11-12 13:49:28,113 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-30) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 14:47:18,968 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2013-11-12 14:47:18,969 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 14:47:18,979 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 14:47:18,979 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 14:47:18,982 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 14:47:18,983 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 14:47:18,984 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 14:49:28,247 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-81) [55890714] Start refreshing all users data >2013-11-12 14:49:28,280 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-81) [55890714] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 15:47:18,984 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) Attempting to update VMs/Templates Ovf. >2013-11-12 15:47:18,985 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 15:47:18,994 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 15:47:18,994 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 15:47:18,998 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 15:47:18,998 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 15:47:18,999 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 15:49:28,373 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-28) Start refreshing all users data >2013-11-12 15:49:29,037 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-28) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 16:47:18,999 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Attempting to update VMs/Templates Ovf. >2013-11-12 16:47:19,000 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 16:47:19,009 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 16:47:19,009 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 16:47:19,013 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 16:47:19,013 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 16:47:19,014 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 16:49:29,075 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-77) Start refreshing all users data >2013-11-12 16:49:29,423 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-77) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 17:47:19,014 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update VMs/Templates Ovf. >2013-11-12 17:47:19,016 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 17:47:19,024 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 17:47:19,024 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 17:47:19,028 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 17:47:19,028 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 17:47:19,029 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 17:49:29,549 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-63) Start refreshing all users data >2013-11-12 17:49:29,589 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-63) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 18:47:19,030 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VMs/Templates Ovf. >2013-11-12 18:47:19,031 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 18:47:19,043 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 18:47:19,043 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 18:47:19,046 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 18:47:19,046 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 18:47:19,047 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 18:49:29,643 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-9) Start refreshing all users data >2013-11-12 18:49:30,360 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-9) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 19:47:19,047 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Attempting to update VMs/Templates Ovf. >2013-11-12 19:47:19,049 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 19:47:19,058 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 19:47:19,058 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 19:47:19,058 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 19:47:19,058 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 19:47:19,059 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 19:49:30,396 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-59) Start refreshing all users data >2013-11-12 19:49:30,571 ERROR [org.ovirt.engine.core.bll.adbroker.LDAPTemplateWrapper] (DefaultQuartzScheduler_Worker-59) Error in running LDAP query. BaseDN is , filter is (|(objectGUID=9b9002d1-ec33-4083-8a7b-31f6b8931648)). Exception message is: Request: 4 cancelled; nested exception is javax.naming.CommunicationException: Request: 4 cancelled; remaining name '' >2013-11-12 19:49:30,571 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-59) Error in communicating with LDAP server Request: 4 cancelled; nested exception is javax.naming.CommunicationException: Request: 4 cancelled; remaining name '' >2013-11-12 19:49:30,572 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-59) Failed ldap search server LDAP://qa1.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to communication error. We should try the next server >2013-11-12 19:50:01,220 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-59) Error in communicating with LDAP server qa2-tlv.qa.lab.tlv.redhat.com:389; nested exception is javax.naming.CommunicationException: qa2-tlv.qa.lab.tlv.redhat.com:389 [Root exception is java.net.SocketTimeoutException: connect timed out] >2013-11-12 19:50:01,220 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-59) Failed ldap search server LDAP://qa2-tlv.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to connection timeout. We should try the next server >2013-11-12 19:50:01,221 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-59) User vdcadmin not found in directory server, its status switched to InActive >2013-11-12 20:47:19,060 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) [53ee3835] Attempting to update VMs/Templates Ovf. >2013-11-12 20:47:19,061 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) [53ee3835] Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 20:47:19,069 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) [53ee3835] Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 20:47:19,069 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) [53ee3835] Attempting to update template OVFs in Data Center new-nfs >2013-11-12 20:47:19,073 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) [53ee3835] Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 20:47:19,073 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) [53ee3835] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 20:47:19,073 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) [53ee3835] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 20:50:01,224 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-76) Start refreshing all users data >2013-11-12 20:50:01,939 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-76) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 20:50:02,037 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-76) Inactive User vdcadmin found in directory server, its status switched to Active >2013-11-12 21:47:19,074 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Attempting to update VMs/Templates Ovf. >2013-11-12 21:47:19,075 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 21:47:19,076 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 21:47:19,076 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 21:47:19,076 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 21:47:19,076 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 21:47:19,077 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 21:50:02,042 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-39) Start refreshing all users data >2013-11-12 21:50:02,075 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-39) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 22:47:19,077 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update VMs/Templates Ovf. >2013-11-12 22:47:19,078 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 22:47:19,087 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 22:47:19,087 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update template OVFs in Data Center new-nfs >2013-11-12 22:47:19,090 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 22:47:19,090 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 22:47:19,091 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 22:50:02,104 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-20) Start refreshing all users data >2013-11-12 22:50:02,310 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-20) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-12 23:47:19,091 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Attempting to update VMs/Templates Ovf. >2013-11-12 23:47:19,092 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Attempting to update VM OVFs in Data Center new-nfs >2013-11-12 23:47:19,101 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Successfully updated VM OVFs in Data Center new-nfs >2013-11-12 23:47:19,101 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Attempting to update template OVFs in Data Center new-nfs >2013-11-12 23:47:19,104 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Successfully updated templates OVFs in Data Center new-nfs >2013-11-12 23:47:19,104 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 23:47:19,105 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-12 23:50:02,386 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-33) Start refreshing all users data >2013-11-12 23:50:02,506 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-33) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 00:47:19,105 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Attempting to update VMs/Templates Ovf. >2013-11-13 00:47:19,106 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 00:47:19,107 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 00:47:19,107 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 00:47:19,110 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 00:47:19,110 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 00:47:19,111 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 00:50:02,541 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-91) Start refreshing all users data >2013-11-13 00:50:02,570 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-91) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 01:47:19,111 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Attempting to update VMs/Templates Ovf. >2013-11-13 01:47:19,113 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 01:47:19,122 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 01:47:19,122 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 01:47:19,125 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 01:47:19,125 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 01:47:19,126 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 01:50:02,650 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-54) Start refreshing all users data >2013-11-13 01:50:02,722 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-54) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 02:47:19,126 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Attempting to update VMs/Templates Ovf. >2013-11-13 02:47:19,127 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 02:47:19,128 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 02:47:19,128 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 02:47:19,129 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 02:47:19,129 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 02:47:19,129 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 02:48:15,496 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-13) Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:15,498 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-89) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:15,520 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-13) [51857563] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 02:48:15,522 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:15,725 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-13) [51857563] Correlation ID: 51857563, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center new-nfs. Setting Data Center status to Non Responsive (On host rose07.qa.lab.tlv.redhat.com, Error: Network error during communication with the Host.). >2013-11-13 02:48:15,727 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) [51857563] IrsBroker::Failed::GetStoragePoolInfoVDS due to: UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:18,647 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) [7994e59e] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:18,647 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) [7994e59e] Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:21,770 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) [6bd22279] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:21,770 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) [6bd22279] Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:24,893 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:24,894 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:58,740 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:58,742 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [2e0a2893] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:58,742 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [2e0a2893] Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:48:58,743 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) IrsBroker::Failed::GetStoragePoolInfoVDS due to: UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:49:01,869 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:49:01,870 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Server failed to respond, vds_id = d71e8f58-702c-42f3-9464-787ae4db30d6, vds_name = rose07.qa.lab.tlv.redhat.com, vm_count = 0, spm_status = SPM, non-responsive_timeout (seconds) = 80, error = java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:49:02,004 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:102) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:28) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:532) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:493) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:344) > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:235) > at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) > at com.sun.proxy.$Proxy61.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor121.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:225) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) >, Custom Event ID: -1, Message: Host rose07.qa.lab.tlv.redhat.com is non responsive. >2013-11-13 02:49:02,030 INFO [org.ovirt.engine.core.bll.VdsEventListener] (pool-5-thread-49) ResourceManager::vdsNotResponding entered for Host d71e8f58-702c-42f3-9464-787ae4db30d6, rose07.qa.lab.tlv.redhat.com >2013-11-13 02:49:02,182 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (pool-5-thread-49) [8660fd5] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-13 02:49:02,500 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (pool-5-thread-49) [8660fd5] SSH connection to host rose07.qa.lab.tlv.redhat.com failed: java.nio.channels.UnresolvedAddressException: java.nio.channels.UnresolvedAddressException > at sun.nio.ch.Net.checkAddress(Net.java:127) [rt.jar:1.7.0_40] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:640) [rt.jar:1.7.0_40] > at org.apache.mina.transport.socket.nio.NioSocketConnector.connect(NioSocketConnector.java:188) [mina-core.jar:] > at org.apache.mina.transport.socket.nio.NioSocketConnector.connect(NioSocketConnector.java:46) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector.connect0(AbstractPollingIoConnector.java:319) [mina-core.jar:] > at org.apache.mina.core.service.AbstractIoConnector.connect(AbstractIoConnector.java:262) [mina-core.jar:] > at org.apache.mina.core.service.AbstractIoConnector.connect(AbstractIoConnector.java:172) [mina-core.jar:] > at org.apache.sshd.SshClient.connect(SshClient.java:194) > at org.apache.sshd.SshClient.connect(SshClient.java:185) > at org.ovirt.engine.core.utils.ssh.SSHClient.connect(SSHClient.java:278) [utils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:73) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:43) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1131) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1216) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1892) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1236) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:636) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_40] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:235) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:218) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$1.run(VdsEventListener.java:132) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 02:49:02,752 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-5-thread-49) [58cdab0b] Fence failed on vds rose07.qa.lab.tlv.redhat.com which is spm of pool a44a18ab-6f2e-48d4-ab60-889845f498d3 - moving pool to non operational >2013-11-13 02:49:02,753 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-5-thread-49) [4b26d036] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 02:49:02,876 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [4b26d036] Correlation ID: 4b26d036, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager rose07.qa.lab.tlv.redhat.com for Data Center new-nfs. Setting status to Non-Operational. >2013-11-13 02:49:02,876 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-5-thread-49) [4b26d036] Failed to run Fence script on vds:rose07.qa.lab.tlv.redhat.com, VMs moved to UnKnown instead. >2013-11-13 02:49:02,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [4b26d036] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rose07.qa.lab.tlv.redhat.com became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2013-11-13 02:49:02,995 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-5-thread-49) [4b26d036] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VDS_FENCE_DISABLED >2013-11-13 02:49:05,168 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:49:08,288 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:49:44,427 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:49:47,567 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:49:50,695 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) [11a13db6] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:49:53,820 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) [6fa9173a] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:50:02,772 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-5) Start refreshing all users data >2013-11-13 02:50:29,969 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:50:33,108 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:50:35,788 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-5) Error in communicating with LDAP server qa1.qa.lab.tlv.redhat.com:389; nested exception is javax.naming.CommunicationException: qa1.qa.lab.tlv.redhat.com:389 [Root exception is java.net.UnknownHostException: qa1.qa.lab.tlv.redhat.com] >2013-11-13 02:50:35,789 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-5) Failed ldap search server LDAP://qa1.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to qa1.qa.lab.tlv.redhat.com:389; nested exception is javax.naming.CommunicationException: qa1.qa.lab.tlv.redhat.com:389 [Root exception is java.net.UnknownHostException: qa1.qa.lab.tlv.redhat.com]. We should try the next server >2013-11-13 02:50:36,233 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) [1889a93e] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:50:39,362 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 02:50:56,549 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GetHardwareInfoVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, vds=Host[rose07.qa.lab.tlv.redhat.com]), log id: 55cf25d9 >2013-11-13 02:50:56,565 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GetHardwareInfoVDSCommand, log id: 55cf25d9 >2013-11-13 02:50:56,716 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host rose07.qa.lab.tlv.redhat.com. >2013-11-13 02:50:56,722 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-34) [4f37746f] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 02:50:56,724 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-34) [7d44d9ae] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 02:50:56,852 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-34) [7d44d9ae] START, ConnectStorageServerVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 3d20d980 >2013-11-13 02:50:56,864 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-34) [7d44d9ae] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 3d20d980 >2013-11-13 02:50:56,864 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-34) [7d44d9ae] Host rose07.qa.lab.tlv.redhat.com storage connection was succeeded >2013-11-13 02:50:56,987 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-50) START, ConnectStoragePoolVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 1, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 552d8685 >2013-11-13 02:51:00,160 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-50) FINISH, ConnectStoragePoolVDSCommand, log id: 552d8685 >2013-11-13 02:51:00,439 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-34) [7d44d9ae] START, SetMOMPolicyParametersVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6), log id: 6604d42c >2013-11-13 02:51:00,479 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-34) [7d44d9ae] FINISH, SetMOMPolicyParametersVDSCommand, log id: 6604d42c >2013-11-13 02:51:00,606 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) [7d44d9ae] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rose07.qa.lab.tlv.redhat.com. >2013-11-13 02:51:00,730 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-34) [44824b87] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-13 02:51:00,853 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-34) [3ab0b0fd] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-13 02:51:05,795 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-5) Error in communicating with LDAP server qa2-tlv.qa.lab.tlv.redhat.com:389; nested exception is javax.naming.CommunicationException: qa2-tlv.qa.lab.tlv.redhat.com:389 [Root exception is java.net.SocketTimeoutException: connect timed out] >2013-11-13 02:51:05,796 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-5) Failed ldap search server LDAP://qa2-tlv.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to connection timeout. We should try the next server >2013-11-13 02:51:05,796 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-5) User vdcadmin not found in directory server, its status switched to InActive >2013-11-13 03:17:06,836 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:17:06,850 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-47) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:17:06,861 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-33) [44ca52eb] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 03:17:06,864 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:17:06,996 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [44ca52eb] Correlation ID: 44ca52eb, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center new-nfs. Setting Data Center status to Non Responsive (On host rose07.qa.lab.tlv.redhat.com, Error: Network error during communication with the Host.). >2013-11-13 03:17:06,999 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) [44ca52eb] IrsBroker::Failed::GetStoragePoolInfoVDS due to: UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:17:09,986 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:17:09,986 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:17:13,106 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:17:13,107 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:17:16,239 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:17:16,239 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:17:22,079 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) [18a26fcf] START, GetHardwareInfoVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, vds=Host[rose07.qa.lab.tlv.redhat.com]), log id: e8f8d08 >2013-11-13 03:17:22,099 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) [18a26fcf] FINISH, GetHardwareInfoVDSCommand, log id: e8f8d08 >2013-11-13 03:17:22,226 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [18a26fcf] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host rose07.qa.lab.tlv.redhat.com. >2013-11-13 03:17:22,232 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-6) [70f0ec85] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 03:17:22,233 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-6) [306288c1] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 03:17:22,358 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-6) [306288c1] START, ConnectStorageServerVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4410406f >2013-11-13 03:17:22,369 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-6) [306288c1] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 4410406f >2013-11-13 03:17:22,369 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-6) [306288c1] Host rose07.qa.lab.tlv.redhat.com storage connection was succeeded >2013-11-13 03:17:22,485 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-49) START, ConnectStoragePoolVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 1, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 47958108 >2013-11-13 03:17:27,653 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-49) FINISH, ConnectStoragePoolVDSCommand, log id: 47958108 >2013-11-13 03:17:27,893 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-6) [306288c1] START, SetMOMPolicyParametersVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6), log id: 7bfd25dd >2013-11-13 03:17:27,933 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-6) [306288c1] FINISH, SetMOMPolicyParametersVDSCommand, log id: 7bfd25dd >2013-11-13 03:17:28,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [306288c1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rose07.qa.lab.tlv.redhat.com. >2013-11-13 03:17:28,189 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-6) [7b1a9d83] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-13 03:17:28,314 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [e83e742] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-13 03:22:30,713 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-58) [11a13db6] Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:22:30,721 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-54) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:22:30,722 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-58) [18e72917] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 03:22:30,725 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:22:30,853 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-58) [18e72917] Correlation ID: 18e72917, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center new-nfs. Setting Data Center status to Non Responsive (On host rose07.qa.lab.tlv.redhat.com, Error: Network error during communication with the Host.). >2013-11-13 03:22:30,855 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) [18e72917] IrsBroker::Failed::GetStoragePoolInfoVDS due to: UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:22:33,855 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:22:33,855 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:22:36,975 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:22:36,975 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:22:40,098 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:22:40,099 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:23:13,877 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-100) Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:23:13,879 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:23:13,879 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Failed to refresh VDS , vds = d71e8f58-702c-42f3-9464-787ae4db30d6 : rose07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:23:13,880 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) IrsBroker::Failed::GetStoragePoolInfoVDS due to: UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:23:17,003 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:23:17,004 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Server failed to respond, vds_id = d71e8f58-702c-42f3-9464-787ae4db30d6, vds_name = rose07.qa.lab.tlv.redhat.com, vm_count = 0, spm_status = SPM, non-responsive_timeout (seconds) = 80, error = java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:23:17,148 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:102) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:28) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:532) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:493) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:344) > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:235) > at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) > at com.sun.proxy.$Proxy61.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor121.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:225) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) >, Custom Event ID: -1, Message: Host rose07.qa.lab.tlv.redhat.com is non responsive. >2013-11-13 03:23:17,152 INFO [org.ovirt.engine.core.bll.VdsEventListener] (pool-5-thread-50) ResourceManager::vdsNotResponding entered for Host d71e8f58-702c-42f3-9464-787ae4db30d6, rose07.qa.lab.tlv.redhat.com >2013-11-13 03:23:17,292 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (pool-5-thread-50) [259bf4f4] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-13 03:23:17,518 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (pool-5-thread-50) [259bf4f4] SSH connection to host rose07.qa.lab.tlv.redhat.com failed: java.nio.channels.UnresolvedAddressException: java.nio.channels.UnresolvedAddressException > at sun.nio.ch.Net.checkAddress(Net.java:127) [rt.jar:1.7.0_40] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:640) [rt.jar:1.7.0_40] > at org.apache.mina.transport.socket.nio.NioSocketConnector.connect(NioSocketConnector.java:188) [mina-core.jar:] > at org.apache.mina.transport.socket.nio.NioSocketConnector.connect(NioSocketConnector.java:46) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector.connect0(AbstractPollingIoConnector.java:319) [mina-core.jar:] > at org.apache.mina.core.service.AbstractIoConnector.connect(AbstractIoConnector.java:262) [mina-core.jar:] > at org.apache.mina.core.service.AbstractIoConnector.connect(AbstractIoConnector.java:172) [mina-core.jar:] > at org.apache.sshd.SshClient.connect(SshClient.java:194) > at org.apache.sshd.SshClient.connect(SshClient.java:185) > at org.ovirt.engine.core.utils.ssh.SSHClient.connect(SSHClient.java:278) [utils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:73) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:43) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1131) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1216) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1892) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1236) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:636) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_40] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:235) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:218) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$1.run(VdsEventListener.java:132) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 03:23:17,706 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-5-thread-50) [762a27ea] Fence failed on vds rose07.qa.lab.tlv.redhat.com which is spm of pool a44a18ab-6f2e-48d4-ab60-889845f498d3 - moving pool to non operational >2013-11-13 03:23:17,715 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-5-thread-50) [7bfe6414] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 03:23:17,839 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-50) [7bfe6414] Correlation ID: 7bfe6414, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager rose07.qa.lab.tlv.redhat.com for Data Center new-nfs. Setting status to Non-Operational. >2013-11-13 03:23:17,839 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-5-thread-50) [7bfe6414] Failed to run Fence script on vds:rose07.qa.lab.tlv.redhat.com, VMs moved to UnKnown instead. >2013-11-13 03:23:17,959 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-50) [7bfe6414] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rose07.qa.lab.tlv.redhat.com became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2013-11-13 03:23:17,959 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-5-thread-50) [7bfe6414] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VDS_FENCE_DISABLED >2013-11-13 03:23:20,305 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:23:23,436 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:23:59,582 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:24:02,703 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:24:05,827 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:24:08,950 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:24:45,084 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:24:48,207 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:24:51,331 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:24:54,456 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:25:30,592 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:25:33,721 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:25:36,849 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:25:39,976 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:26:16,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) [44ca52eb] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:26:19,257 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:26:22,382 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:26:25,505 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) [55890714] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:27:01,659 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) [3b758810] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:27:04,786 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:27:07,932 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:27:11,058 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:27:47,206 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) [353ac9e2] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:27:50,344 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:27:53,479 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:27:56,602 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:28:32,743 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:28:35,874 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:28:38,999 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:28:42,124 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:29:18,265 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:29:21,408 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:29:24,534 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) [20124fbd] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:29:27,660 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:30:03,825 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:30:06,955 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:30:10,087 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:30:13,213 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:30:49,351 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) [39a5d292] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:30:52,482 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:30:55,604 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) [60655011] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:30:58,722 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) [55890714] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:31:34,868 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:31:37,996 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:31:41,142 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:31:44,269 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:32:20,420 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:32:23,559 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:32:26,685 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:32:29,809 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:33:05,966 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:33:09,093 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:33:12,215 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:33:15,338 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:33:51,489 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:33:54,615 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:33:57,745 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:34:00,885 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:34:37,061 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:34:40,182 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:34:43,307 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) [53ee3835] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:34:46,430 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:35:22,574 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:35:25,696 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) [4a80fae] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:35:28,817 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:35:31,938 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:35:35,011 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-83) Start deleteAgedOutAuditLogs >2013-11-13 03:35:35,012 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-83) Finished deleteAgedOutAuditLogs >2013-11-13 03:36:08,091 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:36:11,246 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:36:14,386 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:36:17,511 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:36:53,666 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:36:56,787 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:36:59,917 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:37:03,041 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:37:39,178 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:37:42,305 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:37:45,431 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) [60655011] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:37:48,559 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:38:24,695 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:38:27,855 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:38:30,989 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:38:34,122 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:39:10,280 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:39:13,415 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) [353ac9e2] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:39:16,553 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:39:19,691 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:39:55,847 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:39:58,971 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:40:02,110 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:40:05,273 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:40:41,414 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:40:44,540 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:40:47,665 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:40:50,790 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) [20124fbd] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:41:26,928 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:41:30,055 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:41:33,183 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) [53ee3835] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:41:36,310 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:42:12,439 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) [148ca110] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:42:15,565 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:42:18,714 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:42:21,835 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:42:57,996 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:43:01,127 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:43:04,258 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:43:07,391 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:43:43,546 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:43:46,676 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:43:49,812 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:43:52,933 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:44:29,073 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:44:32,204 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:44:35,328 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:44:38,451 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: rose07.qa.lab.tlv.redhat.com >2013-11-13 03:44:55,646 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GetHardwareInfoVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, vds=Host[rose07.qa.lab.tlv.redhat.com]), log id: 40087d9c >2013-11-13 03:44:55,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GetHardwareInfoVDSCommand, log id: 40087d9c >2013-11-13 03:44:55,807 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host rose07.qa.lab.tlv.redhat.com. >2013-11-13 03:44:55,813 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-45) [f620b1d] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 03:44:55,815 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-45) [432db9fc] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 03:44:55,940 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-45) [432db9fc] START, ConnectStorageServerVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 18d3c419 >2013-11-13 03:44:55,954 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-45) [432db9fc] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 18d3c419 >2013-11-13 03:44:55,954 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-45) [432db9fc] Host rose07.qa.lab.tlv.redhat.com storage connection was succeeded >2013-11-13 03:44:56,069 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-49) START, ConnectStoragePoolVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 1, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 331c3c86 >2013-11-13 03:44:59,217 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-49) FINISH, ConnectStoragePoolVDSCommand, log id: 331c3c86 >2013-11-13 03:44:59,453 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-45) [432db9fc] START, SetMOMPolicyParametersVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6), log id: 675fa116 >2013-11-13 03:44:59,495 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-45) [432db9fc] FINISH, SetMOMPolicyParametersVDSCommand, log id: 675fa116 >2013-11-13 03:44:59,612 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) [432db9fc] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rose07.qa.lab.tlv.redhat.com. >2013-11-13 03:44:59,745 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-45) [3cf860b1] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-13 03:44:59,894 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-45) [43870d9e] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-13 03:47:19,129 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to update VMs/Templates Ovf. >2013-11-13 03:47:19,139 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 03:47:19,140 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 03:47:19,140 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 03:47:19,144 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 03:47:19,144 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 03:47:19,144 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 03:51:05,799 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-65) Start refreshing all users data >2013-11-13 03:51:06,189 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-65) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 03:51:06,220 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-65) Inactive User vdcadmin found in directory server, its status switched to Active >2013-11-13 04:47:19,144 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to update VMs/Templates Ovf. >2013-11-13 04:47:19,145 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 04:47:19,154 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 04:47:19,154 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 04:47:19,157 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 04:47:19,157 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 04:47:19,158 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 04:51:06,222 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-31) Start refreshing all users data >2013-11-13 04:51:06,251 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-31) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 05:47:19,159 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) [51857563] Attempting to update VMs/Templates Ovf. >2013-11-13 05:47:19,160 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) [51857563] Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 05:47:19,169 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) [51857563] Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 05:47:19,169 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) [51857563] Attempting to update template OVFs in Data Center new-nfs >2013-11-13 05:47:19,172 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) [51857563] Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 05:47:19,172 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) [51857563] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 05:47:19,173 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) [51857563] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 05:51:06,291 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-30) Start refreshing all users data >2013-11-13 05:51:06,489 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-30) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 06:47:19,173 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VMs/Templates Ovf. >2013-11-13 06:47:19,174 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 06:47:19,183 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 06:47:19,183 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 06:47:19,186 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 06:47:19,186 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 06:47:19,186 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 06:51:06,563 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-51) Start refreshing all users data >2013-11-13 06:51:07,527 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-51) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 07:47:19,188 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Attempting to update VMs/Templates Ovf. >2013-11-13 07:47:19,189 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 07:47:19,198 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 07:47:19,198 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 07:47:19,201 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 07:47:19,201 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 07:47:19,202 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 07:51:07,652 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-29) Start refreshing all users data >2013-11-13 07:51:07,699 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-29) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 08:47:19,202 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2013-11-13 08:47:19,203 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 08:47:19,204 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 08:47:19,204 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 08:47:19,205 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 08:47:19,205 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 08:47:19,206 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 08:51:07,769 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-65) Start refreshing all users data >2013-11-13 08:51:08,039 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-65) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 09:47:19,206 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) [353ac9e2] Attempting to update VMs/Templates Ovf. >2013-11-13 09:47:19,207 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) [353ac9e2] Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 09:47:19,216 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) [353ac9e2] Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 09:47:19,216 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) [353ac9e2] Attempting to update template OVFs in Data Center new-nfs >2013-11-13 09:47:19,219 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) [353ac9e2] Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 09:47:19,220 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) [353ac9e2] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 09:47:19,221 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) [353ac9e2] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 09:51:08,213 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-41) Start refreshing all users data >2013-11-13 09:51:08,251 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-41) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 10:00:52,241 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginAdminUserCommand internal: false. >2013-11-13 10:00:52,261 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:00:52,357 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [44851fb1] Running command: LogoutUserCommand internal: false. >2013-11-13 10:00:52,369 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [44851fb1] Correlation ID: 44851fb1, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 10:00:52,407 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-11-13 10:00:52,425 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:02:05,602 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:02:05,602 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:02:05,603 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-4) [2ed27ea5] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:02:05,619 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-4) [2ed27ea5] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Installing, nonOperationalReason=NONE), log id: 775fdcba >2013-11-13 10:02:05,622 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-4) [2ed27ea5] FINISH, SetVdsStatusVDSCommand, log id: 775fdcba >2013-11-13 10:02:05,622 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:02:05,622 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:02:05,625 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (ajp-/127.0.0.1:8702-4) [2ed27ea5] Lock Acquired to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:02:05,627 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [2ed27ea5] Running command: InstallVdsCommand internal: true. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:02:05,627 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [2ed27ea5] Before Installation host 427deb7c-ec9c-4e96-b725-ec56cfb8c168, monique05 >2013-11-13 10:02:05,629 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [2ed27ea5] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Installing, nonOperationalReason=NONE), log id: 2696aaac >2013-11-13 10:02:05,633 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [2ed27ea5] FINISH, SetVdsStatusVDSCommand, log id: 2696aaac >2013-11-13 10:02:05,637 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [2ed27ea5] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:02:05,685 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [2ed27ea5] Correlation ID: 2ed27ea5, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 10:02:06,031 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-5-thread-50) [2ed27ea5] Installation monique-vds05.tlv.redhat.com: Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17 >2013-11-13 10:02:06,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-50) [2ed27ea5] Correlation ID: 2ed27ea5, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17. >2013-11-13 10:02:08,351 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-50) [2ed27ea5] Error during host monique-vds05.tlv.redhat.com install: javax.naming.AuthenticationException: SSH authentication to 'root@monique-vds05.tlv.redhat.com' failed. Please verify provided credentials. Make sure host is configured for password authentication > at org.ovirt.engine.core.utils.ssh.SSHClient.authenticate(SSHClient.java:359) [utils.jar:] > at org.ovirt.engine.core.utils.ssh.SSHDialog.authenticate(SSHDialog.java:264) [utils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1024) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:192) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:105) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1131) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1216) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1892) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1236) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:175) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:94) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 10:02:08,351 ERROR [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [2ed27ea5] Host installation failed for host 427deb7c-ec9c-4e96-b725-ec56cfb8c168, monique05.: javax.naming.AuthenticationException: SSH authentication to 'root@monique-vds05.tlv.redhat.com' failed. Please verify provided credentials. Make sure host is configured for password authentication > at org.ovirt.engine.core.utils.ssh.SSHClient.authenticate(SSHClient.java:359) [utils.jar:] > at org.ovirt.engine.core.utils.ssh.SSHDialog.authenticate(SSHDialog.java:264) [utils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1024) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:192) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:105) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1131) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1216) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1892) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1236) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:175) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:94) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 10:02:08,353 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [2ed27ea5] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=InstallFailed, nonOperationalReason=NONE), log id: 49e1f7b9 >2013-11-13 10:02:08,356 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [2ed27ea5] FINISH, SetVdsStatusVDSCommand, log id: 49e1f7b9 >2013-11-13 10:02:08,362 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-50) [2ed27ea5] Correlation ID: 2ed27ea5, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installation failed. SSH authentication to 'root@monique-vds05.tlv.redhat.com' failed. Please verify provided credentials. Make sure host is configured for password authentication. >2013-11-13 10:02:08,362 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [2ed27ea5] Lock freed to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:02:18,758 WARN [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-2) [3ea64e41] CanDoAction of action MaintenanceNumberOfVdss failed. Reasons:VAR__TYPE__HOST,VAR__ACTION__MAINTENANCE,VDS_CANNOT_MAINTENANCE_VDS_IS_NOT_OPERATIONAL >2013-11-13 10:03:15,335 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:03:15,336 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:03:15,336 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-1) [38a59e4e] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:03:15,377 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:03:15,377 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:03:15,382 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [38a59e4e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:03:15,461 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [38a59e4e] Correlation ID: 38a59e4e, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 10:03:26,896 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-8) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:03:26,896 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-8) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:03:26,896 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-8) [77ff46be] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:03:26,923 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-8) [77ff46be] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Installing, nonOperationalReason=NONE), log id: 6cde6c30 >2013-11-13 10:03:26,932 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-8) [77ff46be] FINISH, SetVdsStatusVDSCommand, log id: 6cde6c30 >2013-11-13 10:03:26,932 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-8) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:03:26,932 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-8) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:03:26,933 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (ajp-/127.0.0.1:8702-8) [77ff46be] Lock Acquired to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:03:26,935 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [77ff46be] Running command: InstallVdsCommand internal: true. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:03:26,935 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [77ff46be] Before Installation host 427deb7c-ec9c-4e96-b725-ec56cfb8c168, monique05 >2013-11-13 10:03:26,939 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [77ff46be] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Installing, nonOperationalReason=NONE), log id: 71c6c14a >2013-11-13 10:03:26,968 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [77ff46be] FINISH, SetVdsStatusVDSCommand, log id: 71c6c14a >2013-11-13 10:03:26,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [77ff46be] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:03:26,992 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [77ff46be] Correlation ID: 77ff46be, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 10:03:27,077 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-49) [77ff46be] Error during host monique-vds05.tlv.redhat.com install: java.security.GeneralSecurityException: Invalid fingerprint c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17, expected ewgweaagewgagweaegwg > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:78) [bll.jar:] > at org.ovirt.engine.core.utils.ssh.SSHDialog.connect(SSHDialog.java:243) [utils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1015) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:192) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:105) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1131) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1216) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1892) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1236) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:175) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:94) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 10:03:27,078 ERROR [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [77ff46be] Host installation failed for host 427deb7c-ec9c-4e96-b725-ec56cfb8c168, monique05.: java.security.GeneralSecurityException: Invalid fingerprint c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17, expected ewgweaagewgagweaegwg > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:78) [bll.jar:] > at org.ovirt.engine.core.utils.ssh.SSHDialog.connect(SSHDialog.java:243) [utils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1015) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:192) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:105) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1131) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1216) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1892) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1236) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:175) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:94) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 10:03:27,080 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [77ff46be] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=InstallFailed, nonOperationalReason=NONE), log id: 76a9b539 >2013-11-13 10:03:27,083 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [77ff46be] FINISH, SetVdsStatusVDSCommand, log id: 76a9b539 >2013-11-13 10:03:27,092 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [77ff46be] Correlation ID: 77ff46be, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installation failed. Invalid fingerprint c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17, expected ewgweaagewgagweaegwg. >2013-11-13 10:03:27,092 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [77ff46be] Lock freed to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:07:13,103 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-5) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:07:13,103 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-5) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:07:13,103 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-5) [7d987b11] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:07:13,141 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-5) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:07:13,141 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-5) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:07:13,157 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [7d987b11] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:07:13,226 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [7d987b11] Correlation ID: 7d987b11, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 10:07:17,746 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:07:17,746 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:07:17,746 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-1) [6c866d71] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:07:17,755 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-1) [6c866d71] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Installing, nonOperationalReason=NONE), log id: 293f760e >2013-11-13 10:07:17,758 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-1) [6c866d71] FINISH, SetVdsStatusVDSCommand, log id: 293f760e >2013-11-13 10:07:17,758 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:07:17,759 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:07:17,760 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (ajp-/127.0.0.1:8702-1) [6c866d71] Lock Acquired to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:07:17,772 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [6c866d71] Running command: InstallVdsCommand internal: true. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:07:17,772 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [6c866d71] Before Installation host 427deb7c-ec9c-4e96-b725-ec56cfb8c168, monique05 >2013-11-13 10:07:17,788 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [6c866d71] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:07:17,831 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [6c866d71] Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 10:07:17,903 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [6c866d71] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Installing, nonOperationalReason=NONE), log id: 7d22641b >2013-11-13 10:07:17,906 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [6c866d71] FINISH, SetVdsStatusVDSCommand, log id: 7d22641b >2013-11-13 10:07:17,958 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-5-thread-49) [6c866d71] Installation monique-vds05.tlv.redhat.com: Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17 >2013-11-13 10:07:17,973 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [6c866d71] Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17. >2013-11-13 10:07:18,840 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-49) [6c866d71] Installation of monique-vds05.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-11-13 10:07:18,913 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-5-thread-49) SSH execute root@monique-vds05.tlv.redhat.com 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-11-13 10:07:19,665 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Initializing >2013-11-13 10:07:19,672 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Initializing. >2013-11-13 10:07:19,682 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment setup >2013-11-13 10:07:19,688 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment setup. >2013-11-13 10:07:19,688 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment packages setup >2013-11-13 10:07:19,693 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment packages setup. >2013-11-13 10:07:19,693 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Programs detection >2013-11-13 10:07:19,698 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Programs detection. >2013-11-13 10:07:19,698 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment customization >2013-11-13 10:07:19,705 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment customization. >2013-11-13 10:07:19,717 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20131113100724.log' >2013-11-13 10:07:19,723 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Logs at host located at: '/tmp/ovirt-host-deploy-20131113100724.log'. >2013-11-13 10:07:19,884 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host monique-vds05.tlv.redhat.com reports unique id 4C4C4544-0032-4310-8051-B7C04F39354A >2013-11-13 10:07:19,911 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0032-4310-8051-B7C04F39354A to Host monique-vds05.tlv.redhat.com >2013-11-13 10:07:19,966 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Setup validation >2013-11-13 10:07:19,970 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Setup validation. >2013-11-13 10:07:20,006 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Hardware supports virtualization >2013-11-13 10:07:20,011 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Hardware supports virtualization. >2013-11-13 10:07:20,011 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction setup >2013-11-13 10:07:20,015 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction setup. >2013-11-13 10:07:20,015 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 10:07:20,020 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 10:07:20,020 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Package installation >2013-11-13 10:07:20,025 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Package installation. >2013-11-13 10:07:22,643 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 10:07:22,648 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 10:07:23,400 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Enrolling certificate >2013-11-13 10:07:23,422 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Enrolling certificate. >2013-11-13 10:07:24,701 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction commit >2013-11-13 10:07:24,706 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction commit. >2013-11-13 10:07:24,706 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Closing up >2013-11-13 10:07:24,711 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Closing up. >2013-11-13 10:07:25,391 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Starting vdsm >2013-11-13 10:07:25,398 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Starting vdsm. >2013-11-13 10:07:27,167 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Pre-termination >2013-11-13 10:07:27,172 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Pre-termination. >2013-11-13 10:07:27,180 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113100727-monique-vds05.tlv.redhat.com-6c866d71.log' >2013-11-13 10:07:27,184 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113100727-monique-vds05.tlv.redhat.com-6c866d71.log'. >2013-11-13 10:07:27,371 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Termination >2013-11-13 10:07:27,377 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Termination. >2013-11-13 10:07:27,515 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [6c866d71] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 10:07:27,515 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [6c866d71] Command PollVDS execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 10:07:28,041 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-5-thread-49) [6c866d71] Engine managed to communicate with VDSM agent on host monique05 >2013-11-13 10:07:28,353 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [6c866d71] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Initializing, nonOperationalReason=NONE), log id: 123c79e0 >2013-11-13 10:07:28,365 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [6c866d71] FINISH, SetVdsStatusVDSCommand, log id: 123c79e0 >2013-11-13 10:07:28,365 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [6c866d71] After Installation host monique05, VDS >2013-11-13 10:07:28,371 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [6c866d71] Correlation ID: 6c866d71, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installed >2013-11-13 10:07:28,371 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [6c866d71] Lock freed to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:07:29,783 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, vds=Host[monique05]), log id: 71cb1f97 >2013-11-13 10:07:29,842 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GetHardwareInfoVDSCommand, log id: 71cb1f97 >2013-11-13 10:07:29,855 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-69) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 10:07:29,861 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-69) [736264c] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 10:07:29,863 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-69) [26c15215] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 10:07:29,873 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-69) [26c15215] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 8e0a65e >2013-11-13 10:07:29,981 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-69) [26c15215] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 8e0a65e >2013-11-13 10:07:29,981 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-69) [26c15215] Host monique05 storage connection was succeeded >2013-11-13 10:07:29,986 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-50) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 2c9bdafb >2013-11-13 10:07:33,703 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-50) FINISH, ConnectStoragePoolVDSCommand, log id: 2c9bdafb >2013-11-13 10:07:33,716 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-69) [26c15215] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168), log id: 45d3dea3 >2013-11-13 10:07:33,763 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-69) [26c15215] FINISH, SetMOMPolicyParametersVDSCommand, log id: 45d3dea3 >2013-11-13 10:07:33,771 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-69) [26c15215] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:07:33,779 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-69) [1487204] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:07:33,782 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-69) [28563c48] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:07:41,752 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (pool-5-thread-50) [4cc8d600] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:07:41,755 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [4cc8d600] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 5263c790 >2013-11-13 10:07:41,760 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [4cc8d600] FINISH, SetVdsStatusVDSCommand, log id: 5263c790 >2013-11-13 10:07:41,850 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (pool-5-thread-50) [4cc8d600] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:07:41,875 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-50) [4cc8d600] Correlation ID: 4cc8d600, Job ID: 528c4bf1-c335-4939-b9e1-0ecd1c39314b, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was switched to Maintenance mode by admin@internal. >2013-11-13 10:07:42,822 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 427deb7c-ec9c-4e96-b725-ec56cfb8c168 : monique05 >2013-11-13 10:07:42,829 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) Clearing cache of pool: a44a18ab-6f2e-48d4-ab60-889845f498d3 for problematic entities of VDS: monique05. >2013-11-13 10:07:42,831 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-2) START, DisconnectStoragePoolVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2), log id: 473b09fa >2013-11-13 10:07:44,881 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, DisconnectStoragePoolVDSCommand, log id: 473b09fa >2013-11-13 10:07:44,881 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-2) [15da280] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 10:07:44,895 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-2) [15da280] START, DisconnectStorageServerVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 43b48ecc >2013-11-13 10:07:47,015 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-2) [15da280] FINISH, DisconnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 43b48ecc >2013-11-13 10:09:47,656 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-3) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:09:47,656 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-3) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:09:47,657 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-3) [a0cbb4f] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:09:47,695 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-3) [a0cbb4f] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Installing, nonOperationalReason=NONE), log id: 483f4d6b >2013-11-13 10:09:47,708 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-3) [a0cbb4f] FINISH, SetVdsStatusVDSCommand, log id: 483f4d6b >2013-11-13 10:09:47,709 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-3) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:09:47,709 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-3) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:09:47,710 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (ajp-/127.0.0.1:8702-3) [a0cbb4f] Lock Acquired to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:09:47,711 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [a0cbb4f] Running command: InstallVdsCommand internal: true. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:09:47,711 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [a0cbb4f] Before Installation host 427deb7c-ec9c-4e96-b725-ec56cfb8c168, monique05 >2013-11-13 10:09:47,727 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [a0cbb4f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:09:47,794 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [a0cbb4f] Correlation ID: a0cbb4f, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 10:09:47,865 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [a0cbb4f] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Installing, nonOperationalReason=NONE), log id: 69b0e092 >2013-11-13 10:09:47,868 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [a0cbb4f] FINISH, SetVdsStatusVDSCommand, log id: 69b0e092 >2013-11-13 10:09:47,925 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-5-thread-50) [a0cbb4f] Installation monique-vds05.tlv.redhat.com: Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17 >2013-11-13 10:09:47,931 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-50) [a0cbb4f] Correlation ID: a0cbb4f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17. >2013-11-13 10:09:47,942 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-50) [a0cbb4f] Error during host monique-vds05.tlv.redhat.com install: javax.naming.AuthenticationException: SSH authentication to 'root@monique-vds05.tlv.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host > at org.ovirt.engine.core.utils.ssh.SSHClient.authenticate(SSHClient.java:359) [utils.jar:] > at org.ovirt.engine.core.utils.ssh.SSHDialog.authenticate(SSHDialog.java:264) [utils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1024) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:192) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:105) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1131) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1216) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1892) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1236) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:175) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:94) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 10:09:47,943 ERROR [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [a0cbb4f] Host installation failed for host 427deb7c-ec9c-4e96-b725-ec56cfb8c168, monique05.: javax.naming.AuthenticationException: SSH authentication to 'root@monique-vds05.tlv.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host > at org.ovirt.engine.core.utils.ssh.SSHClient.authenticate(SSHClient.java:359) [utils.jar:] > at org.ovirt.engine.core.utils.ssh.SSHDialog.authenticate(SSHDialog.java:264) [utils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1024) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:192) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:105) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1131) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1216) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1892) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1236) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:175) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:94) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 10:09:47,945 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [a0cbb4f] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=InstallFailed, nonOperationalReason=NONE), log id: 3a6eb470 >2013-11-13 10:09:47,948 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [a0cbb4f] FINISH, SetVdsStatusVDSCommand, log id: 3a6eb470 >2013-11-13 10:09:47,953 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-50) [a0cbb4f] Correlation ID: a0cbb4f, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installation failed. SSH authentication to 'root@monique-vds05.tlv.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host. >2013-11-13 10:09:47,953 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [a0cbb4f] Lock freed to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:11:32,714 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:11:32,714 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:11:32,714 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-13) [4477d95f] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:11:32,763 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:11:32,763 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:11:32,768 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) [4477d95f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:11:32,815 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) [4477d95f] Correlation ID: 4477d95f, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 10:11:36,801 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:11:36,801 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:11:36,801 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-4) [7d27aeea] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:11:36,904 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-4) [7d27aeea] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Installing, nonOperationalReason=NONE), log id: 7f269d08 >2013-11-13 10:11:36,907 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-4) [7d27aeea] FINISH, SetVdsStatusVDSCommand, log id: 7f269d08 >2013-11-13 10:11:36,907 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:11:36,907 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 10:11:36,908 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (ajp-/127.0.0.1:8702-4) [7d27aeea] Lock Acquired to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:11:36,909 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [7d27aeea] Running command: InstallVdsCommand internal: true. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:11:36,909 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [7d27aeea] Before Installation host 427deb7c-ec9c-4e96-b725-ec56cfb8c168, monique05 >2013-11-13 10:11:36,925 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [7d27aeea] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:11:36,963 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [7d27aeea] Correlation ID: 7d27aeea, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 10:11:37,038 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [7d27aeea] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=Installing, nonOperationalReason=NONE), log id: 58ab2068 >2013-11-13 10:11:37,040 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [7d27aeea] FINISH, SetVdsStatusVDSCommand, log id: 58ab2068 >2013-11-13 10:11:37,095 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-50) [7d27aeea] Error during host monique-vds05.tlv.redhat.com install: java.security.GeneralSecurityException: Invalid fingerprint c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17, expected safasfsfasf32r23r23rf23 > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:78) [bll.jar:] > at org.ovirt.engine.core.utils.ssh.SSHDialog.connect(SSHDialog.java:243) [utils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1015) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:192) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:105) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1131) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1216) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1892) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1236) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:175) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:94) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 10:11:37,096 ERROR [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [7d27aeea] Host installation failed for host 427deb7c-ec9c-4e96-b725-ec56cfb8c168, monique05.: java.security.GeneralSecurityException: Invalid fingerprint c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17, expected safasfsfasf32r23r23rf23 > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:78) [bll.jar:] > at org.ovirt.engine.core.utils.ssh.SSHDialog.connect(SSHDialog.java:243) [utils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1015) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:192) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:105) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1131) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1216) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1892) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1236) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:175) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:94) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 10:11:37,098 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [7d27aeea] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168, status=InstallFailed, nonOperationalReason=NONE), log id: 13ba14d8 >2013-11-13 10:11:37,101 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-50) [7d27aeea] FINISH, SetVdsStatusVDSCommand, log id: 13ba14d8 >2013-11-13 10:11:37,109 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-50) [7d27aeea] Correlation ID: 7d27aeea, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installation failed. Invalid fingerprint c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17, expected safasfsfasf32r23r23rf23. >2013-11-13 10:11:37,110 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-50) [7d27aeea] Lock freed to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:32:03,351 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-13) [21c0c2fe] Lock Acquired to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:32:03,355 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (pool-5-thread-50) [21c0c2fe] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 Type: VDS >2013-11-13 10:32:03,663 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (pool-5-thread-50) [21c0c2fe] START, RemoveVdsVDSCommand( HostId = 427deb7c-ec9c-4e96-b725-ec56cfb8c168), log id: 3dfda3b8 >2013-11-13 10:32:03,663 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (pool-5-thread-50) [21c0c2fe] vdsManager::disposing >2013-11-13 10:32:03,663 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (pool-5-thread-50) [21c0c2fe] FINISH, RemoveVdsVDSCommand, log id: 3dfda3b8 >2013-11-13 10:32:03,667 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-50) [21c0c2fe] Correlation ID: 21c0c2fe, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was removed by admin@internal. >2013-11-13 10:32:03,667 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (pool-5-thread-50) [21c0c2fe] Lock freed to object EngineLock [exclusiveLocks= key: 427deb7c-ec9c-4e96-b725-ec56cfb8c168 value: VDS >, sharedLocks= ] >2013-11-13 10:33:55,972 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-11-13 10:33:55,983 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:34:21,745 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-11-13 10:34:21,756 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:34:21,762 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [21e446bb] Running command: LogoutUserCommand internal: false. >2013-11-13 10:34:21,797 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [21e446bb] Correlation ID: 21e446bb, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 10:34:45,636 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-11-13 10:34:45,659 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:34:45,737 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [37fc5f4f] Running command: LogoutUserCommand internal: false. >2013-11-13 10:34:45,750 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [37fc5f4f] Correlation ID: 37fc5f4f, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 10:35:02,861 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-11-13 10:35:02,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:35:03,143 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [4049bbc8] Running command: LogoutUserCommand internal: false. >2013-11-13 10:35:03,149 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [4049bbc8] Correlation ID: 4049bbc8, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 10:37:58,964 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2013-11-13 10:37:58,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:39:50,662 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-11-13 10:39:50,676 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:39:56,992 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-11-13 10:39:57,010 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:40:21,091 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2013-11-13 10:40:21,099 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:40:21,108 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-8) Operation Failed: Entity not found: Cluster: name=new-nfs >2013-11-13 10:40:21,117 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [23812147] Running command: LogoutUserCommand internal: false. >2013-11-13 10:40:21,121 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [23812147] Correlation ID: 23812147, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 10:40:47,107 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-11-13 10:40:47,127 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 10:40:48,660 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-2) [23e213f1] Running command: AddVdsCommand internal: false. Entities affected : ID: b273ff0a-4519-4d34-a9a9-a8f1a3506fca Type: VdsGroups >2013-11-13 10:40:48,692 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-2) [617d60e4] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 26b31124-7591-467a-955d-2083c2a2322f Type: VDS >2013-11-13 10:40:48,699 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-2) [617d60e4] Lock freed to object EngineLock [exclusiveLocks= key: a44a18ab-6f2e-48d4-ab60-889845f498d3 value: REGISTER_VDS >, sharedLocks= ] >2013-11-13 10:40:48,699 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-2) [617d60e4] Cannot get vdsManager for vdsid=26b31124-7591-467a-955d-2083c2a2322f >2013-11-13 10:40:48,701 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [617d60e4] START, RemoveVdsVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f), log id: 6fcec8da >2013-11-13 10:40:48,701 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-2) [617d60e4] Cannot get vdsManager for vdsid=26b31124-7591-467a-955d-2083c2a2322f >2013-11-13 10:40:48,701 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [617d60e4] FINISH, RemoveVdsVDSCommand, log id: 6fcec8da >2013-11-13 10:40:48,701 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-2) [617d60e4] Cannot get vdsManager for vdsid=26b31124-7591-467a-955d-2083c2a2322f >2013-11-13 10:40:48,702 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [617d60e4] START, AddVdsVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f), log id: 22c103cf >2013-11-13 10:40:48,702 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [617d60e4] AddVds - entered , starting logic to add VDS 26b31124-7591-467a-955d-2083c2a2322f >2013-11-13 10:40:48,703 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [617d60e4] AddVds - VDS 26b31124-7591-467a-955d-2083c2a2322f was added, will try to add it to the resource manager >2013-11-13 10:40:48,703 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-2) [617d60e4] Entered VdsManager constructor >2013-11-13 10:40:48,816 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-2) [617d60e4] Initialize vdsBroker (monique-vds05.tlv.redhat.com,54,321) >2013-11-13 10:40:48,816 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-2) [617d60e4] VDS 26b31124-7591-467a-955d-2083c2a2322f was added to the Resource Manager >2013-11-13 10:40:48,816 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [617d60e4] FINISH, AddVdsVDSCommand, log id: 22c103cf >2013-11-13 10:40:48,821 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [617d60e4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:40:48,853 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [279b8a9b] Lock Acquired to object EngineLock [exclusiveLocks= key: 26b31124-7591-467a-955d-2083c2a2322f value: VDS >, sharedLocks= ] >2013-11-13 10:40:48,855 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [617d60e4] Correlation ID: 23e213f1, Job ID: f0de71e9-0c97-4152-950f-3d13e62c2ac8, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was added by admin@internal. >2013-11-13 10:40:48,863 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [3cded0d8] Running command: LogoutUserCommand internal: false. >2013-11-13 10:40:48,874 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [279b8a9b] Running command: InstallVdsCommand internal: true. Entities affected : ID: 26b31124-7591-467a-955d-2083c2a2322f Type: VDS >2013-11-13 10:40:48,874 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [279b8a9b] Before Installation host 26b31124-7591-467a-955d-2083c2a2322f, monique05 >2013-11-13 10:40:48,876 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [279b8a9b] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f, status=Installing, nonOperationalReason=NONE), log id: 672beff2 >2013-11-13 10:40:48,877 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [3cded0d8] Correlation ID: 3cded0d8, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 10:40:48,908 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [279b8a9b] FINISH, SetVdsStatusVDSCommand, log id: 672beff2 >2013-11-13 10:40:49,031 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-5-thread-49) [279b8a9b] Installation monique-vds05.tlv.redhat.com: Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17 >2013-11-13 10:40:49,036 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [279b8a9b] Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17. >2013-11-13 10:40:49,055 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-49) [279b8a9b] Installation of monique-vds05.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-11-13 10:40:49,063 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-5-thread-49) SSH execute root@monique-vds05.tlv.redhat.com 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-11-13 10:40:49,742 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Initializing >2013-11-13 10:40:49,747 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Initializing. >2013-11-13 10:40:49,756 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment setup >2013-11-13 10:40:49,760 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment setup. >2013-11-13 10:40:49,761 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment packages setup >2013-11-13 10:40:49,765 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment packages setup. >2013-11-13 10:40:49,765 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Programs detection >2013-11-13 10:40:49,769 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Programs detection. >2013-11-13 10:40:49,769 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment customization >2013-11-13 10:40:49,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment customization. >2013-11-13 10:40:49,794 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20131113104054.log' >2013-11-13 10:40:49,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Logs at host located at: '/tmp/ovirt-host-deploy-20131113104054.log'. >2013-11-13 10:40:49,929 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host monique-vds05.tlv.redhat.com reports unique id 4C4C4544-0032-4310-8051-B7C04F39354A >2013-11-13 10:40:49,955 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0032-4310-8051-B7C04F39354A to Host monique-vds05.tlv.redhat.com >2013-11-13 10:40:50,025 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Setup validation >2013-11-13 10:40:50,036 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Setup validation. >2013-11-13 10:40:50,065 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Hardware supports virtualization >2013-11-13 10:40:50,071 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Hardware supports virtualization. >2013-11-13 10:40:50,071 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction setup >2013-11-13 10:40:50,107 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction setup. >2013-11-13 10:40:50,107 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 10:40:50,126 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 10:40:50,126 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Package installation >2013-11-13 10:40:50,130 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Package installation. >2013-11-13 10:40:51,818 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Initializing Host: monique05 >2013-11-13 10:40:52,698 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 10:40:52,702 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 10:40:53,635 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Enrolling certificate >2013-11-13 10:40:53,640 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Enrolling certificate. >2013-11-13 10:40:54,894 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction commit >2013-11-13 10:40:54,900 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction commit. >2013-11-13 10:40:54,900 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Closing up >2013-11-13 10:40:54,905 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Closing up. >2013-11-13 10:40:55,257 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Starting vdsm >2013-11-13 10:40:55,265 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Starting vdsm. >2013-11-13 10:40:56,954 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Pre-termination >2013-11-13 10:40:56,958 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Pre-termination. >2013-11-13 10:40:56,967 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113104056-monique-vds05.tlv.redhat.com-279b8a9b.log' >2013-11-13 10:40:56,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113104056-monique-vds05.tlv.redhat.com-279b8a9b.log'. >2013-11-13 10:40:57,140 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Termination >2013-11-13 10:40:57,144 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 279b8a9b, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Termination. >2013-11-13 10:40:57,276 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [279b8a9b] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 10:40:57,276 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [279b8a9b] Command PollVDS execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 10:40:57,799 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [279b8a9b] org.ovirt.engine.core.vdsbroker.vdsbroker.VDSRecoveringException: Recovering from crash or Initializing >2013-11-13 10:40:57,800 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [279b8a9b] Command PollVDS execution failed. Exception: VDSRecoveringException: Recovering from crash or Initializing >2013-11-13 10:40:58,352 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-5-thread-49) [279b8a9b] Engine managed to communicate with VDSM agent on host monique05 >2013-11-13 10:40:58,478 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [279b8a9b] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f, status=Initializing, nonOperationalReason=NONE), log id: 2554ba37 >2013-11-13 10:40:58,480 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [279b8a9b] FINISH, SetVdsStatusVDSCommand, log id: 2554ba37 >2013-11-13 10:40:58,480 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [279b8a9b] After Installation host monique05, VDS >2013-11-13 10:40:58,508 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [279b8a9b] Correlation ID: 279b8a9b, Job ID: f0de71e9-0c97-4152-950f-3d13e62c2ac8, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installed >2013-11-13 10:40:58,512 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [279b8a9b] Lock freed to object EngineLock [exclusiveLocks= key: 26b31124-7591-467a-955d-2083c2a2322f value: VDS >, sharedLocks= ] >2013-11-13 10:41:00,866 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f, vds=Host[monique05]), log id: 15daae10 >2013-11-13 10:41:00,898 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) FINISH, GetHardwareInfoVDSCommand, log id: 15daae10 >2013-11-13 10:41:00,954 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f, vds=Host[monique05]), log id: 4e8c883d >2013-11-13 10:41:00,973 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) FINISH, GetHardwareInfoVDSCommand, log id: 4e8c883d >2013-11-13 10:41:00,994 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 10:41:01,000 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-30) [429aab24] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 10:41:01,001 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-30) [304482c4] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 10:41:01,011 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-30) [304482c4] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 62705c0b >2013-11-13 10:41:01,127 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-30) [304482c4] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 62705c0b >2013-11-13 10:41:01,128 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-30) [304482c4] Host monique05 storage connection was succeeded >2013-11-13 10:41:01,132 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-50) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 9f9e57e >2013-11-13 10:41:03,450 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-50) FINISH, ConnectStoragePoolVDSCommand, log id: 9f9e57e >2013-11-13 10:41:03,462 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-30) [304482c4] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f), log id: 6b1e2fc6 >2013-11-13 10:41:03,510 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-30) [304482c4] FINISH, SetMOMPolicyParametersVDSCommand, log id: 6b1e2fc6 >2013-11-13 10:41:03,514 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) [304482c4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 10:41:03,518 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-30) [5b8a268b] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 26b31124-7591-467a-955d-2083c2a2322f Type: VDS >2013-11-13 10:41:03,521 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-30) [64f712a3] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 26b31124-7591-467a-955d-2083c2a2322f Type: VDS >2013-11-13 10:47:19,222 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2013-11-13 10:47:19,222 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 10:47:19,223 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 10:47:19,223 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 10:47:19,224 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 10:47:19,224 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 10:47:19,225 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 10:51:08,350 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-38) Start refreshing all users data >2013-11-13 10:51:08,368 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-38) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 11:02:09,910 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-11-13 11:02:09,920 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:03:08,018 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-11-13 11:03:08,028 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:03:59,913 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-13) Can't read file "/usr/share/doc/rhevm-doc/manual/menu.css" for request "/docs/menu.css", will send a 404 error response. >2013-11-13 11:03:59,924 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-2) Can't read file "/usr/share/doc/rhevm-doc/manual/print.css" for request "/docs/print.css", will send a 404 error response. >2013-11-13 11:04:42,415 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-6) Can't read file "/usr/share/doc/rhevm-doc/manual/print.css" for request "/docs/print.css", will send a 404 error response. >2013-11-13 11:04:42,425 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-9) Can't read file "/usr/share/doc/rhevm-doc/manual/menu.css" for request "/docs/menu.css", will send a 404 error response. >2013-11-13 11:04:53,735 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-6) Can't read file "/usr/share/doc/rhevm-doc/manual/print.css" for request "/docs/print.css", will send a 404 error response. >2013-11-13 11:04:53,737 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-5) Can't read file "/usr/share/doc/rhevm-doc/manual/menu.css" for request "/docs/menu.css", will send a 404 error response. >2013-11-13 11:05:05,610 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-3) Can't read file "/usr/share/doc/rhevm-doc/manual/print.css" for request "/docs/print.css", will send a 404 error response. >2013-11-13 11:05:05,613 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-9) Can't read file "/usr/share/doc/rhevm-doc/manual/menu.css" for request "/docs/menu.css", will send a 404 error response. >2013-11-13 11:05:59,352 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-13) Running command: LoginUserCommand internal: false. >2013-11-13 11:05:59,397 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:07:55,806 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-11-13 11:07:55,823 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:07:56,178 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [5a7cfbbd] Running command: LogoutUserCommand internal: false. >2013-11-13 11:07:56,182 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [5a7cfbbd] Correlation ID: 5a7cfbbd, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 11:08:43,532 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-11-13 11:08:43,554 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:08:43,661 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [6ed10061] Running command: LogoutUserCommand internal: false. >2013-11-13 11:08:43,664 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [6ed10061] Correlation ID: 6ed10061, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 11:08:55,180 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-11-13 11:08:55,198 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:08:56,868 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [552d5340] Running command: LogoutUserCommand internal: false. >2013-11-13 11:08:56,891 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [552d5340] Correlation ID: 552d5340, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 11:10:34,547 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-11-13 11:10:34,569 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:10:34,581 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-11) [64d95948] Lock Acquired to object EngineLock [exclusiveLocks= key: 26b31124-7591-467a-955d-2083c2a2322f value: VDS >, sharedLocks= ] >2013-11-13 11:10:34,586 WARN [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-11) [64d95948] CanDoAction of action RemoveVds failed. Reasons:VAR__ACTION__REMOVE,VAR__TYPE__HOST,VDS_CANNOT_REMOVE_VDS_STATUS_ILLEGAL >2013-11-13 11:10:34,587 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-11) [64d95948] Lock freed to object EngineLock [exclusiveLocks= key: 26b31124-7591-467a-955d-2083c2a2322f value: VDS >, sharedLocks= ] >2013-11-13 11:10:34,588 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-11) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2013-11-13 11:10:34,589 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-11) Operation Failed: [Cannot remove Host. Host is operational. Please switch Host to Maintenance mode first.] >2013-11-13 11:10:34,609 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [283e33f] Running command: LogoutUserCommand internal: false. >2013-11-13 11:10:34,613 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [283e33f] Correlation ID: 283e33f, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 11:11:18,960 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-11-13 11:11:18,993 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:11:18,999 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [737b107d] Running command: LogoutUserCommand internal: false. >2013-11-13 11:11:19,004 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [737b107d] Correlation ID: 737b107d, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 11:11:58,702 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-11-13 11:11:58,728 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:11:59,005 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [4676c5aa] Running command: LogoutUserCommand internal: false. >2013-11-13 11:11:59,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [4676c5aa] Correlation ID: 4676c5aa, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 11:12:25,068 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-11-13 11:12:25,089 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:12:25,101 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [53f5f450] Running command: LogoutUserCommand internal: false. >2013-11-13 11:12:25,126 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [53f5f450] Correlation ID: 53f5f450, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 11:15:00,029 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2013-11-13 11:15:00,039 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:15:00,042 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [323d4bbb] Running command: LogoutUserCommand internal: false. >2013-11-13 11:15:00,047 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [323d4bbb] Correlation ID: 323d4bbb, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 11:17:18,311 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [2fa91f7] Running command: LogoutUserCommand internal: false. >2013-11-13 11:17:18,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [2fa91f7] Correlation ID: 2fa91f7, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 11:18:22,675 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginAdminUserCommand internal: false. >2013-11-13 11:18:22,683 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:18:22,783 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2013-11-13 11:18:22,796 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:18:38,922 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (pool-5-thread-48) [4cf6fa8a] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 26b31124-7591-467a-955d-2083c2a2322f Type: VDS >2013-11-13 11:18:38,952 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [4cf6fa8a] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 6e962e >2013-11-13 11:18:38,987 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [4cf6fa8a] FINISH, SetVdsStatusVDSCommand, log id: 6e962e >2013-11-13 11:18:39,170 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (pool-5-thread-48) [4cf6fa8a] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 26b31124-7591-467a-955d-2083c2a2322f Type: VDS >2013-11-13 11:18:39,176 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [4cf6fa8a] Correlation ID: 4cf6fa8a, Job ID: 9f339843-cd21-4197-ae27-c5fb7a5591ee, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was switched to Maintenance mode by admin@internal. >2013-11-13 11:18:41,020 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [2e0a2893] Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 26b31124-7591-467a-955d-2083c2a2322f : monique05 >2013-11-13 11:18:41,027 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) Clearing cache of pool: a44a18ab-6f2e-48d4-ab60-889845f498d3 for problematic entities of VDS: monique05. >2013-11-13 11:18:41,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-42) [2e0a2893] START, DisconnectStoragePoolVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2), log id: 218057a1 >2013-11-13 11:18:44,040 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-42) [2e0a2893] FINISH, DisconnectStoragePoolVDSCommand, log id: 218057a1 >2013-11-13 11:18:44,041 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-42) [1c73cb07] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 11:18:44,057 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-42) [1c73cb07] START, DisconnectStorageServerVDSCommand(HostName = monique05, HostId = 26b31124-7591-467a-955d-2083c2a2322f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5ecad9b2 >2013-11-13 11:18:46,208 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-42) [1c73cb07] FINISH, DisconnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 5ecad9b2 >2013-11-13 11:23:49,432 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-11-13 11:23:49,444 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:23:54,827 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-11-13 11:23:54,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:23:54,871 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-3) [6c8ab38f] Lock Acquired to object EngineLock [exclusiveLocks= key: 26b31124-7591-467a-955d-2083c2a2322f value: VDS >, sharedLocks= ] >2013-11-13 11:23:54,874 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-3) [6c8ab38f] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 26b31124-7591-467a-955d-2083c2a2322f Type: VDS >2013-11-13 11:23:55,163 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [6c8ab38f] START, RemoveVdsVDSCommand( HostId = 26b31124-7591-467a-955d-2083c2a2322f), log id: 76fa22c9 >2013-11-13 11:23:55,163 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-3) [6c8ab38f] vdsManager::disposing >2013-11-13 11:23:55,163 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [6c8ab38f] FINISH, RemoveVdsVDSCommand, log id: 76fa22c9 >2013-11-13 11:23:55,166 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [6c8ab38f] Correlation ID: 6c8ab38f, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was removed by admin@internal. >2013-11-13 11:23:55,166 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-3) [6c8ab38f] Lock freed to object EngineLock [exclusiveLocks= key: 26b31124-7591-467a-955d-2083c2a2322f value: VDS >, sharedLocks= ] >2013-11-13 11:23:55,169 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [58c09b48] Running command: LogoutUserCommand internal: false. >2013-11-13 11:23:55,172 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [58c09b48] Correlation ID: 58c09b48, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 11:46:39,469 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-11-13 11:46:39,476 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:47:19,225 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Attempting to update VMs/Templates Ovf. >2013-11-13 11:47:19,226 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 11:47:19,227 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 11:47:19,227 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 11:47:19,227 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 11:47:19,227 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 11:47:19,228 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 11:48:07,382 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-11-13 11:48:07,390 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:50:07,380 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-11-13 11:50:07,388 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 11:51:08,396 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-10) Start refreshing all users data >2013-11-13 11:51:08,425 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-10) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 11:53:37,860 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-13) Operation Failed: Entity not found: Cluster: name=clus-nsf >2013-11-13 11:54:09,334 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-12) [2d42ad67] Running command: AddVdsCommand internal: false. Entities affected : ID: b273ff0a-4519-4d34-a9a9-a8f1a3506fca Type: VdsGroups >2013-11-13 11:54:09,359 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-12) [2ffa1872] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 24a09956-fd5d-49e2-b03a-270b1468dd1f Type: VDS >2013-11-13 11:54:09,365 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-12) [2ffa1872] Lock freed to object EngineLock [exclusiveLocks= key: a44a18ab-6f2e-48d4-ab60-889845f498d3 value: REGISTER_VDS >, sharedLocks= ] >2013-11-13 11:54:09,365 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-12) [2ffa1872] Cannot get vdsManager for vdsid=24a09956-fd5d-49e2-b03a-270b1468dd1f >2013-11-13 11:54:09,367 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [2ffa1872] START, RemoveVdsVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f), log id: c5119ad >2013-11-13 11:54:09,367 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-12) [2ffa1872] Cannot get vdsManager for vdsid=24a09956-fd5d-49e2-b03a-270b1468dd1f >2013-11-13 11:54:09,367 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [2ffa1872] FINISH, RemoveVdsVDSCommand, log id: c5119ad >2013-11-13 11:54:09,367 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-12) [2ffa1872] Cannot get vdsManager for vdsid=24a09956-fd5d-49e2-b03a-270b1468dd1f >2013-11-13 11:54:09,368 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [2ffa1872] START, AddVdsVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f), log id: 31797dde >2013-11-13 11:54:09,368 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [2ffa1872] AddVds - entered , starting logic to add VDS 24a09956-fd5d-49e2-b03a-270b1468dd1f >2013-11-13 11:54:09,369 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [2ffa1872] AddVds - VDS 24a09956-fd5d-49e2-b03a-270b1468dd1f was added, will try to add it to the resource manager >2013-11-13 11:54:09,369 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-12) [2ffa1872] Entered VdsManager constructor >2013-11-13 11:54:09,499 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-12) [2ffa1872] Initialize vdsBroker (monique-vds05.tlv.redhat.com,54,321) >2013-11-13 11:54:09,500 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-12) [2ffa1872] VDS 24a09956-fd5d-49e2-b03a-270b1468dd1f was added to the Resource Manager >2013-11-13 11:54:09,500 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [2ffa1872] FINISH, AddVdsVDSCommand, log id: 31797dde >2013-11-13 11:54:09,504 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [2ffa1872] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 11:54:09,530 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [65f8b538] Lock Acquired to object EngineLock [exclusiveLocks= key: 24a09956-fd5d-49e2-b03a-270b1468dd1f value: VDS >, sharedLocks= ] >2013-11-13 11:54:09,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [2ffa1872] Correlation ID: 2d42ad67, Job ID: 26c0a546-d0de-4189-8256-98b795585a38, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was added by admin@internal. >2013-11-13 11:54:09,608 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [65f8b538] Running command: InstallVdsCommand internal: true. Entities affected : ID: 24a09956-fd5d-49e2-b03a-270b1468dd1f Type: VDS >2013-11-13 11:54:09,608 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [65f8b538] Before Installation host 24a09956-fd5d-49e2-b03a-270b1468dd1f, monique05 >2013-11-13 11:54:09,609 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [65f8b538] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f, status=Installing, nonOperationalReason=NONE), log id: 32bf88a4 >2013-11-13 11:54:09,632 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [65f8b538] FINISH, SetVdsStatusVDSCommand, log id: 32bf88a4 >2013-11-13 11:54:09,717 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-5-thread-49) [65f8b538] Installation monique-vds05.tlv.redhat.com: Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17 >2013-11-13 11:54:09,760 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [65f8b538] Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17. >2013-11-13 11:54:09,784 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-49) [65f8b538] Installation of monique-vds05.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-11-13 11:54:09,817 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-5-thread-49) SSH execute root@monique-vds05.tlv.redhat.com 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-11-13 11:54:10,481 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Initializing >2013-11-13 11:54:10,486 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Initializing. >2013-11-13 11:54:10,495 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment setup >2013-11-13 11:54:10,500 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment setup. >2013-11-13 11:54:10,501 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment packages setup >2013-11-13 11:54:10,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment packages setup. >2013-11-13 11:54:10,507 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Programs detection >2013-11-13 11:54:10,564 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Programs detection. >2013-11-13 11:54:10,564 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment customization >2013-11-13 11:54:10,568 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment customization. >2013-11-13 11:54:10,596 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20131113115415.log' >2013-11-13 11:54:10,620 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Logs at host located at: '/tmp/ovirt-host-deploy-20131113115415.log'. >2013-11-13 11:54:10,793 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host monique-vds05.tlv.redhat.com reports unique id 4C4C4544-0032-4310-8051-B7C04F39354A >2013-11-13 11:54:10,819 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0032-4310-8051-B7C04F39354A to Host monique-vds05.tlv.redhat.com >2013-11-13 11:54:10,929 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Setup validation >2013-11-13 11:54:10,934 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Setup validation. >2013-11-13 11:54:10,970 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Hardware supports virtualization >2013-11-13 11:54:10,976 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Hardware supports virtualization. >2013-11-13 11:54:10,976 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction setup >2013-11-13 11:54:10,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction setup. >2013-11-13 11:54:10,981 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 11:54:10,985 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 11:54:10,985 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Package installation >2013-11-13 11:54:11,001 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Package installation. >2013-11-13 11:54:12,502 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Initializing Host: monique05 >2013-11-13 11:54:13,602 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 11:54:13,607 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 11:54:14,389 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Enrolling certificate >2013-11-13 11:54:14,394 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Enrolling certificate. >2013-11-13 11:54:15,657 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction commit >2013-11-13 11:54:15,661 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction commit. >2013-11-13 11:54:15,661 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Closing up >2013-11-13 11:54:15,665 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Closing up. >2013-11-13 11:54:16,013 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Starting vdsm >2013-11-13 11:54:16,018 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Starting vdsm. >2013-11-13 11:54:17,602 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Pre-termination >2013-11-13 11:54:17,606 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Pre-termination. >2013-11-13 11:54:17,615 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113115417-monique-vds05.tlv.redhat.com-65f8b538.log' >2013-11-13 11:54:17,619 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113115417-monique-vds05.tlv.redhat.com-65f8b538.log'. >2013-11-13 11:54:17,799 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Termination >2013-11-13 11:54:17,804 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65f8b538, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Termination. >2013-11-13 11:54:17,938 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [65f8b538] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 11:54:17,938 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [65f8b538] Command PollVDS execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 11:54:18,471 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-5-thread-49) [65f8b538] Engine managed to communicate with VDSM agent on host monique05 >2013-11-13 11:54:18,799 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [65f8b538] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f, status=Initializing, nonOperationalReason=NONE), log id: 4468067c >2013-11-13 11:54:18,802 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [65f8b538] FINISH, SetVdsStatusVDSCommand, log id: 4468067c >2013-11-13 11:54:18,802 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [65f8b538] After Installation host monique05, VDS >2013-11-13 11:54:18,806 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [65f8b538] Correlation ID: 65f8b538, Job ID: 26c0a546-d0de-4189-8256-98b795585a38, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installed >2013-11-13 11:54:18,811 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [65f8b538] Lock freed to object EngineLock [exclusiveLocks= key: 24a09956-fd5d-49e2-b03a-270b1468dd1f value: VDS >, sharedLocks= ] >2013-11-13 11:54:21,542 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f, vds=Host[monique05]), log id: 5033b162 >2013-11-13 11:54:21,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GetHardwareInfoVDSCommand, log id: 5033b162 >2013-11-13 11:54:21,609 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f, vds=Host[monique05]), log id: 8438889 >2013-11-13 11:54:21,669 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GetHardwareInfoVDSCommand, log id: 8438889 >2013-11-13 11:54:21,722 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-90) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 11:54:21,746 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-90) [fa7a43f] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 11:54:21,750 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-90) [628510df] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 11:54:21,760 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-90) [628510df] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 29f7217 >2013-11-13 11:54:21,871 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-90) [628510df] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 29f7217 >2013-11-13 11:54:21,871 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-90) [628510df] Host monique05 storage connection was succeeded >2013-11-13 11:54:21,878 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 2cfa3c82 >2013-11-13 11:54:24,194 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) FINISH, ConnectStoragePoolVDSCommand, log id: 2cfa3c82 >2013-11-13 11:54:24,206 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-90) [628510df] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f), log id: 5615aa07 >2013-11-13 11:54:24,253 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-90) [628510df] FINISH, SetMOMPolicyParametersVDSCommand, log id: 5615aa07 >2013-11-13 11:54:24,257 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-90) [628510df] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 11:54:24,265 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-90) [3df4ca3c] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 24a09956-fd5d-49e2-b03a-270b1468dd1f Type: VDS >2013-11-13 11:54:24,272 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-90) [6cce14ce] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 24a09956-fd5d-49e2-b03a-270b1468dd1f Type: VDS >2013-11-13 11:54:51,529 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-9) [79e3c487] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 24a09956-fd5d-49e2-b03a-270b1468dd1f Type: VDS >2013-11-13 11:54:51,531 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-9) [79e3c487] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: ea044e1 >2013-11-13 11:54:51,539 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-9) [79e3c487] FINISH, SetVdsStatusVDSCommand, log id: ea044e1 >2013-11-13 11:54:51,562 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (ajp-/127.0.0.1:8702-9) [79e3c487] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 24a09956-fd5d-49e2-b03a-270b1468dd1f Type: VDS >2013-11-13 11:54:51,569 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [79e3c487] Correlation ID: 79e3c487, Job ID: 11352cc2-0358-4e61-afb5-06484bef732d, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was switched to Maintenance mode by admin@internal. >2013-11-13 11:54:54,512 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) [6fa9173a] Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 24a09956-fd5d-49e2-b03a-270b1468dd1f : monique05 >2013-11-13 11:54:54,526 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) Clearing cache of pool: a44a18ab-6f2e-48d4-ab60-889845f498d3 for problematic entities of VDS: monique05. >2013-11-13 11:54:54,528 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-62) [6fa9173a] START, DisconnectStoragePoolVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2), log id: 132a5467 >2013-11-13 11:54:56,539 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-62) [6fa9173a] FINISH, DisconnectStoragePoolVDSCommand, log id: 132a5467 >2013-11-13 11:54:56,539 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-62) [5f15853b] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 11:54:56,550 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-62) [5f15853b] START, DisconnectStorageServerVDSCommand(HostName = monique05, HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2aaf46fa >2013-11-13 11:54:58,692 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-62) [5f15853b] FINISH, DisconnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 2aaf46fa >2013-11-13 11:55:00,543 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-4) [32ab5bc5] Lock Acquired to object EngineLock [exclusiveLocks= key: 24a09956-fd5d-49e2-b03a-270b1468dd1f value: VDS >, sharedLocks= ] >2013-11-13 11:55:00,546 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-4) [32ab5bc5] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 24a09956-fd5d-49e2-b03a-270b1468dd1f Type: VDS >2013-11-13 11:55:00,700 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [32ab5bc5] START, RemoveVdsVDSCommand( HostId = 24a09956-fd5d-49e2-b03a-270b1468dd1f), log id: 57bbe5c7 >2013-11-13 11:55:00,700 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-4) [32ab5bc5] vdsManager::disposing >2013-11-13 11:55:00,701 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [32ab5bc5] FINISH, RemoveVdsVDSCommand, log id: 57bbe5c7 >2013-11-13 11:55:00,703 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [32ab5bc5] Correlation ID: 32ab5bc5, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was removed by admin@internal. >2013-11-13 11:55:00,703 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-4) [32ab5bc5] Lock freed to object EngineLock [exclusiveLocks= key: 24a09956-fd5d-49e2-b03a-270b1468dd1f value: VDS >, sharedLocks= ] >2013-11-13 11:55:09,689 ERROR [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-3) [6f4f5713] Failed to establish session with host monique05: java.security.GeneralSecurityException: Invalid fingerprint c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17, expected c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:18 > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canConnect(AddVdsCommand.java:423) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canDoAction(AddVdsCommand.java:342) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:741) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:356) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.RunAction(Backend.java:353) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:235) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:218) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view9.RunAction(Unknown Source) [common.jar:] > at org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:242) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:246) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:119) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:109) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:128) > at org.ovirt.engine.api.restapi.resource.BackendHostsResource.add(BackendHostsResource.java:102) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_40] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:167) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:269) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:227) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:216) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:542) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:524) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec.jar:1.0.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:499) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488) > at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 11:55:09,700 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-3) [6f4f5713] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server monique-vds05.tlv.redhat.com,VDS_CANNOT_CONNECT_TO_SERVER >2013-11-13 11:55:09,709 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-3) Operation Failed: [Cannot add Host. Connecting to host via SSH has failed, verify that the host is reachable (IP address, routable address etc.) You may refer to the engine.log file for further details.] >2013-11-13 11:56:39,763 ERROR [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-8) [2375b596] Failed to authenticate session with host monique05: javax.naming.AuthenticationException: SSH authentication to 'root@monique-vds05.tlv.redhat.com' failed. Please verify provided credentials. Make sure host is configured for password authentication > at org.ovirt.engine.core.utils.ssh.SSHClient.authenticate(SSHClient.java:359) [utils.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canConnect(AddVdsCommand.java:424) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canDoAction(AddVdsCommand.java:342) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:741) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:356) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.RunAction(Backend.java:353) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:235) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:218) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view9.RunAction(Unknown Source) [common.jar:] > at org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:242) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:246) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:119) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:109) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:128) > at org.ovirt.engine.api.restapi.resource.BackendHostsResource.add(BackendHostsResource.java:102) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_40] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:167) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:269) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:227) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:216) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:542) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:524) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec.jar:1.0.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:499) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488) > at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 11:56:39,768 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-8) [2375b596] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server monique-vds05.tlv.redhat.com,VDS_CANNOT_AUTHENTICATE_TO_SERVER >2013-11-13 11:56:39,776 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-8) Operation Failed: [Cannot add Host. SSH authentication failed, verify authentication parameters are correct (Username/Password, public-key etc.) You may refer to the engine.log file for further details.] >2013-11-13 11:58:52,859 ERROR [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-6) [7ecbb0d8] Failed to establish session with host monique05: java.security.GeneralSecurityException: Invalid fingerprint c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17, expected c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:18 > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canConnect(AddVdsCommand.java:423) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canDoAction(AddVdsCommand.java:342) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:741) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:356) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.RunAction(Backend.java:353) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:235) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:218) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view9.RunAction(Unknown Source) [common.jar:] > at org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:242) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:246) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:119) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:109) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:128) > at org.ovirt.engine.api.restapi.resource.BackendHostsResource.add(BackendHostsResource.java:102) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_40] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:167) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:269) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:227) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:216) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:542) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:524) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec.jar:1.0.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:499) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488) > at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 11:58:52,865 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-6) [7ecbb0d8] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server monique-vds05.tlv.redhat.com,VDS_CANNOT_CONNECT_TO_SERVER >2013-11-13 11:58:52,872 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-6) Operation Failed: [Cannot add Host. Connecting to host via SSH has failed, verify that the host is reachable (IP address, routable address etc.) You may refer to the engine.log file for further details.] >2013-11-13 12:01:24,115 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-13) [7e0b91d4] Running command: AddVdsCommand internal: false. Entities affected : ID: b273ff0a-4519-4d34-a9a9-a8f1a3506fca Type: VdsGroups >2013-11-13 12:01:24,128 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-13) [323c8ca8] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: cb800b83-a8e0-4aa2-8640-9384ea68f69b Type: VDS >2013-11-13 12:01:24,134 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-13) [323c8ca8] Lock freed to object EngineLock [exclusiveLocks= key: a44a18ab-6f2e-48d4-ab60-889845f498d3 value: REGISTER_VDS >, sharedLocks= ] >2013-11-13 12:01:24,135 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-13) [323c8ca8] Cannot get vdsManager for vdsid=cb800b83-a8e0-4aa2-8640-9384ea68f69b >2013-11-13 12:01:24,136 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-13) [323c8ca8] START, RemoveVdsVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b), log id: 262756e3 >2013-11-13 12:01:24,136 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-13) [323c8ca8] Cannot get vdsManager for vdsid=cb800b83-a8e0-4aa2-8640-9384ea68f69b >2013-11-13 12:01:24,136 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-13) [323c8ca8] FINISH, RemoveVdsVDSCommand, log id: 262756e3 >2013-11-13 12:01:24,137 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-13) [323c8ca8] Cannot get vdsManager for vdsid=cb800b83-a8e0-4aa2-8640-9384ea68f69b >2013-11-13 12:01:24,138 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-13) [323c8ca8] START, AddVdsVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b), log id: 885afde >2013-11-13 12:01:24,138 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-13) [323c8ca8] AddVds - entered , starting logic to add VDS cb800b83-a8e0-4aa2-8640-9384ea68f69b >2013-11-13 12:01:24,138 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-13) [323c8ca8] AddVds - VDS cb800b83-a8e0-4aa2-8640-9384ea68f69b was added, will try to add it to the resource manager >2013-11-13 12:01:24,139 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-13) [323c8ca8] Entered VdsManager constructor >2013-11-13 12:01:24,250 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-13) [323c8ca8] Initialize vdsBroker (monique-vds05.tlv.redhat.com,54,321) >2013-11-13 12:01:24,251 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-13) [323c8ca8] VDS cb800b83-a8e0-4aa2-8640-9384ea68f69b was added to the Resource Manager >2013-11-13 12:01:24,251 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-13) [323c8ca8] FINISH, AddVdsVDSCommand, log id: 885afde >2013-11-13 12:01:24,255 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) [323c8ca8] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:01:24,268 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [5ad18d9e] Lock Acquired to object EngineLock [exclusiveLocks= key: cb800b83-a8e0-4aa2-8640-9384ea68f69b value: VDS >, sharedLocks= ] >2013-11-13 12:01:24,270 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) [323c8ca8] Correlation ID: 7e0b91d4, Job ID: 151ccf46-6b0e-4e48-be0a-44583bb79ea8, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was added by admin@internal. >2013-11-13 12:01:24,297 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [5ad18d9e] Running command: InstallVdsCommand internal: true. Entities affected : ID: cb800b83-a8e0-4aa2-8640-9384ea68f69b Type: VDS >2013-11-13 12:01:24,297 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [5ad18d9e] Before Installation host cb800b83-a8e0-4aa2-8640-9384ea68f69b, monique05 >2013-11-13 12:01:24,526 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [5ad18d9e] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b, status=Installing, nonOperationalReason=NONE), log id: e6b1887 >2013-11-13 12:01:24,549 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [5ad18d9e] FINISH, SetVdsStatusVDSCommand, log id: e6b1887 >2013-11-13 12:01:24,637 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-5-thread-49) [5ad18d9e] Installation monique-vds05.tlv.redhat.com: Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17 >2013-11-13 12:01:24,664 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [5ad18d9e] Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17. >2013-11-13 12:01:24,681 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-49) [5ad18d9e] Installation of monique-vds05.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-11-13 12:01:24,701 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-5-thread-49) SSH execute root@monique-vds05.tlv.redhat.com 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-11-13 12:01:25,378 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Initializing >2013-11-13 12:01:25,383 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Initializing. >2013-11-13 12:01:25,392 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment setup >2013-11-13 12:01:25,396 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment setup. >2013-11-13 12:01:25,397 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment packages setup >2013-11-13 12:01:25,401 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment packages setup. >2013-11-13 12:01:25,402 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Programs detection >2013-11-13 12:01:25,410 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Programs detection. >2013-11-13 12:01:25,410 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment customization >2013-11-13 12:01:25,415 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment customization. >2013-11-13 12:01:25,425 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20131113120130.log' >2013-11-13 12:01:25,430 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Logs at host located at: '/tmp/ovirt-host-deploy-20131113120130.log'. >2013-11-13 12:01:25,554 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host monique-vds05.tlv.redhat.com reports unique id 4C4C4544-0032-4310-8051-B7C04F39354A >2013-11-13 12:01:25,579 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0032-4310-8051-B7C04F39354A to Host monique-vds05.tlv.redhat.com >2013-11-13 12:01:25,640 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Setup validation >2013-11-13 12:01:25,645 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Setup validation. >2013-11-13 12:01:25,681 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Hardware supports virtualization >2013-11-13 12:01:25,685 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Hardware supports virtualization. >2013-11-13 12:01:25,685 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction setup >2013-11-13 12:01:25,690 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction setup. >2013-11-13 12:01:25,690 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 12:01:25,694 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 12:01:25,694 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Package installation >2013-11-13 12:01:25,698 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Package installation. >2013-11-13 12:01:27,253 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Initializing Host: monique05 >2013-11-13 12:01:28,313 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 12:01:28,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 12:01:29,109 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Enrolling certificate >2013-11-13 12:01:29,113 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Enrolling certificate. >2013-11-13 12:01:30,351 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction commit >2013-11-13 12:01:30,356 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction commit. >2013-11-13 12:01:30,356 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Closing up >2013-11-13 12:01:30,359 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Closing up. >2013-11-13 12:01:30,711 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Starting vdsm >2013-11-13 12:01:30,716 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Starting vdsm. >2013-11-13 12:01:32,364 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Pre-termination >2013-11-13 12:01:32,368 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Pre-termination. >2013-11-13 12:01:32,376 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113120132-monique-vds05.tlv.redhat.com-5ad18d9e.log' >2013-11-13 12:01:32,380 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113120132-monique-vds05.tlv.redhat.com-5ad18d9e.log'. >2013-11-13 12:01:32,569 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Termination >2013-11-13 12:01:32,573 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5ad18d9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Termination. >2013-11-13 12:01:32,705 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [5ad18d9e] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 12:01:32,705 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [5ad18d9e] Command PollVDS execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 12:01:33,225 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-5-thread-49) [5ad18d9e] Engine managed to communicate with VDSM agent on host monique05 >2013-11-13 12:01:33,534 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [5ad18d9e] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b, status=Initializing, nonOperationalReason=NONE), log id: 6980ae8c >2013-11-13 12:01:33,537 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [5ad18d9e] FINISH, SetVdsStatusVDSCommand, log id: 6980ae8c >2013-11-13 12:01:33,537 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [5ad18d9e] After Installation host monique05, VDS >2013-11-13 12:01:33,554 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [5ad18d9e] Correlation ID: 5ad18d9e, Job ID: 151ccf46-6b0e-4e48-be0a-44583bb79ea8, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installed >2013-11-13 12:01:33,559 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [5ad18d9e] Lock freed to object EngineLock [exclusiveLocks= key: cb800b83-a8e0-4aa2-8640-9384ea68f69b value: VDS >, sharedLocks= ] >2013-11-13 12:01:36,304 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b, vds=Host[monique05]), log id: 685f9245 >2013-11-13 12:01:36,337 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GetHardwareInfoVDSCommand, log id: 685f9245 >2013-11-13 12:01:36,395 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b, vds=Host[monique05]), log id: 2f35eccb >2013-11-13 12:01:36,401 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GetHardwareInfoVDSCommand, log id: 2f35eccb >2013-11-13 12:01:36,422 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-85) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 12:01:36,427 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-85) [5d6abeec] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:01:36,429 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-85) [4ef90e7e] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:01:36,440 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-85) [4ef90e7e] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 85b38da >2013-11-13 12:01:36,542 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-85) [4ef90e7e] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 85b38da >2013-11-13 12:01:36,543 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-85) [4ef90e7e] Host monique05 storage connection was succeeded >2013-11-13 12:01:36,547 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 3b6f7c5b >2013-11-13 12:01:38,852 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) FINISH, ConnectStoragePoolVDSCommand, log id: 3b6f7c5b >2013-11-13 12:01:38,865 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-85) [4ef90e7e] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b), log id: 19426f6 >2013-11-13 12:01:38,912 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-85) [4ef90e7e] FINISH, SetMOMPolicyParametersVDSCommand, log id: 19426f6 >2013-11-13 12:01:38,916 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-85) [4ef90e7e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:01:38,920 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-85) [46b0430a] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: cb800b83-a8e0-4aa2-8640-9384ea68f69b Type: VDS >2013-11-13 12:01:38,923 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-85) [786d7159] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: cb800b83-a8e0-4aa2-8640-9384ea68f69b Type: VDS >2013-11-13 12:06:09,174 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-6) [230f0275] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: cb800b83-a8e0-4aa2-8640-9384ea68f69b Type: VDS >2013-11-13 12:06:09,175 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-6) [230f0275] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 2e047e01 >2013-11-13 12:06:09,179 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-6) [230f0275] FINISH, SetVdsStatusVDSCommand, log id: 2e047e01 >2013-11-13 12:06:09,221 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (ajp-/127.0.0.1:8702-6) [230f0275] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: cb800b83-a8e0-4aa2-8640-9384ea68f69b Type: VDS >2013-11-13 12:06:09,226 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [230f0275] Correlation ID: 230f0275, Job ID: 83afa913-0b29-4aa3-b01e-03f9496d8874, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was switched to Maintenance mode by admin@internal. >2013-11-13 12:06:11,469 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) [353ac9e2] Updated vds status from Preparing for Maintenance to Maintenance in database, vds = cb800b83-a8e0-4aa2-8640-9384ea68f69b : monique05 >2013-11-13 12:06:11,475 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-48) Clearing cache of pool: a44a18ab-6f2e-48d4-ab60-889845f498d3 for problematic entities of VDS: monique05. >2013-11-13 12:06:11,477 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-70) [353ac9e2] START, DisconnectStoragePoolVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2), log id: 216b32f9 >2013-11-13 12:06:13,487 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-70) [353ac9e2] FINISH, DisconnectStoragePoolVDSCommand, log id: 216b32f9 >2013-11-13 12:06:13,487 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-70) [2d55389e] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:06:13,498 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-70) [2d55389e] START, DisconnectStorageServerVDSCommand(HostName = monique05, HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 56875fe2 >2013-11-13 12:06:15,624 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-70) [2d55389e] FINISH, DisconnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 56875fe2 >2013-11-13 12:06:16,955 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-4) [5a4e7476] Lock Acquired to object EngineLock [exclusiveLocks= key: cb800b83-a8e0-4aa2-8640-9384ea68f69b value: VDS >, sharedLocks= ] >2013-11-13 12:06:16,959 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-4) [5a4e7476] Running command: RemoveVdsCommand internal: false. Entities affected : ID: cb800b83-a8e0-4aa2-8640-9384ea68f69b Type: VDS >2013-11-13 12:06:17,114 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [5a4e7476] START, RemoveVdsVDSCommand( HostId = cb800b83-a8e0-4aa2-8640-9384ea68f69b), log id: 69e4e88f >2013-11-13 12:06:17,114 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-4) [5a4e7476] vdsManager::disposing >2013-11-13 12:06:17,114 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [5a4e7476] FINISH, RemoveVdsVDSCommand, log id: 69e4e88f >2013-11-13 12:06:17,117 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [5a4e7476] Correlation ID: 5a4e7476, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was removed by admin@internal. >2013-11-13 12:06:17,117 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-4) [5a4e7476] Lock freed to object EngineLock [exclusiveLocks= key: cb800b83-a8e0-4aa2-8640-9384ea68f69b value: VDS >, sharedLocks= ] >2013-11-13 12:06:47,311 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-11-13 12:06:47,376 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:06:47,713 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [fbc9490] Running command: LogoutUserCommand internal: false. >2013-11-13 12:06:47,736 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [fbc9490] Correlation ID: fbc9490, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:08:06,275 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2013-11-13 12:08:06,301 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:08:06,494 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-12) [43164f4f] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server monique-vds05.tlv.redhat.com,VDS_CANNOT_INSTALL_EMPTY_PASSWORD >2013-11-13 12:08:06,505 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-12) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2013-11-13 12:08:06,512 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-12) Operation Failed: [Cannot install Host with empty password.] >2013-11-13 12:08:06,515 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-12) [73cf67e9] Running command: LogoutUserCommand internal: false. >2013-11-13 12:08:06,522 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [73cf67e9] Correlation ID: 73cf67e9, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:08:34,039 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-13) Running command: LoginUserCommand internal: false. >2013-11-13 12:08:34,070 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:08:34,368 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-13) [c863756] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server monique-vds05.tlv.redhat.com,VDS_CANNOT_INSTALL_EMPTY_PASSWORD >2013-11-13 12:08:34,377 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-13) Operation Failed: [Cannot install Host with empty password.] >2013-11-13 12:08:34,379 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-13) [12805fe6] Running command: LogoutUserCommand internal: false. >2013-11-13 12:08:34,404 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) [12805fe6] Correlation ID: 12805fe6, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:11:12,197 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-11-13 12:11:12,206 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:11:12,444 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-3) [13637afd] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server monique-vds05.tlv.redhat.com,VDS_CANNOT_INSTALL_EMPTY_PASSWORD >2013-11-13 12:11:12,451 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-3) Operation Failed: [Cannot install Host with empty password.] >2013-11-13 12:11:12,454 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [5ea9031b] Running command: LogoutUserCommand internal: false. >2013-11-13 12:11:12,458 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [5ea9031b] Correlation ID: 5ea9031b, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:14:04,955 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-11-13 12:14:04,966 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:14:06,230 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-9) [2a51de28] Running command: AddVdsCommand internal: false. Entities affected : ID: b273ff0a-4519-4d34-a9a9-a8f1a3506fca Type: VdsGroups >2013-11-13 12:14:06,243 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-9) [1b51151f] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:14:06,256 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-9) [1b51151f] Lock freed to object EngineLock [exclusiveLocks= key: a44a18ab-6f2e-48d4-ab60-889845f498d3 value: REGISTER_VDS >, sharedLocks= ] >2013-11-13 12:14:06,256 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-9) [1b51151f] Cannot get vdsManager for vdsid=85d3567e-3953-4f88-8575-972fa401aa7d >2013-11-13 12:14:06,257 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [1b51151f] START, RemoveVdsVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d), log id: 1451c14f >2013-11-13 12:14:06,257 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-9) [1b51151f] Cannot get vdsManager for vdsid=85d3567e-3953-4f88-8575-972fa401aa7d >2013-11-13 12:14:06,258 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [1b51151f] FINISH, RemoveVdsVDSCommand, log id: 1451c14f >2013-11-13 12:14:06,258 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-9) [1b51151f] Cannot get vdsManager for vdsid=85d3567e-3953-4f88-8575-972fa401aa7d >2013-11-13 12:14:06,259 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [1b51151f] START, AddVdsVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d), log id: 46a63417 >2013-11-13 12:14:06,259 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [1b51151f] AddVds - entered , starting logic to add VDS 85d3567e-3953-4f88-8575-972fa401aa7d >2013-11-13 12:14:06,260 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [1b51151f] AddVds - VDS 85d3567e-3953-4f88-8575-972fa401aa7d was added, will try to add it to the resource manager >2013-11-13 12:14:06,260 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-9) [1b51151f] Entered VdsManager constructor >2013-11-13 12:14:06,375 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-9) [1b51151f] Initialize vdsBroker (monique-vds05.tlv.redhat.com,54,321) >2013-11-13 12:14:06,375 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-9) [1b51151f] VDS 85d3567e-3953-4f88-8575-972fa401aa7d was added to the Resource Manager >2013-11-13 12:14:06,376 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [1b51151f] FINISH, AddVdsVDSCommand, log id: 46a63417 >2013-11-13 12:14:06,380 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [1b51151f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:14:06,392 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [6d92b57e] Lock Acquired to object EngineLock [exclusiveLocks= key: 85d3567e-3953-4f88-8575-972fa401aa7d value: VDS >, sharedLocks= ] >2013-11-13 12:14:06,398 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [1b51151f] Correlation ID: 2a51de28, Job ID: 9f2969b7-4bf9-4788-9aea-be38ebb8d33f, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was added by admin@internal. >2013-11-13 12:14:06,407 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [6d92b57e] Running command: InstallVdsCommand internal: true. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:14:06,407 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [6d92b57e] Before Installation host 85d3567e-3953-4f88-8575-972fa401aa7d, monique05 >2013-11-13 12:14:06,418 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [2a19076c] Running command: LogoutUserCommand internal: false. >2013-11-13 12:14:06,431 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [2a19076c] Correlation ID: 2a19076c, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:14:06,535 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [6d92b57e] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, status=Installing, nonOperationalReason=NONE), log id: 4bab634 >2013-11-13 12:14:06,538 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [6d92b57e] FINISH, SetVdsStatusVDSCommand, log id: 4bab634 >2013-11-13 12:14:06,592 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-5-thread-48) [6d92b57e] Installation monique-vds05.tlv.redhat.com: Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17 >2013-11-13 12:14:06,597 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [6d92b57e] Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17. >2013-11-13 12:14:06,620 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-48) [6d92b57e] Installation of monique-vds05.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-11-13 12:14:06,628 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-5-thread-48) SSH execute root@monique-vds05.tlv.redhat.com 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-11-13 12:14:07,386 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Initializing >2013-11-13 12:14:07,391 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Initializing. >2013-11-13 12:14:07,403 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment setup >2013-11-13 12:14:07,437 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment setup. >2013-11-13 12:14:07,437 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment packages setup >2013-11-13 12:14:07,474 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment packages setup. >2013-11-13 12:14:07,475 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Programs detection >2013-11-13 12:14:07,505 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Programs detection. >2013-11-13 12:14:07,505 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment customization >2013-11-13 12:14:07,509 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment customization. >2013-11-13 12:14:07,516 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20131113121412.log' >2013-11-13 12:14:07,525 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Logs at host located at: '/tmp/ovirt-host-deploy-20131113121412.log'. >2013-11-13 12:14:07,686 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host monique-vds05.tlv.redhat.com reports unique id 4C4C4544-0032-4310-8051-B7C04F39354A >2013-11-13 12:14:07,712 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0032-4310-8051-B7C04F39354A to Host monique-vds05.tlv.redhat.com >2013-11-13 12:14:07,850 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Setup validation >2013-11-13 12:14:07,854 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Setup validation. >2013-11-13 12:14:07,893 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Hardware supports virtualization >2013-11-13 12:14:07,898 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Hardware supports virtualization. >2013-11-13 12:14:07,898 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction setup >2013-11-13 12:14:07,902 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction setup. >2013-11-13 12:14:07,902 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 12:14:07,905 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 12:14:07,905 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Package installation >2013-11-13 12:14:07,909 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Package installation. >2013-11-13 12:14:09,377 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Initializing Host: monique05 >2013-11-13 12:14:10,527 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 12:14:10,532 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 12:14:11,362 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Enrolling certificate >2013-11-13 12:14:11,367 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Enrolling certificate. >2013-11-13 12:14:12,607 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction commit >2013-11-13 12:14:12,612 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction commit. >2013-11-13 12:14:12,612 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Closing up >2013-11-13 12:14:12,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Closing up. >2013-11-13 12:14:12,965 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Starting vdsm >2013-11-13 12:14:12,969 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Starting vdsm. >2013-11-13 12:14:14,627 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Pre-termination >2013-11-13 12:14:14,632 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Pre-termination. >2013-11-13 12:14:14,641 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113121414-monique-vds05.tlv.redhat.com-6d92b57e.log' >2013-11-13 12:14:14,644 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113121414-monique-vds05.tlv.redhat.com-6d92b57e.log'. >2013-11-13 12:14:14,823 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Termination >2013-11-13 12:14:14,827 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6d92b57e, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Termination. >2013-11-13 12:14:14,960 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-48) [6d92b57e] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 12:14:14,961 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-48) [6d92b57e] Command PollVDS execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 12:14:15,482 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-5-thread-48) [6d92b57e] Engine managed to communicate with VDSM agent on host monique05 >2013-11-13 12:14:15,762 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [6d92b57e] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, status=Initializing, nonOperationalReason=NONE), log id: 23cd5607 >2013-11-13 12:14:15,767 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [6d92b57e] FINISH, SetVdsStatusVDSCommand, log id: 23cd5607 >2013-11-13 12:14:15,767 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [6d92b57e] After Installation host monique05, VDS >2013-11-13 12:14:15,771 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [6d92b57e] Correlation ID: 6d92b57e, Job ID: 9f2969b7-4bf9-4788-9aea-be38ebb8d33f, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installed >2013-11-13 12:14:15,776 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [6d92b57e] Lock freed to object EngineLock [exclusiveLocks= key: 85d3567e-3953-4f88-8575-972fa401aa7d value: VDS >, sharedLocks= ] >2013-11-13 12:14:18,428 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-54) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, vds=Host[monique05]), log id: 6cacc437 >2013-11-13 12:14:18,458 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-54) FINISH, GetHardwareInfoVDSCommand, log id: 6cacc437 >2013-11-13 12:14:18,513 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-54) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, vds=Host[monique05]), log id: 60ca507a >2013-11-13 12:14:18,519 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-54) FINISH, GetHardwareInfoVDSCommand, log id: 60ca507a >2013-11-13 12:14:18,539 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 12:14:18,545 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-54) [350408ec] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:14:18,547 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-54) [2f74ecdd] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:14:18,561 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-54) [2f74ecdd] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 3696fb19 >2013-11-13 12:14:18,667 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-54) [2f74ecdd] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 3696fb19 >2013-11-13 12:14:18,667 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-54) [2f74ecdd] Host monique05 storage connection was succeeded >2013-11-13 12:14:18,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-49) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 2e873415 >2013-11-13 12:14:21,991 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-49) FINISH, ConnectStoragePoolVDSCommand, log id: 2e873415 >2013-11-13 12:14:22,003 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-54) [2f74ecdd] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d), log id: 442198a0 >2013-11-13 12:14:22,050 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-54) [2f74ecdd] FINISH, SetMOMPolicyParametersVDSCommand, log id: 442198a0 >2013-11-13 12:14:22,053 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) [2f74ecdd] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:14:22,058 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-54) [121d4bbd] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:14:22,061 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-54) [5c8d99d7] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:15:37,675 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-11-13 12:15:37,695 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:16:46,937 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-13) Running command: LoginUserCommand internal: false. >2013-11-13 12:16:46,951 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:16:48,672 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-11-13 12:16:48,705 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:17:01,130 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-11-13 12:17:01,142 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:17:01,150 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [472ddc1e] Running command: LogoutUserCommand internal: false. >2013-11-13 12:17:01,156 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [472ddc1e] Correlation ID: 472ddc1e, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:17:57,497 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2013-11-13 12:17:57,505 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:18:58,829 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-11-13 12:18:58,848 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:18:59,001 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-4) [56fe6e28] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:18:59,003 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-4) [56fe6e28] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 16a24bba >2013-11-13 12:18:59,006 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-4) [56fe6e28] FINISH, SetVdsStatusVDSCommand, log id: 16a24bba >2013-11-13 12:18:59,037 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (ajp-/127.0.0.1:8702-4) [56fe6e28] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:18:59,042 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [56fe6e28] Correlation ID: 56fe6e28, Job ID: 223f87cf-0be3-4cdf-b49d-4afdadcba55b, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was switched to Maintenance mode by admin@internal. >2013-11-13 12:18:59,325 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [2bc60c05] Running command: LogoutUserCommand internal: false. >2013-11-13 12:18:59,329 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [2bc60c05] Correlation ID: 2bc60c05, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:18:59,918 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 85d3567e-3953-4f88-8575-972fa401aa7d : monique05 >2013-11-13 12:18:59,926 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-48) Clearing cache of pool: a44a18ab-6f2e-48d4-ab60-889845f498d3 for problematic entities of VDS: monique05. >2013-11-13 12:18:59,928 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-36) START, DisconnectStoragePoolVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2), log id: 6b50b927 >2013-11-13 12:19:02,938 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-36) FINISH, DisconnectStoragePoolVDSCommand, log id: 6b50b927 >2013-11-13 12:19:02,939 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-36) [40a0a9ae] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:19:02,949 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-36) [40a0a9ae] START, DisconnectStorageServerVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: d1e052a >2013-11-13 12:19:05,088 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-36) [40a0a9ae] FINISH, DisconnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: d1e052a >2013-11-13 12:19:29,443 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-11-13 12:19:29,451 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:19:29,505 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-5) [37b5d3ca] Lock Acquired to object EngineLock [exclusiveLocks= key: 85d3567e-3953-4f88-8575-972fa401aa7d value: VDS >, sharedLocks= ] >2013-11-13 12:19:29,511 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-5) [37b5d3ca] Running command: ActivateVdsCommand internal: false. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:19:29,512 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-5) [37b5d3ca] Before acquiring lock in order to prevent monitoring for host monique05 from data-center new-nfs >2013-11-13 12:19:29,512 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-5) [37b5d3ca] Lock acquired, from now a monitoring of host will be skipped for host monique05 from data-center new-nfs >2013-11-13 12:19:29,520 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-5) [37b5d3ca] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, status=Unassigned, nonOperationalReason=NONE), log id: 4e513538 >2013-11-13 12:19:29,522 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-5) [37b5d3ca] FINISH, SetVdsStatusVDSCommand, log id: 4e513538 >2013-11-13 12:19:29,572 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [37b5d3ca] START, ActivateVdsVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d), log id: 251e90a8 >2013-11-13 12:19:29,639 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (ajp-/127.0.0.1:8702-5) [37b5d3ca] START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, vds=Host[monique05]), log id: 71b53179 >2013-11-13 12:19:29,653 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (ajp-/127.0.0.1:8702-5) [37b5d3ca] FINISH, GetHardwareInfoVDSCommand, log id: 71b53179 >2013-11-13 12:19:29,673 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (ajp-/127.0.0.1:8702-5) [7c01dcc0] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:19:29,675 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (ajp-/127.0.0.1:8702-5) [3b4775f2] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:19:29,675 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [3b4775f2] FINISH, ActivateVdsVDSCommand, return: Host[monique05], log id: 251e90a8 >2013-11-13 12:19:29,998 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-5) [3b4775f2] Activate finished. Lock released. Monitoring can run now for host monique05 from data-center new-nfs >2013-11-13 12:19:30,003 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [3b4775f2] Correlation ID: 37b5d3ca, Job ID: e4e075dc-3ba1-4c43-8bbf-45e91892c431, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was activated by admin@internal. >2013-11-13 12:19:30,007 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-5) [3b4775f2] Lock freed to object EngineLock [exclusiveLocks= key: 85d3567e-3953-4f88-8575-972fa401aa7d value: VDS >, sharedLocks= ] >2013-11-13 12:19:30,010 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [2f7e78bc] Running command: LogoutUserCommand internal: false. >2013-11-13 12:19:30,013 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [2f7e78bc] Correlation ID: 2f7e78bc, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:19:32,184 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-99) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, vds=Host[monique05]), log id: 1c5e20d5 >2013-11-13 12:19:32,190 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-99) FINISH, GetHardwareInfoVDSCommand, log id: 1c5e20d5 >2013-11-13 12:19:32,235 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-99) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 12:19:32,245 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-99) [c7c7e92] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:19:32,259 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-99) [7cb027ea] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:19:32,332 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-99) [7cb027ea] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 34352556 >2013-11-13 12:19:32,382 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-99) [7cb027ea] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 34352556 >2013-11-13 12:19:32,383 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-99) [7cb027ea] Host monique05 storage connection was succeeded >2013-11-13 12:19:32,392 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 27f1979d >2013-11-13 12:19:34,669 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) FINISH, ConnectStoragePoolVDSCommand, log id: 27f1979d >2013-11-13 12:19:34,731 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-99) [7cb027ea] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d), log id: 446bc41d >2013-11-13 12:19:34,779 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-99) [7cb027ea] FINISH, SetMOMPolicyParametersVDSCommand, log id: 446bc41d >2013-11-13 12:19:34,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-99) [7cb027ea] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:19:34,805 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-99) [1689af6d] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:19:34,821 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-99) [5faae8b8] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:20:22,589 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-6) [61dc16dc] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server monique-vds05.tlv.redhat.com,ACTION_TYPE_FAILED_NAME_ALREADY_USED >2013-11-13 12:20:22,609 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-6) Operation Failed: [Cannot add Host. The Host name is already in use, please choose a unique name and try again.] >2013-11-13 12:20:26,869 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-7) [7e3f5e51] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:20:26,870 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-7) [7e3f5e51] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 359e8cbd >2013-11-13 12:20:26,873 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-7) [7e3f5e51] FINISH, SetVdsStatusVDSCommand, log id: 359e8cbd >2013-11-13 12:20:26,914 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (ajp-/127.0.0.1:8702-7) [7e3f5e51] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:20:26,922 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [7e3f5e51] Correlation ID: 7e3f5e51, Job ID: e0c51e24-9a57-468c-abee-58e01118cb87, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was switched to Maintenance mode by admin@internal. >2013-11-13 12:20:29,405 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) [60655011] Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 85d3567e-3953-4f88-8575-972fa401aa7d : monique05 >2013-11-13 12:20:29,445 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-48) Clearing cache of pool: a44a18ab-6f2e-48d4-ab60-889845f498d3 for problematic entities of VDS: monique05. >2013-11-13 12:20:29,450 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-74) [60655011] START, DisconnectStoragePoolVDSCommand(HostName = monique05, HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2), log id: 4010bb08 >2013-11-13 12:20:29,880 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-11) [8fae9ab] Lock Acquired to object EngineLock [exclusiveLocks= key: 85d3567e-3953-4f88-8575-972fa401aa7d value: VDS >, sharedLocks= ] >2013-11-13 12:20:29,884 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-11) [8fae9ab] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 85d3567e-3953-4f88-8575-972fa401aa7d Type: VDS >2013-11-13 12:20:30,032 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-11) [8fae9ab] START, RemoveVdsVDSCommand( HostId = 85d3567e-3953-4f88-8575-972fa401aa7d), log id: 4b4032bb >2013-11-13 12:20:31,485 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-74) [60655011] FINISH, DisconnectStoragePoolVDSCommand, log id: 4010bb08 >2013-11-13 12:20:31,486 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-74) [6395987] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:20:31,492 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-74) [6395987] START, DisconnectStorageServerVDSCommand( HostId = 85d3567e-3953-4f88-8575-972fa401aa7d, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 79fa6149 >2013-11-13 12:20:33,674 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-74) [6395987] FINISH, DisconnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 79fa6149 >2013-11-13 12:20:33,703 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-11) [8fae9ab] vdsManager::disposing >2013-11-13 12:20:33,703 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-11) [8fae9ab] FINISH, RemoveVdsVDSCommand, log id: 4b4032bb >2013-11-13 12:20:33,707 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [8fae9ab] Correlation ID: 8fae9ab, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was removed by admin@internal. >2013-11-13 12:20:33,707 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-11) [8fae9ab] Lock freed to object EngineLock [exclusiveLocks= key: 85d3567e-3953-4f88-8575-972fa401aa7d value: VDS >, sharedLocks= ] >2013-11-13 12:20:41,492 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-12) [67d56086] Running command: AddVdsCommand internal: false. Entities affected : ID: b273ff0a-4519-4d34-a9a9-a8f1a3506fca Type: VdsGroups >2013-11-13 12:20:41,507 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-12) [478ab598] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 5b61ee40-3b89-4de8-b823-f689886c679f Type: VDS >2013-11-13 12:20:41,512 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-12) [478ab598] Lock freed to object EngineLock [exclusiveLocks= key: a44a18ab-6f2e-48d4-ab60-889845f498d3 value: REGISTER_VDS >, sharedLocks= ] >2013-11-13 12:20:41,512 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-12) [478ab598] Cannot get vdsManager for vdsid=5b61ee40-3b89-4de8-b823-f689886c679f >2013-11-13 12:20:41,515 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [478ab598] START, RemoveVdsVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f), log id: 7d195341 >2013-11-13 12:20:41,515 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-12) [478ab598] Cannot get vdsManager for vdsid=5b61ee40-3b89-4de8-b823-f689886c679f >2013-11-13 12:20:41,515 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [478ab598] FINISH, RemoveVdsVDSCommand, log id: 7d195341 >2013-11-13 12:20:41,515 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-12) [478ab598] Cannot get vdsManager for vdsid=5b61ee40-3b89-4de8-b823-f689886c679f >2013-11-13 12:20:41,516 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [478ab598] START, AddVdsVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f), log id: 6642017 >2013-11-13 12:20:41,516 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [478ab598] AddVds - entered , starting logic to add VDS 5b61ee40-3b89-4de8-b823-f689886c679f >2013-11-13 12:20:41,517 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [478ab598] AddVds - VDS 5b61ee40-3b89-4de8-b823-f689886c679f was added, will try to add it to the resource manager >2013-11-13 12:20:41,517 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-12) [478ab598] Entered VdsManager constructor >2013-11-13 12:20:41,632 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-12) [478ab598] Initialize vdsBroker (monique-vds05.tlv.redhat.com,54,321) >2013-11-13 12:20:41,632 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-12) [478ab598] VDS 5b61ee40-3b89-4de8-b823-f689886c679f was added to the Resource Manager >2013-11-13 12:20:41,633 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-12) [478ab598] FINISH, AddVdsVDSCommand, log id: 6642017 >2013-11-13 12:20:41,640 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [478ab598] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:20:41,663 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [478ab598] Correlation ID: 67d56086, Job ID: 0f07771e-d516-499e-bb09-5ab7df5b14e3, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was added by admin@internal. >2013-11-13 12:20:41,669 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [404e5a9f] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b61ee40-3b89-4de8-b823-f689886c679f value: VDS >, sharedLocks= ] >2013-11-13 12:20:41,734 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [404e5a9f] Running command: InstallVdsCommand internal: true. Entities affected : ID: 5b61ee40-3b89-4de8-b823-f689886c679f Type: VDS >2013-11-13 12:20:41,734 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [404e5a9f] Before Installation host 5b61ee40-3b89-4de8-b823-f689886c679f, monique05 >2013-11-13 12:20:41,966 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [404e5a9f] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f, status=Installing, nonOperationalReason=NONE), log id: c4018a2 >2013-11-13 12:20:41,989 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [404e5a9f] FINISH, SetVdsStatusVDSCommand, log id: c4018a2 >2013-11-13 12:20:42,091 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-5-thread-49) [404e5a9f] Installation monique-vds05.tlv.redhat.com: Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17 >2013-11-13 12:20:42,106 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [404e5a9f] Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17. >2013-11-13 12:20:42,132 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-49) [404e5a9f] Installation of monique-vds05.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-11-13 12:20:42,139 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-5-thread-49) SSH execute root@monique-vds05.tlv.redhat.com 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-11-13 12:20:42,863 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Initializing >2013-11-13 12:20:42,868 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Initializing. >2013-11-13 12:20:42,877 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment setup >2013-11-13 12:20:42,886 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment setup. >2013-11-13 12:20:42,886 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment packages setup >2013-11-13 12:20:42,903 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment packages setup. >2013-11-13 12:20:42,903 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Programs detection >2013-11-13 12:20:42,910 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Programs detection. >2013-11-13 12:20:42,911 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment customization >2013-11-13 12:20:42,918 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment customization. >2013-11-13 12:20:42,925 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20131113122047.log' >2013-11-13 12:20:42,929 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Logs at host located at: '/tmp/ovirt-host-deploy-20131113122047.log'. >2013-11-13 12:20:43,066 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host monique-vds05.tlv.redhat.com reports unique id 4C4C4544-0032-4310-8051-B7C04F39354A >2013-11-13 12:20:43,093 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0032-4310-8051-B7C04F39354A to Host monique-vds05.tlv.redhat.com >2013-11-13 12:20:43,171 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Setup validation >2013-11-13 12:20:43,182 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Setup validation. >2013-11-13 12:20:43,212 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Hardware supports virtualization >2013-11-13 12:20:43,216 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Hardware supports virtualization. >2013-11-13 12:20:43,217 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction setup >2013-11-13 12:20:43,221 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction setup. >2013-11-13 12:20:43,221 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 12:20:43,225 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 12:20:43,225 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Package installation >2013-11-13 12:20:43,232 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Package installation. >2013-11-13 12:20:44,640 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) [5f15853b] Initializing Host: monique05 >2013-11-13 12:20:45,839 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 12:20:45,845 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 12:20:46,583 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Enrolling certificate >2013-11-13 12:20:46,588 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Enrolling certificate. >2013-11-13 12:20:47,819 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction commit >2013-11-13 12:20:47,824 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction commit. >2013-11-13 12:20:47,824 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Closing up >2013-11-13 12:20:47,840 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Closing up. >2013-11-13 12:20:48,169 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Starting vdsm >2013-11-13 12:20:48,174 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Starting vdsm. >2013-11-13 12:20:49,865 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Pre-termination >2013-11-13 12:20:49,869 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Pre-termination. >2013-11-13 12:20:49,877 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113122049-monique-vds05.tlv.redhat.com-404e5a9f.log' >2013-11-13 12:20:49,882 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113122049-monique-vds05.tlv.redhat.com-404e5a9f.log'. >2013-11-13 12:20:50,161 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Termination >2013-11-13 12:20:50,186 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 404e5a9f, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Termination. >2013-11-13 12:20:50,276 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [404e5a9f] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 12:20:50,276 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-49) [404e5a9f] Command PollVDS execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 12:20:50,926 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-5-thread-49) [404e5a9f] Engine managed to communicate with VDSM agent on host monique05 >2013-11-13 12:20:51,137 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [404e5a9f] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f, status=Initializing, nonOperationalReason=NONE), log id: 1928af2e >2013-11-13 12:20:51,140 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [404e5a9f] FINISH, SetVdsStatusVDSCommand, log id: 1928af2e >2013-11-13 12:20:51,140 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [404e5a9f] After Installation host monique05, VDS >2013-11-13 12:20:51,162 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [404e5a9f] Correlation ID: 404e5a9f, Job ID: 0f07771e-d516-499e-bb09-5ab7df5b14e3, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installed >2013-11-13 12:20:51,165 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-49) [404e5a9f] Lock freed to object EngineLock [exclusiveLocks= key: 5b61ee40-3b89-4de8-b823-f689886c679f value: VDS >, sharedLocks= ] >2013-11-13 12:20:53,687 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f, vds=Host[monique05]), log id: 59fdd403 >2013-11-13 12:20:53,715 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] FINISH, GetHardwareInfoVDSCommand, log id: 59fdd403 >2013-11-13 12:20:53,769 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f, vds=Host[monique05]), log id: 205530ab >2013-11-13 12:20:53,775 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] FINISH, GetHardwareInfoVDSCommand, log id: 205530ab >2013-11-13 12:20:53,839 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) [69ba2f4c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 12:20:53,846 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-92) [3569a61e] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:20:53,854 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-92) [206212d5] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:20:53,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-92) [206212d5] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 16e10a92 >2013-11-13 12:20:54,020 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-92) [206212d5] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 16e10a92 >2013-11-13 12:20:54,020 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-92) [206212d5] Host monique05 storage connection was succeeded >2013-11-13 12:20:54,025 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 6ab7c652 >2013-11-13 12:20:57,391 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) FINISH, ConnectStoragePoolVDSCommand, log id: 6ab7c652 >2013-11-13 12:20:57,404 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-92) [206212d5] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f), log id: 6bac0b9a >2013-11-13 12:20:57,452 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-92) [206212d5] FINISH, SetMOMPolicyParametersVDSCommand, log id: 6bac0b9a >2013-11-13 12:20:57,456 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) [206212d5] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:20:57,460 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-92) [5605cb9f] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 5b61ee40-3b89-4de8-b823-f689886c679f Type: VDS >2013-11-13 12:20:57,462 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-92) [3fd01ed] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 5b61ee40-3b89-4de8-b823-f689886c679f Type: VDS >2013-11-13 12:22:05,945 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-4) [4bd696bb] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 5b61ee40-3b89-4de8-b823-f689886c679f Type: VDS >2013-11-13 12:22:05,947 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-4) [4bd696bb] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 7ee76dfd >2013-11-13 12:22:05,949 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-4) [4bd696bb] FINISH, SetVdsStatusVDSCommand, log id: 7ee76dfd >2013-11-13 12:22:05,976 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (ajp-/127.0.0.1:8702-4) [4bd696bb] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 5b61ee40-3b89-4de8-b823-f689886c679f Type: VDS >2013-11-13 12:22:05,983 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [4bd696bb] Correlation ID: 4bd696bb, Job ID: 042b77e7-901d-4429-ba84-08b31f35d581, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was switched to Maintenance mode by admin@internal. >2013-11-13 12:22:06,963 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 5b61ee40-3b89-4de8-b823-f689886c679f : monique05 >2013-11-13 12:22:06,969 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-48) Clearing cache of pool: a44a18ab-6f2e-48d4-ab60-889845f498d3 for problematic entities of VDS: monique05. >2013-11-13 12:22:06,971 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-17) START, DisconnectStoragePoolVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2), log id: 745978b6 >2013-11-13 12:22:08,989 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, DisconnectStoragePoolVDSCommand, log id: 745978b6 >2013-11-13 12:22:08,989 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-17) [28517b14] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:22:09,062 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-17) [28517b14] START, DisconnectStorageServerVDSCommand(HostName = monique05, HostId = 5b61ee40-3b89-4de8-b823-f689886c679f, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5e9effbe >2013-11-13 12:22:09,193 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-9) [1564fd50] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b61ee40-3b89-4de8-b823-f689886c679f value: VDS >, sharedLocks= ] >2013-11-13 12:22:09,196 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-9) [1564fd50] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 5b61ee40-3b89-4de8-b823-f689886c679f Type: VDS >2013-11-13 12:22:09,367 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [1564fd50] START, RemoveVdsVDSCommand( HostId = 5b61ee40-3b89-4de8-b823-f689886c679f), log id: 2a587620 >2013-11-13 12:22:11,243 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-17) [28517b14] FINISH, DisconnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 5e9effbe >2013-11-13 12:22:11,270 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-9) [1564fd50] vdsManager::disposing >2013-11-13 12:22:11,270 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [1564fd50] FINISH, RemoveVdsVDSCommand, log id: 2a587620 >2013-11-13 12:22:11,292 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [1564fd50] Correlation ID: 1564fd50, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was removed by admin@internal. >2013-11-13 12:22:11,293 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-9) [1564fd50] Lock freed to object EngineLock [exclusiveLocks= key: 5b61ee40-3b89-4de8-b823-f689886c679f value: VDS >, sharedLocks= ] >2013-11-13 12:22:20,008 ERROR [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-13) [392d4493] Failed to authenticate session with host monique05: javax.naming.AuthenticationException: SSH authentication to 'root@monique-vds05.tlv.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host > at org.ovirt.engine.core.utils.ssh.SSHClient.authenticate(SSHClient.java:359) [utils.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canConnect(AddVdsCommand.java:424) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canDoAction(AddVdsCommand.java:342) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:741) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:356) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.RunAction(Backend.java:353) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:235) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:218) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view9.RunAction(Unknown Source) [common.jar:] > at org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:242) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:246) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:119) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:109) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:128) > at org.ovirt.engine.api.restapi.resource.BackendHostsResource.add(BackendHostsResource.java:102) > at sun.reflect.GeneratedMethodAccessor209.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:167) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:269) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:227) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:216) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:542) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:524) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec.jar:1.0.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:499) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488) > at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 12:22:20,013 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-13) [392d4493] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server monique-vds05.tlv.redhat.com,VDS_CANNOT_AUTHENTICATE_TO_SERVER >2013-11-13 12:22:20,025 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-13) Operation Failed: [Cannot add Host. SSH authentication failed, verify authentication parameters are correct (Username/Password, public-key etc.) You may refer to the engine.log file for further details.] >2013-11-13 12:26:35,226 ERROR [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-6) [5accc7e8] Failed to establish session with host monique05: java.security.GeneralSecurityException: Invalid fingerprint c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17, expected c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:18 > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canConnect(AddVdsCommand.java:423) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canDoAction(AddVdsCommand.java:342) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:741) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:356) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.RunAction(Backend.java:353) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:235) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:218) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view9.RunAction(Unknown Source) [common.jar:] > at org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:242) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:246) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:119) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:109) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:128) > at org.ovirt.engine.api.restapi.resource.BackendHostsResource.add(BackendHostsResource.java:102) > at sun.reflect.GeneratedMethodAccessor209.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:167) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:269) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:227) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:216) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:542) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:524) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec.jar:1.0.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:499) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488) > at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 12:26:35,232 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-6) [5accc7e8] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server monique-vds05.tlv.redhat.com,VDS_CANNOT_CONNECT_TO_SERVER >2013-11-13 12:26:35,240 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-6) Operation Failed: [Cannot add Host. Connecting to host via SSH has failed, verify that the host is reachable (IP address, routable address etc.) You may refer to the engine.log file for further details.] >2013-11-13 12:27:41,621 ERROR [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-7) [62682f7f] Failed to establish session with host monique05: java.security.GeneralSecurityException: Invalid fingerprint c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17, expected c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:18 > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canConnect(AddVdsCommand.java:423) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canDoAction(AddVdsCommand.java:342) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:741) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:356) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.RunAction(Backend.java:353) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:235) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:218) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.2.1.Final-redhat-10] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view9.RunAction(Unknown Source) [common.jar:] > at org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:242) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:246) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:119) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:109) > at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:128) > at org.ovirt.engine.api.restapi.resource.BackendHostsResource.add(BackendHostsResource.java:102) > at sun.reflect.GeneratedMethodAccessor209.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:167) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:269) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:227) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:216) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:542) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:524) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) [resteasy-jaxrs.jar:2.3.6.Final-redhat-1] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec.jar:1.0.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:499) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488) > at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40] > >2013-11-13 12:27:41,626 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-7) [62682f7f] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server monique-vds05.tlv.redhat.com,VDS_CANNOT_CONNECT_TO_SERVER >2013-11-13 12:27:41,640 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-7) Operation Failed: [Cannot add Host. Connecting to host via SSH has failed, verify that the host is reachable (IP address, routable address etc.) You may refer to the engine.log file for further details.] >2013-11-13 12:28:21,243 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-3) [7bd2751b] Running command: AddVdsCommand internal: false. Entities affected : ID: b273ff0a-4519-4d34-a9a9-a8f1a3506fca Type: VdsGroups >2013-11-13 12:28:21,285 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-3) [1223ce53] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:28:21,298 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-3) [1223ce53] Lock freed to object EngineLock [exclusiveLocks= key: a44a18ab-6f2e-48d4-ab60-889845f498d3 value: REGISTER_VDS >, sharedLocks= ] >2013-11-13 12:28:21,298 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-3) [1223ce53] Cannot get vdsManager for vdsid=66d9cb34-58a7-4693-91b3-aaeedf863e84 >2013-11-13 12:28:21,300 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [1223ce53] START, RemoveVdsVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84), log id: 3454e132 >2013-11-13 12:28:21,300 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-3) [1223ce53] Cannot get vdsManager for vdsid=66d9cb34-58a7-4693-91b3-aaeedf863e84 >2013-11-13 12:28:21,300 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [1223ce53] FINISH, RemoveVdsVDSCommand, log id: 3454e132 >2013-11-13 12:28:21,300 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-3) [1223ce53] Cannot get vdsManager for vdsid=66d9cb34-58a7-4693-91b3-aaeedf863e84 >2013-11-13 12:28:21,301 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [1223ce53] START, AddVdsVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84), log id: 34d5c53e >2013-11-13 12:28:21,301 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [1223ce53] AddVds - entered , starting logic to add VDS 66d9cb34-58a7-4693-91b3-aaeedf863e84 >2013-11-13 12:28:21,302 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [1223ce53] AddVds - VDS 66d9cb34-58a7-4693-91b3-aaeedf863e84 was added, will try to add it to the resource manager >2013-11-13 12:28:21,302 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-3) [1223ce53] Entered VdsManager constructor >2013-11-13 12:28:21,420 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-3) [1223ce53] Initialize vdsBroker (monique-vds05.tlv.redhat.com,54,321) >2013-11-13 12:28:21,421 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-3) [1223ce53] VDS 66d9cb34-58a7-4693-91b3-aaeedf863e84 was added to the Resource Manager >2013-11-13 12:28:21,421 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [1223ce53] FINISH, AddVdsVDSCommand, log id: 34d5c53e >2013-11-13 12:28:21,425 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [1223ce53] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:28:21,445 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [3c6c9922] Lock Acquired to object EngineLock [exclusiveLocks= key: 66d9cb34-58a7-4693-91b3-aaeedf863e84 value: VDS >, sharedLocks= ] >2013-11-13 12:28:21,475 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [1223ce53] Correlation ID: 7bd2751b, Job ID: f312a962-4295-4b98-9a90-d5a80b47569e, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was added by admin@internal. >2013-11-13 12:28:21,504 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [3c6c9922] Running command: InstallVdsCommand internal: true. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:28:21,505 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [3c6c9922] Before Installation host 66d9cb34-58a7-4693-91b3-aaeedf863e84, monique05 >2013-11-13 12:28:21,733 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [3c6c9922] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, status=Installing, nonOperationalReason=NONE), log id: 5c554e1c >2013-11-13 12:28:21,756 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [3c6c9922] FINISH, SetVdsStatusVDSCommand, log id: 5c554e1c >2013-11-13 12:28:21,840 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-5-thread-48) [3c6c9922] Installation monique-vds05.tlv.redhat.com: Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17 >2013-11-13 12:28:21,858 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [3c6c9922] Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17. >2013-11-13 12:28:21,884 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-48) [3c6c9922] Installation of monique-vds05.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-11-13 12:28:22,091 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-5-thread-48) SSH execute root@monique-vds05.tlv.redhat.com 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-11-13 12:28:22,731 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Initializing >2013-11-13 12:28:22,738 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Initializing. >2013-11-13 12:28:22,747 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment setup >2013-11-13 12:28:22,752 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment setup. >2013-11-13 12:28:22,752 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment packages setup >2013-11-13 12:28:22,757 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment packages setup. >2013-11-13 12:28:22,757 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Programs detection >2013-11-13 12:28:22,761 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Programs detection. >2013-11-13 12:28:22,762 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment customization >2013-11-13 12:28:22,765 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment customization. >2013-11-13 12:28:22,781 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20131113122827.log' >2013-11-13 12:28:22,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Logs at host located at: '/tmp/ovirt-host-deploy-20131113122827.log'. >2013-11-13 12:28:22,929 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host monique-vds05.tlv.redhat.com reports unique id 4C4C4544-0032-4310-8051-B7C04F39354A >2013-11-13 12:28:22,956 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0032-4310-8051-B7C04F39354A to Host monique-vds05.tlv.redhat.com >2013-11-13 12:28:23,010 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Setup validation >2013-11-13 12:28:23,018 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Setup validation. >2013-11-13 12:28:23,051 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Hardware supports virtualization >2013-11-13 12:28:23,055 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Hardware supports virtualization. >2013-11-13 12:28:23,055 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction setup >2013-11-13 12:28:23,073 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction setup. >2013-11-13 12:28:23,073 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 12:28:23,077 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 12:28:23,077 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Package installation >2013-11-13 12:28:23,081 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Package installation. >2013-11-13 12:28:24,423 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Initializing Host: monique05 >2013-11-13 12:28:25,682 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 12:28:25,687 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 12:28:26,492 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Enrolling certificate >2013-11-13 12:28:26,500 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Enrolling certificate. >2013-11-13 12:28:27,772 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction commit >2013-11-13 12:28:27,777 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction commit. >2013-11-13 12:28:27,777 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Closing up >2013-11-13 12:28:27,781 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Closing up. >2013-11-13 12:28:28,126 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Starting vdsm >2013-11-13 12:28:28,130 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Starting vdsm. >2013-11-13 12:28:29,699 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Pre-termination >2013-11-13 12:28:29,728 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Pre-termination. >2013-11-13 12:28:29,736 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113122829-monique-vds05.tlv.redhat.com-3c6c9922.log' >2013-11-13 12:28:29,741 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113122829-monique-vds05.tlv.redhat.com-3c6c9922.log'. >2013-11-13 12:28:29,967 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Termination >2013-11-13 12:28:29,992 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3c6c9922, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Termination. >2013-11-13 12:28:30,091 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-48) [3c6c9922] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 12:28:30,091 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-48) [3c6c9922] Command PollVDS execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 12:28:30,622 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-5-thread-48) [3c6c9922] Engine managed to communicate with VDSM agent on host monique05 >2013-11-13 12:28:30,864 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [3c6c9922] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, status=Initializing, nonOperationalReason=NONE), log id: 7294c519 >2013-11-13 12:28:30,866 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [3c6c9922] FINISH, SetVdsStatusVDSCommand, log id: 7294c519 >2013-11-13 12:28:30,867 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [3c6c9922] After Installation host monique05, VDS >2013-11-13 12:28:30,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [3c6c9922] Correlation ID: 3c6c9922, Job ID: f312a962-4295-4b98-9a90-d5a80b47569e, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installed >2013-11-13 12:28:30,874 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [3c6c9922] Lock freed to object EngineLock [exclusiveLocks= key: 66d9cb34-58a7-4693-91b3-aaeedf863e84 value: VDS >, sharedLocks= ] >2013-11-13 12:28:33,526 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-71) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vds=Host[monique05]), log id: 323806a5 >2013-11-13 12:28:33,554 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-71) FINISH, GetHardwareInfoVDSCommand, log id: 323806a5 >2013-11-13 12:28:33,623 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-71) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vds=Host[monique05]), log id: 24d3d491 >2013-11-13 12:28:33,629 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-71) FINISH, GetHardwareInfoVDSCommand, log id: 24d3d491 >2013-11-13 12:28:33,703 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 12:28:33,709 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-71) [749ef120] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:28:33,710 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-71) [af7b6a8] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:28:33,767 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-71) [af7b6a8] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2a1f643a >2013-11-13 12:28:33,873 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-71) [af7b6a8] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 2a1f643a >2013-11-13 12:28:33,873 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-71) [af7b6a8] Host monique05 storage connection was succeeded >2013-11-13 12:28:33,877 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-49) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 3c60e72e >2013-11-13 12:28:37,231 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-49) FINISH, ConnectStoragePoolVDSCommand, log id: 3c60e72e >2013-11-13 12:28:37,262 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-71) [af7b6a8] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84), log id: 1dd77e15 >2013-11-13 12:28:37,311 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-71) [af7b6a8] FINISH, SetMOMPolicyParametersVDSCommand, log id: 1dd77e15 >2013-11-13 12:28:37,364 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) [af7b6a8] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:28:37,419 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-71) [1f8b8005] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:28:37,471 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-71) [7bcc5e02] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:32:23,962 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-13) Running command: LoginUserCommand internal: false. >2013-11-13 12:32:23,969 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:35:31,491 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (pool-5-thread-49) [711e2b7f] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:35:31,493 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [711e2b7f] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 276c873e >2013-11-13 12:35:31,496 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [711e2b7f] FINISH, SetVdsStatusVDSCommand, log id: 276c873e >2013-11-13 12:35:31,511 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (pool-5-thread-49) [711e2b7f] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:35:31,518 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [711e2b7f] Correlation ID: 711e2b7f, Job ID: f4ce675a-49bf-45da-bffb-01be33ac4772, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was switched to Maintenance mode by admin@internal. >2013-11-13 12:35:32,460 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 66d9cb34-58a7-4693-91b3-aaeedf863e84 : monique05 >2013-11-13 12:35:32,466 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-48) Clearing cache of pool: a44a18ab-6f2e-48d4-ab60-889845f498d3 for problematic entities of VDS: monique05. >2013-11-13 12:35:32,468 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-39) START, DisconnectStoragePoolVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2), log id: 38d57024 >2013-11-13 12:35:34,479 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, DisconnectStoragePoolVDSCommand, log id: 38d57024 >2013-11-13 12:35:34,479 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-39) [582a2bdd] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:35:34,558 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-39) [582a2bdd] START, DisconnectStorageServerVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5560668f >2013-11-13 12:35:36,693 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-39) [582a2bdd] FINISH, DisconnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 5560668f >2013-11-13 12:35:44,762 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-6) [68278832] Lock Acquired to object EngineLock [exclusiveLocks= key: 66d9cb34-58a7-4693-91b3-aaeedf863e84 value: VDS >, sharedLocks= ] >2013-11-13 12:35:44,788 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-49) [68278832] Running command: ActivateVdsCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:35:44,788 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-49) [68278832] Before acquiring lock in order to prevent monitoring for host monique05 from data-center new-nfs >2013-11-13 12:35:44,788 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-49) [68278832] Lock acquired, from now a monitoring of host will be skipped for host monique05 from data-center new-nfs >2013-11-13 12:35:44,795 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [68278832] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, status=Unassigned, nonOperationalReason=NONE), log id: 306e2503 >2013-11-13 12:35:44,798 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [68278832] FINISH, SetVdsStatusVDSCommand, log id: 306e2503 >2013-11-13 12:35:44,800 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-5-thread-49) [68278832] START, ActivateVdsVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84), log id: 11e5a310 >2013-11-13 12:35:44,848 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-5-thread-49) [68278832] START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vds=Host[monique05]), log id: 5abb7b39 >2013-11-13 12:35:44,859 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-5-thread-49) [68278832] FINISH, GetHardwareInfoVDSCommand, log id: 5abb7b39 >2013-11-13 12:35:44,884 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (pool-5-thread-49) [6d8b7702] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:35:44,885 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (pool-5-thread-49) [4d1dc1e0] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:35:44,886 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-5-thread-49) [4d1dc1e0] FINISH, ActivateVdsVDSCommand, return: Host[monique05], log id: 11e5a310 >2013-11-13 12:35:46,030 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-49) [4d1dc1e0] Activate finished. Lock released. Monitoring can run now for host monique05 from data-center new-nfs >2013-11-13 12:35:46,051 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [4d1dc1e0] Correlation ID: 68278832, Job ID: 5838716d-947a-4f8b-9db4-2c45bea3515e, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was activated by admin@internal. >2013-11-13 12:35:46,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-49) [4d1dc1e0] Lock freed to object EngineLock [exclusiveLocks= key: 66d9cb34-58a7-4693-91b3-aaeedf863e84 value: VDS >, sharedLocks= ] >2013-11-13 12:35:48,799 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vds=Host[monique05]), log id: 6655d6d5 >2013-11-13 12:35:48,811 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GetHardwareInfoVDSCommand, log id: 6655d6d5 >2013-11-13 12:35:48,823 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-56) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 12:35:48,829 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-56) [395c7a97] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:35:48,831 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-56) [23ad04c7] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 12:35:49,783 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-56) [23ad04c7] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 563312b3 >2013-11-13 12:35:49,842 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-56) [23ad04c7] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 563312b3 >2013-11-13 12:35:49,842 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-56) [23ad04c7] Host monique05 storage connection was succeeded >2013-11-13 12:35:49,860 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 7c230d0f >2013-11-13 12:35:52,074 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) FINISH, ConnectStoragePoolVDSCommand, log id: 7c230d0f >2013-11-13 12:35:52,087 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-56) [23ad04c7] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84), log id: 3c31d6ac >2013-11-13 12:35:52,135 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-56) [23ad04c7] FINISH, SetMOMPolicyParametersVDSCommand, log id: 3c31d6ac >2013-11-13 12:35:52,138 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-56) [23ad04c7] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:35:52,142 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-56) [7f7c1e23] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:35:52,145 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-56) [581f320f] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:37:30,982 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:30,994 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:30,999 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [47e53b8c] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:31,020 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [47e53b8c] Correlation ID: 47e53b8c, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:37:31,075 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-13) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:31,088 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:31,091 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-13) [5eb87d7] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:31,095 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) [5eb87d7] Correlation ID: 5eb87d7, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:37:31,971 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:31,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:32,008 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [510edd0f] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:32,018 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [510edd0f] Correlation ID: 510edd0f, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:37:32,115 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:32,127 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:32,139 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [2b37cab8] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:32,142 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [2b37cab8] Correlation ID: 2b37cab8, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:37:32,996 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:33,011 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:33,018 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [578baa8c] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:33,029 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [578baa8c] Correlation ID: 578baa8c, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:37:33,217 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:33,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:33,233 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [30fe176] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:33,243 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [30fe176] Correlation ID: 30fe176, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:37:33,300 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:33,311 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:33,316 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [71e3b6a0] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:33,322 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [71e3b6a0] Correlation ID: 71e3b6a0, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:37:33,387 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:33,398 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:33,414 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [61e8525b] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:33,438 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [61e8525b] Correlation ID: 61e8525b, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:37:37,689 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:37,712 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:38,075 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-12) [370f2015] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:38,079 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [370f2015] Correlation ID: 370f2015, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:37:38,203 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:38,216 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:38,555 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [6cdcf454] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:38,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [6cdcf454] Correlation ID: 6cdcf454, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:37:46,091 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-11-13 12:37:46,102 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 12:37:46,134 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [79e0a1dc] Running command: LogoutUserCommand internal: false. >2013-11-13 12:37:46,138 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [79e0a1dc] Correlation ID: 79e0a1dc, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 12:43:52,218 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-8) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:43:52,218 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-8) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:43:52,218 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-8) [467d01c1] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:43:52,266 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-8) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:43:52,266 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-8) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:43:52,911 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [467d01c1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:43:52,917 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [467d01c1] Correlation ID: 467d01c1, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 12:45:14,399 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:45:14,399 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:45:14,399 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-4) [43928e0c] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:45:14,414 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:45:14,414 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:45:14,738 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [43928e0c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:45:14,759 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [43928e0c] Correlation ID: 43928e0c, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 12:46:03,681 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-11) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:46:03,681 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-11) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:46:03,681 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-11) [527471d2] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:46:03,704 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-11) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:46:03,704 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-11) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:46:04,003 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [527471d2] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:46:04,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [527471d2] Correlation ID: 527471d2, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 12:46:26,724 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-12) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:46:26,724 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-12) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:46:26,724 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-12) [d76acaf] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:46:26,792 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-12) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:46:26,792 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-12) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:46:27,172 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [d76acaf] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:46:27,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [d76acaf] Correlation ID: d76acaf, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 12:47:05,055 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:47:05,055 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:47:05,055 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-4) [3e9194ea] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:47:05,060 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:47:05,060 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:47:05,530 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3e9194ea] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:47:05,535 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3e9194ea] Correlation ID: 3e9194ea, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 12:47:19,228 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2013-11-13 12:47:19,230 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 12:47:19,239 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 12:47:19,239 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 12:47:19,242 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 12:47:19,242 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 12:47:19,243 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 12:48:05,359 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-12) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:48:05,359 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-12) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:48:05,359 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-12) [234d0ab1] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 12:48:05,385 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-12) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:48:05,385 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-12) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 12:48:05,772 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [234d0ab1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 12:48:05,777 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [234d0ab1] Correlation ID: 234d0ab1, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 12:51:08,465 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-14) Start refreshing all users data >2013-11-13 12:51:08,502 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-14) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 13:38:49,600 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 13:38:49,600 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 13:38:49,600 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-13) [6782968b] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 13:38:49,615 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 13:38:49,615 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 13:38:49,972 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) [6782968b] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 13:38:49,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) [6782968b] Correlation ID: 6782968b, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 13:39:08,031 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-13) [25076281] Lock Acquired to object EngineLock [exclusiveLocks= key: bd30025e-6b7c-415d-af8a-3cd52ac976f4 value: VM >, sharedLocks= ] >2013-11-13 13:39:08,611 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-13) [25076281] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_DISK_WITHOUT_DISK >2013-11-13 13:39:08,612 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-13) [25076281] Lock freed to object EngineLock [exclusiveLocks= key: bd30025e-6b7c-415d-af8a-3cd52ac976f4 value: VM >, sharedLocks= ] >2013-11-13 13:39:22,853 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-5) [512c8113] Lock Acquired to object EngineLock [exclusiveLocks= key: bd30025e-6b7c-415d-af8a-3cd52ac976f4 value: VM_DISK_BOOT >, sharedLocks= key: bd30025e-6b7c-415d-af8a-3cd52ac976f4 value: VM >] >2013-11-13 13:39:22,971 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-5) [512c8113] Running command: AddDiskCommand internal: false. Entities affected : ID: bd30025e-6b7c-415d-af8a-3cd52ac976f4 Type: VM, ID: 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53 Type: Storage >2013-11-13 13:39:23,536 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53 Type: Storage >2013-11-13 13:39:23,777 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] Lock freed to object EngineLock [exclusiveLocks= key: bd30025e-6b7c-415d-af8a-3cd52ac976f4 value: VM_DISK_BOOT >, sharedLocks= key: bd30025e-6b7c-415d-af8a-3cd52ac976f4 value: VM >] >2013-11-13 13:39:23,811 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] START, CreateImageVDSCommand( storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, ignoreFailoverLimit = false, storageDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, imageGroupId = 6ea35e90-8019-4a49-83fe-b9bc72698933, imageSizeInBytes = 12884901888, volumeFormat = RAW, newImageId = ea620010-d5e0-4042-81f7-d1e4e061f968, newImageDescription = ), log id: 59f0175d >2013-11-13 13:39:23,811 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2013-11-13 13:39:23,811 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] -- createVolume parameters: > sdUUID=05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53 > spUUID=a44a18ab-6f2e-48d4-ab60-889845f498d3 > imgGUID=6ea35e90-8019-4a49-83fe-b9bc72698933 > size=12,884,901,888 bytes > volFormat=RAW > volType=Sparse > volUUID=ea620010-d5e0-4042-81f7-d1e4e061f968 > descr= > srcImgGUID=00000000-0000-0000-0000-000000000000 > srcVolUUID=00000000-0000-0000-0000-000000000000 > >2013-11-13 13:39:23,872 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] FINISH, CreateImageVDSCommand, return: ea620010-d5e0-4042-81f7-d1e4e061f968, log id: 59f0175d >2013-11-13 13:39:24,526 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 47620d96-f671-4047-a2b6-bd8d6d3ae503 >2013-11-13 13:39:24,527 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] CommandMultiAsyncTasks::AttachTask: Attaching task 13afbd74-df22-4ff2-83e6-c9b6af63d603 to command 47620d96-f671-4047-a2b6-bd8d6d3ae503. >2013-11-13 13:39:24,599 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] Adding task 13afbd74-df22-4ff2-83e6-c9b6af63d603 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2013-11-13 13:39:24,862 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] Correlation ID: 512c8113, Job ID: 2cc08f31-3444-44de-9ab7-19a4d5a831e5, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of dssdg_Disk1 was initiated on VM dssdg by admin@internal. >2013-11-13 13:39:24,863 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (ajp-/127.0.0.1:8702-5) [5f8ff4e9] BaseAsyncTask::StartPollingTask: Starting to poll task 13afbd74-df22-4ff2-83e6-c9b6af63d603. >2013-11-13 13:39:25,512 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-43) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2013-11-13 13:39:25,545 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-43) SPMAsyncTask::PollTask: Polling task 13afbd74-df22-4ff2-83e6-c9b6af63d603 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2013-11-13 13:39:25,551 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-43) BaseAsyncTask::OnTaskEndSuccess: Task 13afbd74-df22-4ff2-83e6-c9b6af63d603 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2013-11-13 13:39:25,551 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (DefaultQuartzScheduler_Worker-43) CommandAsyncTask::EndActionIfNecessary: All tasks of command 47620d96-f671-4047-a2b6-bd8d6d3ae503 has ended -> executing EndAction >2013-11-13 13:39:25,552 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (DefaultQuartzScheduler_Worker-43) CommandAsyncTask::EndAction: Ending action for 1 tasks (command ID: 47620d96-f671-4047-a2b6-bd8d6d3ae503): calling EndAction . >2013-11-13 13:39:25,578 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-5-thread-48) CommandAsyncTask::EndCommandAction [within thread] context: Attempting to EndAction AddDisk, executionIndex: 0 >2013-11-13 13:39:25,648 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (pool-5-thread-48) [512c8113] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2013-11-13 13:39:25,764 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (pool-5-thread-48) [5f8ff4e9] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2013-11-13 13:39:25,826 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (pool-5-thread-48) [5f8ff4e9] START, GetImageInfoVDSCommand( storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, ignoreFailoverLimit = false, storageDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, imageGroupId = 6ea35e90-8019-4a49-83fe-b9bc72698933, imageId = ea620010-d5e0-4042-81f7-d1e4e061f968), log id: 78f5d6ad >2013-11-13 13:39:25,878 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (pool-5-thread-48) [5f8ff4e9] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@c2488338, log id: 78f5d6ad >2013-11-13 13:39:25,933 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) Correlation ID: 512c8113, Job ID: 2cc08f31-3444-44de-9ab7-19a4d5a831e5, Call Stack: null, Custom Event ID: -1, Message: The disk dssdg_Disk1 was successfully added to VM dssdg. >2013-11-13 13:39:25,934 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-5-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: EndAction for action type AddDisk completed, handling the result. >2013-11-13 13:39:25,934 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-5-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: EndAction for action type AddDisk succeeded, clearing tasks. >2013-11-13 13:39:25,949 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-5-thread-48) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 13afbd74-df22-4ff2-83e6-c9b6af63d603 >2013-11-13 13:39:25,954 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (pool-5-thread-48) START, SPMClearTaskVDSCommand( storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, ignoreFailoverLimit = false, taskId = 13afbd74-df22-4ff2-83e6-c9b6af63d603), log id: 4497bbe4 >2013-11-13 13:39:26,078 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (pool-5-thread-48) START, HSMClearTaskVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, taskId=13afbd74-df22-4ff2-83e6-c9b6af63d603), log id: 45b098ea >2013-11-13 13:39:26,092 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (pool-5-thread-48) FINISH, HSMClearTaskVDSCommand, log id: 45b098ea >2013-11-13 13:39:26,092 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (pool-5-thread-48) FINISH, SPMClearTaskVDSCommand, log id: 4497bbe4 >2013-11-13 13:39:26,100 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-5-thread-48) BaseAsyncTask::RemoveTaskFromDB: Removed task 13afbd74-df22-4ff2-83e6-c9b6af63d603 from DataBase >2013-11-13 13:39:26,100 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-5-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 47620d96-f671-4047-a2b6-bd8d6d3ae503 >2013-11-13 13:39:43,726 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-5) [6ef54352] Lock Acquired to object EngineLock [exclusiveLocks= key: bd30025e-6b7c-415d-af8a-3cd52ac976f4 value: VM >, sharedLocks= ] >2013-11-13 13:39:43,793 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-5) [6ef54352] START, IsVmDuringInitiatingVDSCommand( vmId = bd30025e-6b7c-415d-af8a-3cd52ac976f4), log id: 2018034d >2013-11-13 13:39:43,793 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-5) [6ef54352] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 2018034d >2013-11-13 13:39:44,104 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-5-thread-48) [6ef54352] Running command: RunVmCommand internal: false. Entities affected : ID: bd30025e-6b7c-415d-af8a-3cd52ac976f4 Type: VM >2013-11-13 13:39:44,717 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-5-thread-48) [6ef54352] START, IsoPrefixVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, storagePoolId=a44a18ab-6f2e-48d4-ab60-889845f498d3), log id: dce12b7 >2013-11-13 13:39:44,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-5-thread-48) [6ef54352] FINISH, IsoPrefixVDSCommand, return: , log id: dce12b7 >2013-11-13 13:39:44,732 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-5-thread-48) [6ef54352] START, CreateVmVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vmId=bd30025e-6b7c-415d-af8a-3cd52ac976f4, vm=VM [dssdg]), log id: 6572c0d9 >2013-11-13 13:39:44,793 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-5-thread-48) [6ef54352] START, CreateVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vmId=bd30025e-6b7c-415d-af8a-3cd52ac976f4, vm=VM [dssdg]), log id: 34e3eb22 >2013-11-13 13:39:45,163 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-5-thread-48) [6ef54352] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=1024,kvmEnable=true,smp=1,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=1024,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=0,transparentHugePages=true,vmId=bd30025e-6b7c-415d-af8a-3cd52ac976f4,devices=[Ljava.util.HashMap;@258aafd8,acpiEnable=true,vmName=dssdg,cpuType=Conroe,custom={} >2013-11-13 13:39:45,163 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-5-thread-48) [6ef54352] FINISH, CreateVDSCommand, log id: 34e3eb22 >2013-11-13 13:39:45,271 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-5-thread-48) [6ef54352] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 6572c0d9 >2013-11-13 13:39:45,272 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-5-thread-48) [6ef54352] Lock freed to object EngineLock [exclusiveLocks= key: bd30025e-6b7c-415d-af8a-3cd52ac976f4 value: VM >, sharedLocks= ] >2013-11-13 13:39:45,317 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [6ef54352] Correlation ID: 6ef54352, Job ID: 4d647c0f-b892-462b-b28b-9a2e6813f04f, Call Stack: null, Custom Event ID: -1, Message: VM dssdg was started by admin@internal (Host: monique05). >2013-11-13 13:39:46,048 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) VM dssdg bd30025e-6b7c-415d-af8a-3cd52ac976f4 moved from WaitForLaunch --> PoweringUp >2013-11-13 13:39:46,057 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-49) START, FullListVdsCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vds=Host[monique05], vmIds=[bd30025e-6b7c-415d-af8a-3cd52ac976f4]), log id: c35e001 >2013-11-13 13:39:46,086 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-49) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@5e946a3, log id: c35e001 >2013-11-13 13:40:47,093 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) VM dssdg bd30025e-6b7c-415d-af8a-3cd52ac976f4 moved from PoweringUp --> Up >2013-11-13 13:40:47,113 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) Correlation ID: 6ef54352, Job ID: 4d647c0f-b892-462b-b28b-9a2e6813f04f, Call Stack: null, Custom Event ID: -1, Message: VM dssdg started on Host monique05 >2013-11-13 13:40:48,685 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-23) Setting new tasks map. The map contains now 0 tasks >2013-11-13 13:40:48,685 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-23) Cleared all tasks of pool a44a18ab-6f2e-48d4-ab60-889845f498d3. >2013-11-13 13:41:36,903 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (pool-5-thread-48) [3dff5af3] Running command: ShutdownVmCommand internal: false. Entities affected : ID: bd30025e-6b7c-415d-af8a-3cd52ac976f4 Type: VM >2013-11-13 13:41:36,906 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (pool-5-thread-48) [3dff5af3] Entered (VM dssdg). >2013-11-13 13:41:36,906 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (pool-5-thread-48) [3dff5af3] Sending shutdown command for VM dssdg. >2013-11-13 13:41:36,919 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-5-thread-48) [3dff5af3] START, DestroyVmVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vmId=bd30025e-6b7c-415d-af8a-3cd52ac976f4, force=false, secondsToWait=30, gracefully=true), log id: 5ef14d8c >2013-11-13 13:41:36,964 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-5-thread-48) [3dff5af3] START, DestroyVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vmId=bd30025e-6b7c-415d-af8a-3cd52ac976f4, force=false, secondsToWait=30, gracefully=true), log id: 448aa0c8 >2013-11-13 13:41:36,970 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-5-thread-48) [3dff5af3] FINISH, DestroyVDSCommand, log id: 448aa0c8 >2013-11-13 13:41:37,014 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-5-thread-48) [3dff5af3] FINISH, DestroyVmVDSCommand, return: PoweringDown, log id: 5ef14d8c >2013-11-13 13:41:37,068 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [3dff5af3] Correlation ID: 3dff5af3, Job ID: 7fe26d27-b332-4dea-af6d-735b389a23ca, Call Stack: null, Custom Event ID: -1, Message: VM shutdown initiated by admin@internal on VM dssdg (Host: monique05). >2013-11-13 13:41:41,443 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (pool-5-thread-48) [34d351ab] Running command: ShutdownVmCommand internal: false. Entities affected : ID: bd30025e-6b7c-415d-af8a-3cd52ac976f4 Type: VM >2013-11-13 13:41:41,444 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (pool-5-thread-48) [34d351ab] Entered (VM dssdg). >2013-11-13 13:41:41,444 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (pool-5-thread-48) [34d351ab] Cannot shutdown VM dssdg, status is not up. Stopping instead. >2013-11-13 13:41:41,503 INFO [org.ovirt.engine.core.bll.StopVmCommand] (pool-5-thread-48) [7979bf97] Running command: StopVmCommand internal: true. Entities affected : ID: bd30025e-6b7c-415d-af8a-3cd52ac976f4 Type: VM >2013-11-13 13:41:41,506 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-5-thread-48) [7979bf97] START, DestroyVmVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vmId=bd30025e-6b7c-415d-af8a-3cd52ac976f4, force=false, secondsToWait=0, gracefully=false), log id: 1602d645 >2013-11-13 13:41:41,563 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-5-thread-48) [7979bf97] START, DestroyVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, vmId=bd30025e-6b7c-415d-af8a-3cd52ac976f4, force=false, secondsToWait=0, gracefully=false), log id: 15fa5e55 >2013-11-13 13:41:41,927 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-5-thread-48) [7979bf97] FINISH, DestroyVDSCommand, log id: 15fa5e55 >2013-11-13 13:41:41,979 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-5-thread-48) [7979bf97] FINISH, DestroyVmVDSCommand, return: Down, log id: 1602d645 >2013-11-13 13:41:41,996 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [7979bf97] Correlation ID: 7979bf97, Call Stack: null, Custom Event ID: -1, Message: VM dssdg was powered off ungracefully by admin@internal (Host: monique05). >2013-11-13 13:43:28,137 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 13:43:28,137 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 13:43:28,138 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-13) [6b4b012b] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 13:43:28,148 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 13:43:28,148 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-13) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2013-11-13 13:43:28,589 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) [6b4b012b] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 13:43:28,594 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) [6b4b012b] Correlation ID: 6b4b012b, Call Stack: null, Custom Event ID: -1, Message: Host monique05 configuration was updated by admin@internal. >2013-11-13 13:47:19,243 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Attempting to update VMs/Templates Ovf. >2013-11-13 13:47:19,244 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 13:47:19,771 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UpdateVMVDSCommand] (DefaultQuartzScheduler_Worker-56) [581f320f] START, UpdateVMVDSCommand( storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, ignoreFailoverLimit = false, storageDomainId = 00000000-0000-0000-0000-000000000000, infoDictionary.size = 1), log id: 2e39d125 >2013-11-13 13:47:19,817 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UpdateVMVDSCommand] (DefaultQuartzScheduler_Worker-56) [581f320f] FINISH, UpdateVMVDSCommand, log id: 2e39d125 >2013-11-13 13:47:19,850 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 13:47:19,850 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Attempting to update template OVFs in Data Center new-nfs >2013-11-13 13:47:19,858 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 13:47:19,858 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 13:47:19,859 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 13:51:08,570 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-81) [55890714] Start refreshing all users data >2013-11-13 13:51:08,607 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-81) [55890714] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 14:07:13,820 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (pool-5-thread-48) [2d1bd968] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 14:07:13,822 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [2d1bd968] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 38c8b9a4 >2013-11-13 14:07:13,825 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [2d1bd968] FINISH, SetVdsStatusVDSCommand, log id: 38c8b9a4 >2013-11-13 14:07:13,916 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (pool-5-thread-48) [2d1bd968] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 14:07:13,942 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [2d1bd968] Correlation ID: 2d1bd968, Job ID: a3bfa0d8-1cfc-498c-80b2-d59e4055c640, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was switched to Maintenance mode by admin@internal. >2013-11-13 14:07:14,462 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 66d9cb34-58a7-4693-91b3-aaeedf863e84 : monique05 >2013-11-13 14:07:14,468 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) Clearing cache of pool: a44a18ab-6f2e-48d4-ab60-889845f498d3 for problematic entities of VDS: monique05. >2013-11-13 14:07:14,470 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-78) START, DisconnectStoragePoolVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2), log id: 961002 >2013-11-13 14:07:16,480 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, DisconnectStoragePoolVDSCommand, log id: 961002 >2013-11-13 14:07:16,480 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-78) [3fb675ce] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 14:07:16,559 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-78) [3fb675ce] START, DisconnectStorageServerVDSCommand(HostName = monique05, HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1b50fb90 >2013-11-13 14:07:18,755 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-78) [3fb675ce] FINISH, DisconnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 1b50fb90 >2013-11-13 14:07:30,632 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-7) [402f9bb7] Lock Acquired to object EngineLock [exclusiveLocks= key: 66d9cb34-58a7-4693-91b3-aaeedf863e84 value: VDS >, sharedLocks= ] >2013-11-13 14:07:30,667 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (pool-5-thread-48) [402f9bb7] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 66d9cb34-58a7-4693-91b3-aaeedf863e84 Type: VDS >2013-11-13 14:07:31,006 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (pool-5-thread-48) [402f9bb7] START, RemoveVdsVDSCommand( HostId = 66d9cb34-58a7-4693-91b3-aaeedf863e84), log id: 61b2afe7 >2013-11-13 14:07:31,006 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (pool-5-thread-48) [402f9bb7] vdsManager::disposing >2013-11-13 14:07:31,006 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (pool-5-thread-48) [402f9bb7] FINISH, RemoveVdsVDSCommand, log id: 61b2afe7 >2013-11-13 14:07:31,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [402f9bb7] Correlation ID: 402f9bb7, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was removed by admin@internal. >2013-11-13 14:07:31,008 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (pool-5-thread-48) [402f9bb7] Lock freed to object EngineLock [exclusiveLocks= key: 66d9cb34-58a7-4693-91b3-aaeedf863e84 value: VDS >, sharedLocks= ] >2013-11-13 14:07:53,391 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-11-13 14:07:53,417 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 14:08:05,809 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-11-13 14:08:05,831 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 14:08:07,504 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-10) [1685b19b] Running command: AddVdsCommand internal: false. Entities affected : ID: b273ff0a-4519-4d34-a9a9-a8f1a3506fca Type: VdsGroups >2013-11-13 14:08:07,538 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-10) [9474ca1] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:08:07,543 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-10) [9474ca1] Lock freed to object EngineLock [exclusiveLocks= key: a44a18ab-6f2e-48d4-ab60-889845f498d3 value: REGISTER_VDS >, sharedLocks= ] >2013-11-13 14:08:07,543 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-10) [9474ca1] Cannot get vdsManager for vdsid=d80ddaf1-0131-4088-94cf-d3ad317c10df >2013-11-13 14:08:07,544 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-10) [9474ca1] START, RemoveVdsVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df), log id: 691409f5 >2013-11-13 14:08:07,545 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-10) [9474ca1] Cannot get vdsManager for vdsid=d80ddaf1-0131-4088-94cf-d3ad317c10df >2013-11-13 14:08:07,545 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-10) [9474ca1] FINISH, RemoveVdsVDSCommand, log id: 691409f5 >2013-11-13 14:08:07,545 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-10) [9474ca1] Cannot get vdsManager for vdsid=d80ddaf1-0131-4088-94cf-d3ad317c10df >2013-11-13 14:08:07,546 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-10) [9474ca1] START, AddVdsVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df), log id: 6a2bd307 >2013-11-13 14:08:07,546 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-10) [9474ca1] AddVds - entered , starting logic to add VDS d80ddaf1-0131-4088-94cf-d3ad317c10df >2013-11-13 14:08:07,547 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-10) [9474ca1] AddVds - VDS d80ddaf1-0131-4088-94cf-d3ad317c10df was added, will try to add it to the resource manager >2013-11-13 14:08:07,547 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-10) [9474ca1] Entered VdsManager constructor >2013-11-13 14:08:07,710 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-10) [9474ca1] Initialize vdsBroker (monique-vds05.tlv.redhat.com,54,321) >2013-11-13 14:08:07,711 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-10) [9474ca1] VDS d80ddaf1-0131-4088-94cf-d3ad317c10df was added to the Resource Manager >2013-11-13 14:08:07,711 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-10) [9474ca1] FINISH, AddVdsVDSCommand, log id: 6a2bd307 >2013-11-13 14:08:07,715 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [9474ca1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 14:08:07,798 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [5407ff18] Lock Acquired to object EngineLock [exclusiveLocks= key: d80ddaf1-0131-4088-94cf-d3ad317c10df value: VDS >, sharedLocks= ] >2013-11-13 14:08:07,805 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [5407ff18] Running command: InstallVdsCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:08:07,805 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [5407ff18] Before Installation host d80ddaf1-0131-4088-94cf-d3ad317c10df, monique05 >2013-11-13 14:08:07,899 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [9474ca1] Correlation ID: 1685b19b, Job ID: f07a7edf-2280-47a7-9911-1b1b3885f03c, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was added by admin@internal. >2013-11-13 14:08:07,951 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [4b0e77d6] Running command: LogoutUserCommand internal: false. >2013-11-13 14:08:07,976 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [4b0e77d6] Correlation ID: 4b0e77d6, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 14:08:08,027 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [5407ff18] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, status=Installing, nonOperationalReason=NONE), log id: 3963c081 >2013-11-13 14:08:08,029 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [5407ff18] FINISH, SetVdsStatusVDSCommand, log id: 3963c081 >2013-11-13 14:08:08,092 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-5-thread-48) [5407ff18] Installation monique-vds05.tlv.redhat.com: Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17 >2013-11-13 14:08:08,096 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [5407ff18] Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Connected to host monique-vds05.tlv.redhat.com with SSH key fingerprint: c4:57:be:ac:54:86:71:87:43:7c:38:b8:41:78:9a:17. >2013-11-13 14:08:08,111 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-5-thread-48) [5407ff18] Installation of monique-vds05.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-11-13 14:08:08,263 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-5-thread-48) SSH execute root@monique-vds05.tlv.redhat.com 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-11-13 14:08:08,968 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Initializing >2013-11-13 14:08:08,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Initializing. >2013-11-13 14:08:09,023 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment setup >2013-11-13 14:08:09,053 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment setup. >2013-11-13 14:08:09,053 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment packages setup >2013-11-13 14:08:09,070 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment packages setup. >2013-11-13 14:08:09,070 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Programs detection >2013-11-13 14:08:09,074 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Programs detection. >2013-11-13 14:08:09,074 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Environment customization >2013-11-13 14:08:09,101 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Environment customization. >2013-11-13 14:08:09,108 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20131113140813.log' >2013-11-13 14:08:09,112 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Logs at host located at: '/tmp/ovirt-host-deploy-20131113140813.log'. >2013-11-13 14:08:09,261 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host monique-vds05.tlv.redhat.com reports unique id 4C4C4544-0032-4310-8051-B7C04F39354A >2013-11-13 14:08:09,287 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0032-4310-8051-B7C04F39354A to Host monique-vds05.tlv.redhat.com >2013-11-13 14:08:09,330 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Setup validation >2013-11-13 14:08:09,335 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Setup validation. >2013-11-13 14:08:09,372 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Hardware supports virtualization >2013-11-13 14:08:09,376 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Hardware supports virtualization. >2013-11-13 14:08:09,377 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction setup >2013-11-13 14:08:09,381 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction setup. >2013-11-13 14:08:09,381 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 14:08:09,385 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 14:08:09,385 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Package installation >2013-11-13 14:08:09,389 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Package installation. >2013-11-13 14:08:10,713 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Initializing Host: monique05 >2013-11-13 14:08:12,009 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Misc configuration >2013-11-13 14:08:12,030 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Misc configuration. >2013-11-13 14:08:12,760 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Enrolling certificate >2013-11-13 14:08:12,766 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Enrolling certificate. >2013-11-13 14:08:14,064 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Transaction commit >2013-11-13 14:08:14,100 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Transaction commit. >2013-11-13 14:08:14,101 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Closing up >2013-11-13 14:08:14,125 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Closing up. >2013-11-13 14:08:14,411 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Starting vdsm >2013-11-13 14:08:14,434 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Starting vdsm. >2013-11-13 14:08:16,132 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Pre-termination >2013-11-13 14:08:16,137 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Pre-termination. >2013-11-13 14:08:16,155 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113140816-monique-vds05.tlv.redhat.com-5407ff18.log' >2013-11-13 14:08:16,159 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131113140816-monique-vds05.tlv.redhat.com-5407ff18.log'. >2013-11-13 14:08:16,341 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation monique-vds05.tlv.redhat.com: Stage: Termination >2013-11-13 14:08:16,345 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5407ff18, Call Stack: null, Custom Event ID: -1, Message: Installing Host monique05. Stage: Termination. >2013-11-13 14:08:16,472 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-48) [5407ff18] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 14:08:16,472 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-48) [5407ff18] Command PollVDS execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2013-11-13 14:08:16,997 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-48) [5407ff18] org.ovirt.engine.core.vdsbroker.vdsbroker.VDSRecoveringException: Recovering from crash or Initializing >2013-11-13 14:08:16,997 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (pool-5-thread-48) [5407ff18] Command PollVDS execution failed. Exception: VDSRecoveringException: Recovering from crash or Initializing >2013-11-13 14:08:17,504 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-5-thread-48) [5407ff18] Engine managed to communicate with VDSM agent on host monique05 >2013-11-13 14:08:17,618 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [5407ff18] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, status=Initializing, nonOperationalReason=NONE), log id: 40e05bde >2013-11-13 14:08:17,621 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [5407ff18] FINISH, SetVdsStatusVDSCommand, log id: 40e05bde >2013-11-13 14:08:17,621 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [5407ff18] After Installation host monique05, VDS >2013-11-13 14:08:17,636 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [5407ff18] Correlation ID: 5407ff18, Job ID: f07a7edf-2280-47a7-9911-1b1b3885f03c, Call Stack: null, Custom Event ID: -1, Message: Host monique05 installed >2013-11-13 14:08:17,645 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-5-thread-48) [5407ff18] Lock freed to object EngineLock [exclusiveLocks= key: d80ddaf1-0131-4088-94cf-d3ad317c10df value: VDS >, sharedLocks= ] >2013-11-13 14:08:19,759 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, vds=Host[monique05]), log id: 267b1f89 >2013-11-13 14:08:19,787 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GetHardwareInfoVDSCommand, log id: 267b1f89 >2013-11-13 14:08:19,856 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, vds=Host[monique05]), log id: 5c56b70e >2013-11-13 14:08:19,862 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GetHardwareInfoVDSCommand, log id: 5c56b70e >2013-11-13 14:08:19,884 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 14:08:19,889 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-86) [4ac07f67] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 14:08:19,891 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-86) [439de5d1] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 14:08:19,913 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-86) [439de5d1] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 8f72cbe >2013-11-13 14:08:20,033 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-86) [439de5d1] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 8f72cbe >2013-11-13 14:08:20,033 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-86) [439de5d1] Host monique05 storage connection was succeeded >2013-11-13 14:08:20,037 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-50) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 35f36316 >2013-11-13 14:08:22,686 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-50) FINISH, ConnectStoragePoolVDSCommand, log id: 35f36316 >2013-11-13 14:08:22,698 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-86) [439de5d1] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df), log id: 4dab8f0c >2013-11-13 14:08:22,747 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-86) [439de5d1] FINISH, SetMOMPolicyParametersVDSCommand, log id: 4dab8f0c >2013-11-13 14:08:22,751 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) [439de5d1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 14:08:22,755 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-86) [92a6ce1] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:08:22,772 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-86) [2b040ed5] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:09:19,849 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-11-13 14:09:19,865 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 14:09:20,017 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-4) [ded27fb] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:09:20,019 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-4) [ded27fb] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 4a4308b3 >2013-11-13 14:09:20,025 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-4) [ded27fb] FINISH, SetVdsStatusVDSCommand, log id: 4a4308b3 >2013-11-13 14:09:20,046 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (ajp-/127.0.0.1:8702-4) [ded27fb] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:09:20,051 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [ded27fb] Correlation ID: ded27fb, Job ID: 0d664b2f-ecf0-45ad-ae28-49c2ab69ba5f, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was switched to Maintenance mode by admin@internal. >2013-11-13 14:09:20,449 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = d80ddaf1-0131-4088-94cf-d3ad317c10df : monique05 >2013-11-13 14:09:20,457 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-48) Clearing cache of pool: a44a18ab-6f2e-48d4-ab60-889845f498d3 for problematic entities of VDS: monique05. >2013-11-13 14:09:20,460 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-24) START, DisconnectStoragePoolVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2), log id: 25517c08 >2013-11-13 14:09:20,830 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [a098e27] Running command: LogoutUserCommand internal: false. >2013-11-13 14:09:20,833 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [a098e27] Correlation ID: a098e27, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 14:09:22,471 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, DisconnectStoragePoolVDSCommand, log id: 25517c08 >2013-11-13 14:09:22,471 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-24) [177c3271] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 14:09:22,483 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-24) [177c3271] START, DisconnectStorageServerVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 59fb02ea >2013-11-13 14:09:24,620 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-24) [177c3271] FINISH, DisconnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 59fb02ea >2013-11-13 14:09:40,156 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-11-13 14:09:40,191 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 14:09:40,235 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [54c66b9f] Running command: LogoutUserCommand internal: false. >2013-11-13 14:09:40,272 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [54c66b9f] Correlation ID: 54c66b9f, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 14:11:05,718 INFO [org.ovirt.engine.api.restapi.security.auth.LoginValidator] (ajp-/127.0.0.1:8702-3) Validating session failed, reason: Session does not exist. >2013-11-13 14:12:19,833 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-11-13 14:12:19,843 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 14:14:05,344 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-6) [4da92d39] Lock Acquired to object EngineLock [exclusiveLocks= key: d80ddaf1-0131-4088-94cf-d3ad317c10df value: VDS >, sharedLocks= ] >2013-11-13 14:14:05,365 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-49) [4da92d39] Running command: ActivateVdsCommand internal: false. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:14:05,365 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-49) [4da92d39] Before acquiring lock in order to prevent monitoring for host monique05 from data-center new-nfs >2013-11-13 14:14:05,365 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-49) [4da92d39] Lock acquired, from now a monitoring of host will be skipped for host monique05 from data-center new-nfs >2013-11-13 14:14:05,375 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [4da92d39] START, SetVdsStatusVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, status=Unassigned, nonOperationalReason=NONE), log id: 6158d550 >2013-11-13 14:14:05,377 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-49) [4da92d39] FINISH, SetVdsStatusVDSCommand, log id: 6158d550 >2013-11-13 14:14:05,379 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-5-thread-49) [4da92d39] START, ActivateVdsVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df), log id: 5d4abc77 >2013-11-13 14:14:05,432 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-5-thread-49) [4da92d39] START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, vds=Host[monique05]), log id: 50e028f8 >2013-11-13 14:14:05,442 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-5-thread-49) [4da92d39] FINISH, GetHardwareInfoVDSCommand, log id: 50e028f8 >2013-11-13 14:14:05,474 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (pool-5-thread-49) [7867a428] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:14:05,476 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (pool-5-thread-49) [743412f1] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:14:05,477 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-5-thread-49) [743412f1] FINISH, ActivateVdsVDSCommand, return: Host[monique05], log id: 5d4abc77 >2013-11-13 14:14:05,745 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-49) [743412f1] Activate finished. Lock released. Monitoring can run now for host monique05 from data-center new-nfs >2013-11-13 14:14:05,749 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-49) [743412f1] Correlation ID: 4da92d39, Job ID: afd19a32-2986-430d-b6c3-2fabd9ad31a2, Call Stack: null, Custom Event ID: -1, Message: Host monique05 was activated by admin@internal. >2013-11-13 14:14:05,753 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-5-thread-49) [743412f1] Lock freed to object EngineLock [exclusiveLocks= key: d80ddaf1-0131-4088-94cf-d3ad317c10df value: VDS >, sharedLocks= ] >2013-11-13 14:14:06,893 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, vds=Host[monique05]), log id: 3284ae0e >2013-11-13 14:14:06,899 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GetHardwareInfoVDSCommand, log id: 3284ae0e >2013-11-13 14:14:06,911 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-13 14:14:06,917 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-48) [787b0267] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 14:14:06,918 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-48) [5ecd2846] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-13 14:14:06,926 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-48) [5ecd2846] START, ConnectStorageServerVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 3b448ac2 >2013-11-13 14:14:06,996 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-48) [5ecd2846] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 3b448ac2 >2013-11-13 14:14:06,996 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-48) [5ecd2846] Host monique05 storage connection was succeeded >2013-11-13 14:14:07,000 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) START, ConnectStoragePoolVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 2, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 91b013f >2013-11-13 14:14:09,246 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) FINISH, ConnectStoragePoolVDSCommand, log id: 91b013f >2013-11-13 14:14:09,261 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-48) [5ecd2846] START, SetMOMPolicyParametersVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df), log id: 1255515c >2013-11-13 14:14:09,312 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-48) [5ecd2846] FINISH, SetMOMPolicyParametersVDSCommand, log id: 1255515c >2013-11-13 14:14:09,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) [5ecd2846] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host monique05. >2013-11-13 14:14:09,321 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-48) [721f4423] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:14:09,324 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-48) [4e3a534d] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-13 14:18:04,750 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2013-11-13 14:18:04,768 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 14:18:20,250 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-11-13 14:18:20,257 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 14:18:33,865 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-11-13 14:18:33,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 14:18:47,672 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-11-13 14:18:47,680 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 14:19:02,375 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-11-13 14:19:02,408 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-13 14:19:02,415 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [18782a14] Running command: LogoutUserCommand internal: false. >2013-11-13 14:19:02,436 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [18782a14] Correlation ID: 18782a14, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 14:47:19,728 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [5f5566fa] Running command: LogoutUserCommand internal: false. >2013-11-13 14:47:19,736 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [5f5566fa] Correlation ID: 5f5566fa, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-13 14:47:19,859 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to update VMs/Templates Ovf. >2013-11-13 14:47:19,874 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 14:47:19,875 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 14:47:19,875 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 14:47:19,878 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 14:47:19,878 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 14:47:19,879 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 14:51:08,638 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Start refreshing all users data >2013-11-13 14:51:08,684 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 15:47:19,879 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2013-11-13 15:47:19,881 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 15:47:19,889 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 15:47:19,890 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 15:47:19,890 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 15:47:19,890 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 15:47:19,891 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 15:51:08,724 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) Start refreshing all users data >2013-11-13 15:51:08,745 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-1) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 16:47:19,892 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update VMs/Templates Ovf. >2013-11-13 16:47:19,893 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 16:47:19,902 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 16:47:19,902 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 16:47:19,905 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 16:47:19,905 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 16:47:19,907 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 16:51:08,775 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-22) Start refreshing all users data >2013-11-13 16:51:08,792 ERROR [org.ovirt.engine.core.bll.adbroker.LDAPTemplateWrapper] (DefaultQuartzScheduler_Worker-22) Error in running LDAP query. BaseDN is , filter is (|(objectGUID=9b9002d1-ec33-4083-8a7b-31f6b8931648)). Exception message is: Request: 4 cancelled; nested exception is javax.naming.CommunicationException: Request: 4 cancelled; remaining name '' >2013-11-13 16:51:08,792 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-22) Error in communicating with LDAP server Request: 4 cancelled; nested exception is javax.naming.CommunicationException: Request: 4 cancelled; remaining name '' >2013-11-13 16:51:08,793 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-22) Failed ldap search server LDAP://qa1.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to communication error. We should try the next server >2013-11-13 16:51:38,797 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-22) Error in communicating with LDAP server qa2-tlv.qa.lab.tlv.redhat.com:389; nested exception is javax.naming.CommunicationException: qa2-tlv.qa.lab.tlv.redhat.com:389 [Root exception is java.net.SocketTimeoutException: connect timed out] >2013-11-13 16:51:38,797 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-22) Failed ldap search server LDAP://qa2-tlv.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to connection timeout. We should try the next server >2013-11-13 16:51:38,797 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-22) User vdcadmin not found in directory server, its status switched to InActive >2013-11-13 17:47:19,908 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update VMs/Templates Ovf. >2013-11-13 17:47:19,909 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 17:47:19,949 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 17:47:19,950 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 17:47:19,953 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 17:47:19,953 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 17:47:19,953 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 17:51:38,801 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-60) [3b758810] Start refreshing all users data >2013-11-13 17:51:39,426 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-60) [3b758810] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 17:51:39,450 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-60) [3b758810] Inactive User vdcadmin found in directory server, its status switched to Active >2013-11-13 18:47:19,954 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Attempting to update VMs/Templates Ovf. >2013-11-13 18:47:19,955 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 18:47:19,956 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 18:47:19,956 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 18:47:19,959 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 18:47:19,959 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 18:47:19,960 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 18:51:39,458 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-95) Start refreshing all users data >2013-11-13 18:51:39,681 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-95) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 19:47:19,961 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Attempting to update VMs/Templates Ovf. >2013-11-13 19:47:19,962 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 19:47:19,963 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 19:47:19,963 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Attempting to update template OVFs in Data Center new-nfs >2013-11-13 19:47:19,979 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 19:47:19,979 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 19:47:19,980 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 19:51:39,728 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-26) Start refreshing all users data >2013-11-13 19:51:39,755 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-26) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 20:47:19,981 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Attempting to update VMs/Templates Ovf. >2013-11-13 20:47:19,982 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 20:47:20,013 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 20:47:20,013 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 20:47:20,016 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 20:47:20,017 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 20:47:20,017 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 20:51:39,805 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-63) Start refreshing all users data >2013-11-13 20:51:39,841 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-63) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 21:47:20,017 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to update VMs/Templates Ovf. >2013-11-13 21:47:20,018 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 21:47:20,019 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 21:47:20,019 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to update template OVFs in Data Center new-nfs >2013-11-13 21:47:20,020 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 21:47:20,020 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 21:47:20,021 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 21:51:39,880 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-98) Start refreshing all users data >2013-11-13 21:51:40,045 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-98) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 22:47:20,021 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) [18e72917] Attempting to update VMs/Templates Ovf. >2013-11-13 22:47:20,023 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) [18e72917] Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 22:47:20,031 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) [18e72917] Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 22:47:20,031 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) [18e72917] Attempting to update template OVFs in Data Center new-nfs >2013-11-13 22:47:20,035 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) [18e72917] Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 22:47:20,035 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) [18e72917] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 22:47:20,035 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) [18e72917] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 22:51:40,084 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-2) [15da280] Start refreshing all users data >2013-11-13 22:51:40,169 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-2) [15da280] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-13 23:47:20,037 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) [5f15853b] Attempting to update VMs/Templates Ovf. >2013-11-13 23:47:20,038 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) [5f15853b] Attempting to update VM OVFs in Data Center new-nfs >2013-11-13 23:47:20,047 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) [5f15853b] Successfully updated VM OVFs in Data Center new-nfs >2013-11-13 23:47:20,047 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) [5f15853b] Attempting to update template OVFs in Data Center new-nfs >2013-11-13 23:47:20,050 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) [5f15853b] Successfully updated templates OVFs in Data Center new-nfs >2013-11-13 23:47:20,050 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) [5f15853b] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 23:47:20,051 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) [5f15853b] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-13 23:51:40,274 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-4) [5034ca1b] Start refreshing all users data >2013-11-13 23:51:40,412 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-4) [5034ca1b] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 00:47:20,051 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2013-11-14 00:47:20,052 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 00:47:20,053 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 00:47:20,053 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update template OVFs in Data Center new-nfs >2013-11-14 00:47:20,054 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 00:47:20,054 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 00:47:20,054 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 00:51:40,535 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-96) [6bd22279] Start refreshing all users data >2013-11-14 00:51:40,573 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-96) [6bd22279] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 01:47:20,054 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2013-11-14 01:47:20,056 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 01:47:20,057 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 01:47:20,057 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update template OVFs in Data Center new-nfs >2013-11-14 01:47:20,057 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 01:47:20,057 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 01:47:20,058 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 01:51:40,629 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-100) Start refreshing all users data >2013-11-14 01:51:40,648 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-100) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 02:47:20,058 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update VMs/Templates Ovf. >2013-11-14 02:47:20,059 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 02:47:20,060 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 02:47:20,060 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update template OVFs in Data Center new-nfs >2013-11-14 02:47:20,063 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 02:47:20,063 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 02:47:20,064 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 02:51:40,683 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-50) [54c97d08] Start refreshing all users data >2013-11-14 02:51:40,713 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-50) [54c97d08] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-12) Start deleteAgedOutAuditLogs >2013-11-14 03:35:35,031 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-12) Finished deleteAgedOutAuditLogs >2013-11-14 03:47:20,064 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-86) [2b040ed5] Attempting to update VMs/Templates Ovf. >2013-11-14 03:47:20,065 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-86) [2b040ed5] Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 03:47:20,074 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-86) [2b040ed5] Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 03:47:20,074 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-86) [2b040ed5] Attempting to update template OVFs in Data Center new-nfs >2013-11-14 03:47:20,075 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-86) [2b040ed5] Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 03:47:20,075 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-86) [2b040ed5] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 03:47:20,076 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-86) [2b040ed5] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 03:51:40,751 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-20) Start refreshing all users data >2013-11-14 03:51:40,787 ERROR [org.ovirt.engine.core.bll.adbroker.LDAPTemplateWrapper] (DefaultQuartzScheduler_Worker-20) Error in running LDAP query. BaseDN is , filter is (|(objectGUID=9b9002d1-ec33-4083-8a7b-31f6b8931648)). Exception message is: Request: 4 cancelled; nested exception is javax.naming.CommunicationException: Request: 4 cancelled; remaining name '' >2013-11-14 03:51:40,787 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-20) Error in communicating with LDAP server Request: 4 cancelled; nested exception is javax.naming.CommunicationException: Request: 4 cancelled; remaining name '' >2013-11-14 03:51:40,788 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-20) Failed ldap search server LDAP://qa1.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to communication error. We should try the next server >2013-11-14 03:52:10,795 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-20) Error in communicating with LDAP server qa2-tlv.qa.lab.tlv.redhat.com:389; nested exception is javax.naming.CommunicationException: qa2-tlv.qa.lab.tlv.redhat.com:389 [Root exception is java.net.SocketTimeoutException: connect timed out] >2013-11-14 03:52:10,796 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-20) Failed ldap search server LDAP://qa2-tlv.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to connection timeout. We should try the next server >2013-11-14 03:52:10,796 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-20) User vdcadmin not found in directory server, its status switched to InActive >2013-11-14 04:47:20,076 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update VMs/Templates Ovf. >2013-11-14 04:47:20,077 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 04:47:20,086 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 04:47:20,087 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update template OVFs in Data Center new-nfs >2013-11-14 04:47:20,087 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 04:47:20,087 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 04:47:20,088 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 04:52:10,799 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-58) [18e72917] Start refreshing all users data >2013-11-14 04:52:11,040 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-58) [18e72917] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 04:52:11,120 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-58) [18e72917] Inactive User vdcadmin found in directory server, its status switched to Active >2013-11-14 05:47:20,088 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Attempting to update VMs/Templates Ovf. >2013-11-14 05:47:20,089 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 05:47:20,090 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 05:47:20,090 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Attempting to update template OVFs in Data Center new-nfs >2013-11-14 05:47:20,093 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 05:47:20,093 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 05:47:20,094 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [55890714] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 05:52:11,135 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) Start refreshing all users data >2013-11-14 05:52:11,195 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-1) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 06:47:20,094 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2013-11-14 06:47:20,107 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 06:47:20,108 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 06:47:20,109 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update template OVFs in Data Center new-nfs >2013-11-14 06:47:20,109 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 06:47:20,109 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 06:47:20,110 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 06:52:11,235 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-65) Start refreshing all users data >2013-11-14 06:52:11,289 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-65) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 07:47:20,110 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Attempting to update VMs/Templates Ovf. >2013-11-14 07:47:20,111 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 07:47:20,112 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 07:47:20,112 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Attempting to update template OVFs in Data Center new-nfs >2013-11-14 07:47:20,115 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 07:47:20,116 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 07:47:20,116 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) [581f320f] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 07:52:11,398 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-70) [2d55389e] Start refreshing all users data >2013-11-14 07:52:11,472 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-70) [2d55389e] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 08:47:20,116 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2013-11-14 08:47:20,117 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 08:47:20,127 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 08:47:20,127 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update template OVFs in Data Center new-nfs >2013-11-14 08:47:20,127 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 08:47:20,128 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 08:47:20,128 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 08:52:11,531 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-11) [1889a93e] Start refreshing all users data >2013-11-14 08:52:11,743 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-11) [1889a93e] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 09:47:20,128 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) [4a80fae] Attempting to update VMs/Templates Ovf. >2013-11-14 09:47:20,129 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) [4a80fae] Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 09:47:20,130 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) [4a80fae] Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 09:47:20,130 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) [4a80fae] Attempting to update template OVFs in Data Center new-nfs >2013-11-14 09:47:20,133 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) [4a80fae] Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 09:47:20,133 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) [4a80fae] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 09:47:20,134 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) [4a80fae] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 09:52:11,819 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-21) Start refreshing all users data >2013-11-14 09:52:11,942 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-21) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 10:47:20,135 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Attempting to update VMs/Templates Ovf. >2013-11-14 10:47:20,136 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 10:47:20,137 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 10:47:20,137 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Attempting to update template OVFs in Data Center new-nfs >2013-11-14 10:47:20,140 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 10:47:20,140 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 10:47:20,141 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 10:52:11,975 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-13) [51857563] Start refreshing all users data >2013-11-14 10:52:11,994 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-13) [51857563] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 11:47:20,141 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Attempting to update VMs/Templates Ovf. >2013-11-14 11:47:20,143 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 11:47:20,151 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 11:47:20,151 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Attempting to update template OVFs in Data Center new-nfs >2013-11-14 11:47:20,152 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 11:47:20,152 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 11:47:20,152 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 11:52:12,024 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-99) [5faae8b8] Start refreshing all users data >2013-11-14 11:52:12,148 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-99) [5faae8b8] Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 12:47:20,152 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2013-11-14 12:47:20,153 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 12:47:20,163 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 12:47:20,163 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update template OVFs in Data Center new-nfs >2013-11-14 12:47:20,178 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 12:47:20,178 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 12:47:20,179 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 12:52:12,182 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-44) Start refreshing all users data >2013-11-14 12:52:12,218 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-44) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 13:47:20,179 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Attempting to update VMs/Templates Ovf. >2013-11-14 13:47:20,180 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 13:47:20,190 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 13:47:20,190 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Attempting to update template OVFs in Data Center new-nfs >2013-11-14 13:47:20,191 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 13:47:20,191 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 13:47:20,192 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) [5ad0c381] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 13:52:12,246 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-20) Start refreshing all users data >2013-11-14 13:52:12,270 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-20) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 14:47:20,192 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) [20124fbd] Attempting to update VMs/Templates Ovf. >2013-11-14 14:47:20,193 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) [20124fbd] Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 14:47:20,214 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) [20124fbd] Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 14:47:20,214 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) [20124fbd] Attempting to update template OVFs in Data Center new-nfs >2013-11-14 14:47:20,229 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) [20124fbd] Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 14:47:20,229 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) [20124fbd] Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 14:47:20,230 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) [20124fbd] Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 14:52:12,324 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-5) Start refreshing all users data >2013-11-14 14:52:12,380 ERROR [org.ovirt.engine.core.bll.adbroker.LDAPTemplateWrapper] (DefaultQuartzScheduler_Worker-5) Error in running LDAP query. BaseDN is , filter is (|(objectGUID=9b9002d1-ec33-4083-8a7b-31f6b8931648)). Exception message is: Request: 4 cancelled; nested exception is javax.naming.CommunicationException: Request: 4 cancelled; remaining name '' >2013-11-14 14:52:12,381 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-5) Error in communicating with LDAP server Request: 4 cancelled; nested exception is javax.naming.CommunicationException: Request: 4 cancelled; remaining name '' >2013-11-14 14:52:12,395 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-5) Failed ldap search server LDAP://qa1.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to communication error. We should try the next server >2013-11-14 14:52:42,404 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-5) Error in communicating with LDAP server qa2-tlv.qa.lab.tlv.redhat.com:389; nested exception is javax.naming.CommunicationException: qa2-tlv.qa.lab.tlv.redhat.com:389 [Root exception is java.net.SocketTimeoutException: connect timed out] >2013-11-14 14:52:42,405 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-5) Failed ldap search server LDAP://qa2-tlv.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to connection timeout. We should try the next server >2013-11-14 14:52:42,405 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-5) User vdcadmin not found in directory server, its status switched to InActive >2013-11-14 14:57:42,930 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2013-11-14 14:57:43,020 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf". >2013-11-14 14:57:43,020 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2013-11-14 14:57:43,020 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2013-11-14 14:57:43,021 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-java.conf". >2013-11-14 14:57:43,021 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2013-11-14 14:57:43,021 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2013-11-14 14:57:43,037 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2013-11-14 14:57:43,037 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2013-11-14 14:57:43,038 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_AJP_ENABLED" is "true". >2013-11-14 14:57:43,038 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_AJP_PORT" is "8702". >2013-11-14 14:57:43,038 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_APPS" is "engine.ear redhat-support-plugin-rhev.ear". >2013-11-14 14:57:43,038 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2013-11-14 14:57:43,038 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2013-11-14 14:57:43,038 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2013-11-14 14:57:43,038 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_DATABASE" is "engine". >2013-11-14 14:57:43,039 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2013-11-14 14:57:43,039 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_HOST" is "localhost". >2013-11-14 14:57:43,039 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2013-11-14 14:57:43,039 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "75". >2013-11-14 14:57:43,039 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_PASSWORD" is "***". >2013-11-14 14:57:43,039 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_PORT" is "5432". >2013-11-14 14:57:43,039 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_SECURED" is "False". >2013-11-14 14:57:43,039 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2013-11-14 14:57:43,040 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_SSL" is "false". >2013-11-14 14:57:43,040 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2013-11-14 14:57:43,040 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_USER" is "engine". >2013-11-14 14:57:43,040 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2013-11-14 14:57:43,040 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2013-11-14 14:57:43,040 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2013-11-14 14:57:43,040 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_FQDN" is "slitmano-rhevm.scl.lab.tlv.redhat.com". >2013-11-14 14:57:43,040 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_GROUP" is "ovirt". >2013-11-14 14:57:43,041 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_HEAP_MAX" is "1g". >2013-11-14 14:57:43,041 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_HEAP_MIN" is "1g". >2013-11-14 14:57:43,041 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2013-11-14 14:57:43,041 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_HTTPS_PORT" is "None". >2013-11-14 14:57:43,041 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2013-11-14 14:57:43,041 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_HTTP_ENABLED" is "false". >2013-11-14 14:57:43,041 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_HTTP_PORT" is "None". >2013-11-14 14:57:43,041 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2013-11-14 14:57:43,042 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2013-11-14 14:57:43,042 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2013-11-14 14:57:43,042 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2013-11-14 14:57:43,042 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PERM_MAX" is "256m". >2013-11-14 14:57:43,042 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PERM_MIN" is "256m". >2013-11-14 14:57:43,042 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2013-11-14 14:57:43,042 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2013-11-14 14:57:43,042 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2013-11-14 14:57:43,043 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2013-11-14 14:57:43,043 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2013-11-14 14:57:43,043 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2013-11-14 14:57:43,043 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2013-11-14 14:57:43,043 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2013-11-14 14:57:43,043 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false". >2013-11-14 14:57:43,043 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PROXY_ENABLED" is "true". >2013-11-14 14:57:43,043 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2013-11-14 14:57:43,044 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2013-11-14 14:57:43,044 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_STOP_INTERVAL" is "1". >2013-11-14 14:57:43,044 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_STOP_TIME" is "10". >2013-11-14 14:57:43,044 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2013-11-14 14:57:43,044 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2013-11-14 14:57:43,044 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_USER" is "ovirt". >2013-11-14 14:57:43,044 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2013-11-14 14:57:43,044 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2013-11-14 14:57:43,044 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "ENGINE_VERBOSE_GC" is "false". >2013-11-14 14:57:43,045 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "JAVA_HOME" is "/usr/lib/jvm/jre-1.7.0-openjdk.x86_64". >2013-11-14 14:57:43,045 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2013-11-14 14:57:43,045 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 45) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2013-11-14 14:57:45,287 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 47) Start initializing Backend >2013-11-14 14:57:47,346 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 47) Running ovirt-engine 3.3.0-0.33.beta1.el6ev >2013-11-14 14:57:47,348 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 47) Start initializing dictionaries >2013-11-14 14:57:47,366 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 47) Finished initializing dictionaries >2013-11-14 14:57:47,367 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 47) Start initializing AuditLogCleanupManager >2013-11-14 14:57:47,367 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 47) Setting audit cleanup manager to run at: 35 35 3 * * ? >2013-11-14 14:57:47,435 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 47) Finished initializing AuditLogCleanupManager >2013-11-14 14:57:47,444 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 47) Start initializing TagsDirector >2013-11-14 14:57:47,473 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 47) Tag root added to tree >2013-11-14 14:57:47,546 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 47) Finished initializing TagsDirector >2013-11-14 14:57:47,549 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 47) Start initializing IsoDomainListSyncronizer >2013-11-14 14:57:47,578 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 47) Finished initializing IsoDomainListSyncronizer >2013-11-14 14:57:47,653 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 47) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2013-11-14 14:57:48,111 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 47) Completed initializing handlers >2013-11-14 14:57:48,112 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 47) Start initializing ErrorTranslatorImpl >2013-11-14 14:57:48,159 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 47) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2013-11-14 14:57:48,160 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 47) Finished initializing ErrorTranslatorImpl >2013-11-14 14:57:48,160 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 47) Start initializing ErrorTranslatorImpl >2013-11-14 14:57:48,160 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 47) Finished initializing ErrorTranslatorImpl >2013-11-14 14:57:48,161 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 47) Mark incomplete jobs as UNKNOWN >2013-11-14 14:57:48,282 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 47) Start initializing JobRepositoryCleanupManager >2013-11-14 14:57:48,283 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 47) Finished initializing JobRepositoryCleanupManager >2013-11-14 14:57:48,284 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 47) Start initializing AutoRecoveryManager >2013-11-14 14:57:48,286 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 47) Finished initializing AutoRecoveryManager >2013-11-14 14:57:48,288 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 47) Start initializing ExecutionMessageDirector >2013-11-14 14:57:48,290 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 47) Finished initializing ExecutionMessageDirector >2013-11-14 14:57:48,368 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (ServerService Thread Pool -- 37) Start initializing UsersDomainsCacheManagerService >2013-11-14 14:57:48,433 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ServerService Thread Pool -- 40) Initialization of AsyncTaskManager completed successfully. >2013-11-14 14:57:48,436 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) Start initializing ResourceManager >2013-11-14 14:57:48,940 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (ServerService Thread Pool -- 37) Start initializing DbUserCacheManager >2013-11-14 14:57:48,940 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (ServerService Thread Pool -- 37) Finished initializing DbUserCacheManager >2013-11-14 14:57:48,941 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (ServerService Thread Pool -- 37) Finished initializing UsersDomainsCacheManagerService >2013-11-14 14:57:49,039 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) Start refreshing all users data >2013-11-14 14:57:49,253 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) Entered VdsManager constructor >2013-11-14 14:57:49,449 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) Initialize vdsBroker (monique-vds05.tlv.redhat.com,54,321) >2013-11-14 14:57:50,133 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) VDS d80ddaf1-0131-4088-94cf-d3ad317c10df was added to the Resource Manager >2013-11-14 14:57:50,133 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) Entered VdsManager constructor >2013-11-14 14:57:50,335 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) Initialize vdsBroker (rose07.qa.lab.tlv.redhat.com,54,321) >2013-11-14 14:57:50,352 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) VDS d71e8f58-702c-42f3-9464-787ae4db30d6 was added to the Resource Manager >2013-11-14 14:57:50,415 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) Finished initializing ResourceManager >2013-11-14 14:57:50,433 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 40) Initialization of OvfDataUpdater completed successfully. >2013-11-14 14:57:50,436 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Start scheduling to enable vds load balancer >2013-11-14 14:57:50,437 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Finished scheduling to enable vds load balancer >2013-11-14 14:57:50,485 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 40) Init VM custom properties utilities >2013-11-14 14:57:50,495 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 40) Init device custom properties utilities >2013-11-14 14:57:50,499 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Initializing Scheduling manager >2013-11-14 14:57:50,594 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-5-thread-1) MacPoolManager(67076126): Start initializing >2013-11-14 14:57:50,731 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) External scheduler disabled, discovery skipped >2013-11-14 14:57:50,731 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Initialized Scheduling manager >2013-11-14 14:57:50,732 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 40) Initializing DWH Heart Beat >2013-11-14 14:57:50,793 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-5-thread-1) MacPoolManager(67076126): Finished initializing >2013-11-14 14:57:55,310 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, vds=Host[monique05]), log id: 2cd55141 >2013-11-14 14:57:55,315 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GetHardwareInfoVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, vds=Host[rose07.qa.lab.tlv.redhat.com]), log id: 6ac0de8e >2013-11-14 14:57:55,395 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GetHardwareInfoVDSCommand, log id: 2cd55141 >2013-11-14 14:57:55,420 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GetHardwareInfoVDSCommand, log id: 6ac0de8e >2013-11-14 14:57:55,817 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: UNASSIGNED not exist in string table >2013-11-14 14:57:55,818 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2013-11-14 14:57:55,818 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2013-11-14 14:57:55,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2013-11-14 14:57:55,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2013-11-14 14:57:55,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: DWH_STOPPED not exist in string table >2013-11-14 14:57:55,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: DWH_STARTED not exist in string table >2013-11-14 14:57:55,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: DWH_ERROR not exist in string table >2013-11-14 14:57:55,863 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: USER_FAILED_REMOVE_VM not have severity. Assumed Normal >2013-11-14 14:57:55,863 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: USER_ATTACH_DISK_TO_VM not have severity. Assumed Normal >2013-11-14 14:57:55,863 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: USER_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2013-11-14 14:57:55,863 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: USER_FAILED_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2013-11-14 14:57:55,863 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not have severity. Assumed Normal >2013-11-14 14:57:55,864 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: GLUSTER_VOLUME_OPTION_CHANGED_FROM_CLI not have severity. Assumed Normal >2013-11-14 14:57:55,864 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: GLUSTER_SERVICES_LIST_NOT_FETCHED not have severity. Assumed Normal >2013-11-14 14:57:55,864 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: GLUSTER_VOLUME_BRICK_ADDED not have severity. Assumed Normal >2013-11-14 14:57:55,864 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: USER_EXTEND_DISK_SIZE_UPDATE_VM_FAILURE not have severity. Assumed Normal >2013-11-14 14:57:55,864 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: VM_MIGRATION_START_SYSTEM_INITIATED not have severity. Assumed Normal >2013-11-14 14:57:55,865 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not have severity. Assumed Normal >2013-11-14 14:57:55,865 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_UP not have severity. Assumed Normal >2013-11-14 14:57:55,865 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_DOWN not have severity. Assumed Normal >2013-11-14 14:57:55,865 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: USER_UPDATED_MOM_POLICIES not have severity. Assumed Normal >2013-11-14 14:57:55,865 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) AuditLogType: USER_FAILED_TO_UPDATE_MOM_POLICIES not have severity. Assumed Normal >2013-11-14 14:57:56,200 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-14 14:57:56,556 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host rose07.qa.lab.tlv.redhat.com. >2013-11-14 14:57:56,809 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Initializing Host: rose07.qa.lab.tlv.redhat.com >2013-11-14 14:57:56,820 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Initializing Host: monique05 >2013-11-14 14:57:57,489 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-4) [581ba5cf] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-14 14:57:57,501 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-3) [1e34a30f] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-14 14:57:57,759 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-12) domain 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53:nfs in problem. vds: rose07.qa.lab.tlv.redhat.com >2013-11-14 14:58:00,620 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-8) [259b117] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-14 14:58:02,020 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) [259b117] Correlation ID: 259b117, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center new-nfs. Setting status to Non Responsive. >2013-11-14 14:58:02,228 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) [259b117] hostFromVds::selectedVds - monique05, spmStatus Free, storage pool new-nfs >2013-11-14 14:58:02,229 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) [259b117] Found spm host d71e8f58-702c-42f3-9464-787ae4db30d6, host name: rose07.qa.lab.tlv.redhat.com, according to spmId: 1. >2013-11-14 14:58:02,273 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) [259b117] SpmStatus on vds d71e8f58-702c-42f3-9464-787ae4db30d6: Unknown_Pool >2013-11-14 14:58:02,299 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) [259b117] starting spm on vds monique05, storage pool new-nfs, prevId 1, LVER 4 >2013-11-14 14:58:02,326 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-8) [259b117] START, SpmStartVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, prevId=1, prevLVER=4, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 54657a67 >2013-11-14 14:58:02,376 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-8) [259b117] spmStart polling started: taskId = f5e05dab-8bfc-47c7-a233-61405dc9389c >2013-11-14 14:58:03,421 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-8) [259b117] spmStart polling ended: taskId = f5e05dab-8bfc-47c7-a233-61405dc9389c task status = finished >2013-11-14 14:58:03,466 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-8) [259b117] spmStart polling ended, spm status: SPM >2013-11-14 14:58:03,470 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-8) [259b117] START, HSMClearTaskVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, taskId=f5e05dab-8bfc-47c7-a233-61405dc9389c), log id: 65942695 >2013-11-14 14:58:03,498 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-8) [259b117] FINISH, HSMClearTaskVDSCommand, log id: 65942695 >2013-11-14 14:58:03,499 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-8) [259b117] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@58169433, log id: 54657a67 >2013-11-14 14:58:03,506 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) [259b117] Initialize Irs proxy from vds: monique-vds05.tlv.redhat.com >2013-11-14 14:58:03,565 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) [259b117] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host monique05 (Address: monique-vds05.tlv.redhat.com). >2013-11-14 14:58:03,865 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-5-thread-21) [259b117] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, ignoreFailoverLimit = false), log id: 4f96259d >2013-11-14 14:58:03,920 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-5-thread-21) [259b117] -- executeIrsBrokerCommand: Attempting on storage pool a44a18ab-6f2e-48d4-ab60-889845f498d3 >2013-11-14 14:58:03,926 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-5-thread-21) [259b117] START, HSMGetAllTasksInfoVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df), log id: 466c4be >2013-11-14 14:58:03,953 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-5-thread-21) [259b117] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 466c4be >2013-11-14 14:58:03,954 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-5-thread-21) [259b117] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 4f96259d >2013-11-14 14:58:03,954 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-5-thread-21) [259b117] Discovered no tasks on Storage Pool new-nfs >2013-11-14 14:58:19,303 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (DefaultQuartzScheduler_Worker-1) Failed to query rootDSE for LDAP server LDAP://qa2-tlv.qa.lab.tlv.redhat.com:389 due to connection timeout >2013-11-14 14:58:19,320 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-1) Failed ldap search server LDAP://qa2-tlv.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to connection timeout. We should try the next server >2013-11-14 14:58:50,301 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-1) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 14:58:50,404 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) Inactive User vdcadmin found in directory server, its status switched to Active >2013-11-14 15:02:57,764 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) starting ProcessDomainRecovery for domain 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53:nfs >2013-11-14 15:02:57,968 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) vds rose07.qa.lab.tlv.redhat.com reported domain 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53:nfs as in problem, attempting to move the vds to status NonOperational >2013-11-14 15:02:58,325 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (pool-5-thread-48) [5849fd1d] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-14 15:02:58,478 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [5849fd1d] START, SetVdsStatusVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, status=NonOperational, nonOperationalReason=STORAGE_DOMAIN_UNREACHABLE), log id: 3888f686 >2013-11-14 15:02:58,492 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-5-thread-48) [5849fd1d] FINISH, SetVdsStatusVDSCommand, log id: 3888f686 >2013-11-14 15:02:58,552 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-48) [5849fd1d] Correlation ID: 5849fd1d, Job ID: 3253dde4-6cbd-44dc-ac49-8ee22b3f3dba, Call Stack: null, Custom Event ID: -1, Message: Host rose07.qa.lab.tlv.redhat.com cannot access one of the Storage Domains attached to the Data Center new-nfs. Setting Host state to Non-Operational. >2013-11-14 15:05:00,142 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-70) Autorecovering 1 hosts >2013-11-14 15:05:00,143 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-70) Autorecovering hosts id: d71e8f58-702c-42f3-9464-787ae4db30d6, name : rose07.qa.lab.tlv.redhat.com >2013-11-14 15:05:00,286 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-70) [1819eab1] Lock Acquired to object EngineLock [exclusiveLocks= key: d71e8f58-702c-42f3-9464-787ae4db30d6 value: VDS >, sharedLocks= ] >2013-11-14 15:05:00,288 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-70) [1819eab1] Running command: ActivateVdsCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-14 15:05:00,288 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-70) [1819eab1] Before acquiring lock in order to prevent monitoring for host rose07.qa.lab.tlv.redhat.com from data-center new-nfs >2013-11-14 15:05:00,288 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-70) [1819eab1] Lock acquired, from now a monitoring of host will be skipped for host rose07.qa.lab.tlv.redhat.com from data-center new-nfs >2013-11-14 15:05:00,438 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-70) [1819eab1] START, SetVdsStatusVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, status=Unassigned, nonOperationalReason=NONE), log id: fc4c76b >2013-11-14 15:05:00,448 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-70) [1819eab1] FINISH, SetVdsStatusVDSCommand, log id: fc4c76b >2013-11-14 15:05:00,575 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-70) [1819eab1] START, ActivateVdsVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6), log id: 24d2f582 >2013-11-14 15:05:01,060 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-70) [1819eab1] START, GetHardwareInfoVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, vds=Host[rose07.qa.lab.tlv.redhat.com]), log id: 824e680 >2013-11-14 15:05:01,076 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-70) [1819eab1] FINISH, GetHardwareInfoVDSCommand, log id: 824e680 >2013-11-14 15:05:01,380 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-70) [1abf8a1d] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-14 15:05:01,545 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-70) [3f92c884] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-14 15:05:01,548 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-70) [3f92c884] FINISH, ActivateVdsVDSCommand, return: Host[rose07.qa.lab.tlv.redhat.com], log id: 24d2f582 >2013-11-14 15:05:02,314 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-70) [3f92c884] Activate finished. Lock released. Monitoring can run now for host rose07.qa.lab.tlv.redhat.com from data-center new-nfs >2013-11-14 15:05:02,475 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) [3f92c884] Correlation ID: 1819eab1, Call Stack: null, Custom Event ID: -1, Message: Host rose07.qa.lab.tlv.redhat.com was autorecovered. >2013-11-14 15:05:02,476 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-70) [3f92c884] Lock freed to object EngineLock [exclusiveLocks= key: d71e8f58-702c-42f3-9464-787ae4db30d6 value: VDS >, sharedLocks= ] >2013-11-14 15:05:04,981 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GetHardwareInfoVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, vds=Host[rose07.qa.lab.tlv.redhat.com]), log id: 448b1ae7 >2013-11-14 15:05:05,009 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GetHardwareInfoVDSCommand, log id: 448b1ae7 >2013-11-14 15:05:05,193 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host rose07.qa.lab.tlv.redhat.com. >2013-11-14 15:05:05,211 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-74) [5d190c2e] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-14 15:05:05,241 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-74) [4eb91236] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-14 15:05:05,477 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-74) [4eb91236] START, ConnectStorageServerVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, storageType = NFS, connectionList = [{ id: 99031bb9-9962-4503-abb7-cdb05be33628, connection: 10.35.160.108:/RHEV/slitmano, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 3cdf5aee >2013-11-14 15:05:05,552 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-74) [4eb91236] FINISH, ConnectStorageServerVDSCommand, return: {99031bb9-9962-4503-abb7-cdb05be33628=0}, log id: 3cdf5aee >2013-11-14 15:05:05,559 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-74) [4eb91236] Host rose07.qa.lab.tlv.redhat.com storage connection was succeeded >2013-11-14 15:05:05,721 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) START, ConnectStoragePoolVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, vds_spm_id = 1, masterDomainId = 05a9f477-dfbe-4d25-8ea8-d9c6f1a8ec53, masterVersion = 1), log id: 3c94db92 >2013-11-14 15:05:10,139 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-5-thread-48) FINISH, ConnectStoragePoolVDSCommand, log id: 3c94db92 >2013-11-14 15:05:10,492 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-74) [4eb91236] START, SetMOMPolicyParametersVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6), log id: 32a74038 >2013-11-14 15:05:10,536 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-74) [4eb91236] FINISH, SetMOMPolicyParametersVDSCommand, log id: 32a74038 >2013-11-14 15:05:10,687 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) [4eb91236] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rose07.qa.lab.tlv.redhat.com. >2013-11-14 15:05:10,830 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-74) [71daf74d] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-14 15:05:10,961 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-74) [d1eec30] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-14 15:57:50,432 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Attempting to update VMs/Templates Ovf. >2013-11-14 15:57:50,445 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 15:57:50,466 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 15:57:50,467 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Attempting to update template OVFs in Data Center new-nfs >2013-11-14 15:57:50,476 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 15:57:50,476 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 15:57:50,483 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 15:58:50,444 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-65) Start refreshing all users data >2013-11-14 15:58:50,500 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-65) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 16:57:50,482 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) Attempting to update VMs/Templates Ovf. >2013-11-14 16:57:50,483 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) Attempting to update VM OVFs in Data Center new-nfs >2013-11-14 16:57:50,510 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) Successfully updated VM OVFs in Data Center new-nfs >2013-11-14 16:57:50,511 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) Attempting to update template OVFs in Data Center new-nfs >2013-11-14 16:57:50,515 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) Successfully updated templates OVFs in Data Center new-nfs >2013-11-14 16:57:50,515 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) Attempting to remove unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 16:57:50,516 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) Successfully removed unneeded template/vm OVFs in Data Center new-nfs >2013-11-14 16:58:50,574 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-57) Start refreshing all users data >2013-11-14 16:58:50,629 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-57) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-14 17:18:18,648 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:19,966 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:21,693 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:22,143 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-89) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:22,147 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-83) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:22,969 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:23,269 ERROR [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (DefaultQuartzScheduler_Worker-95) Error updating DWH Heart Beat: : org.springframework.dao.DataAccessResourceFailureException: CallableStatementCallback; SQL [{call updatedwhhistorytimekeeping(?, ?, ?)}]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] > at org.ovirt.engine.core.dao.dwh.DwhHistoryTimekeepingDaoDbFacadeImpl.save(DwhHistoryTimekeepingDaoDbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.bll.dwh.DwhHeartBeat.engineIsRunningNotification(DwhHeartBeat.java:42) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:379) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:297) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404) > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1066) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1014) [spring-jdbc.jar:3.1.1.RELEASE] > ... 14 more > >2013-11-14 17:18:24,696 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:25,973 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) [1e34a30f] Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:27,157 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-94) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:27,161 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-6) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:27,700 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:28,976 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:30,703 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:31,988 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:32,184 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-5) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:32,187 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-4) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:33,707 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:34,991 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:36,710 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:37,188 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-21) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:37,191 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-13) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:37,994 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:39,720 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:40,998 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [259b117] Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:42,192 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-17) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:42,197 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-23) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:42,725 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:44,001 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:45,729 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:47,004 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:47,205 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-27) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:47,208 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-34) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:48,348 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-32) Failed to invoke scheduled method updateQuotaCache: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_40] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getquotacount()]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.QuotaDAODbFacadeImpl.getQuotaCount(QuotaDAODbFacadeImpl.java:198) [dal.jar:] > at org.ovirt.engine.core.bll.quota.QuotaManager.isCacheUpdateNeeded(QuotaManager.java:1080) [bll.jar:] > at org.ovirt.engine.core.bll.quota.QuotaManager.updateQuotaCache(QuotaManager.java:1049) [bll.jar:] > ... 7 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more > >2013-11-14 17:18:48,734 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:50,008 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:51,737 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:52,104 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-37) Failed to invoke scheduled method gluster_async_task_poll_event: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:52) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:52,209 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-42) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:52,211 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-44) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:53,010 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:53,274 ERROR [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (DefaultQuartzScheduler_Worker-41) Error updating DWH Heart Beat: : org.springframework.dao.DataAccessResourceFailureException: CallableStatementCallback; SQL [{call updatedwhhistorytimekeeping(?, ?, ?)}]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] > at org.ovirt.engine.core.dao.dwh.DwhHistoryTimekeepingDaoDbFacadeImpl.save(DwhHistoryTimekeepingDaoDbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.bll.dwh.DwhHeartBeat.engineIsRunningNotification(DwhHeartBeat.java:42) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:379) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:297) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404) > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1066) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1014) [spring-jdbc.jar:3.1.1.RELEASE] > ... 14 more > >2013-11-14 17:18:54,741 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:56,014 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:57,213 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-52) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:57,215 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-48) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:18:57,744 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:18:59,027 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:00,753 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:02,030 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:02,224 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-51) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:02,227 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-59) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:03,760 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:05,038 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:06,767 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:07,238 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-66) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:07,241 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-65) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:08,043 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:09,779 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-68) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:11,057 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:12,247 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-69) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:12,249 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-72) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:12,785 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:13,524 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-70) Failed to invoke scheduled method PerformLoadBalancing: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.scheduling.SchedulingManager.PerformLoadBalancing(SchedulingManager.java:562) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:14,064 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:15,790 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:17,072 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:17,256 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-1) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:17,260 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-74) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:18,802 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:20,083 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:21,804 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:22,264 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-87) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:22,266 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-86) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:23,093 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:23,283 ERROR [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (DefaultQuartzScheduler_Worker-90) Error updating DWH Heart Beat: : org.springframework.dao.DataAccessResourceFailureException: CallableStatementCallback; SQL [{call updatedwhhistorytimekeeping(?, ?, ?)}]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] > at org.ovirt.engine.core.dao.dwh.DwhHistoryTimekeepingDaoDbFacadeImpl.save(DwhHistoryTimekeepingDaoDbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.bll.dwh.DwhHeartBeat.engineIsRunningNotification(DwhHeartBeat.java:42) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:379) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:297) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404) > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1066) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1014) [spring-jdbc.jar:3.1.1.RELEASE] > ... 14 more > >2013-11-14 17:19:24,808 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:26,104 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:27,277 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-96) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:27,279 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-3) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:27,812 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:29,112 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:30,821 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:32,115 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:32,280 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-11) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:32,282 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-2) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:33,823 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:35,123 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:36,826 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:37,283 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-9) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:37,286 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-12) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:38,126 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:39,829 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:41,129 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:42,286 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-19) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:42,289 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-18) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:42,832 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [259b117] Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:44,131 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:45,833 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:47,133 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:47,290 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-26) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:47,303 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-25) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:48,835 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:50,136 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:51,838 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-33) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:52,107 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-24) Failed to invoke scheduled method gluster_async_task_poll_event: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:52) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:52,293 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-38) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:52,304 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-40) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:53,139 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:53,286 ERROR [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (DefaultQuartzScheduler_Worker-37) Error updating DWH Heart Beat: : org.springframework.dao.DataAccessResourceFailureException: CallableStatementCallback; SQL [{call updatedwhhistorytimekeeping(?, ?, ?)}]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] > at org.ovirt.engine.core.dao.dwh.DwhHistoryTimekeepingDaoDbFacadeImpl.save(DwhHistoryTimekeepingDaoDbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.bll.dwh.DwhHeartBeat.engineIsRunningNotification(DwhHeartBeat.java:42) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:379) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:297) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404) > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1066) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1014) [spring-jdbc.jar:3.1.1.RELEASE] > ... 14 more > >2013-11-14 17:19:54,840 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:56,141 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:57,296 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-41) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:57,310 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-45) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:19:57,842 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:19:59,144 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:00,000 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-52) Failed to invoke scheduled method onTimer: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getfailingvdss()]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.listFailedAutorecoverables(VdsDAODbFacadeImpl.java:188) [dal.jar:] > at org.ovirt.engine.core.bll.AutoRecoveryManager.check(AutoRecoveryManager.java:121) [bll.jar:] > at org.ovirt.engine.core.bll.AutoRecoveryManager.onTimer(AutoRecoveryManager.java:59) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:00,845 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:02,147 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:02,298 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-53) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:02,313 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-50) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:03,847 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:05,150 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:06,849 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:07,300 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-56) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:07,316 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-55) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:08,153 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:09,851 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:11,155 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:12,303 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-63) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:12,319 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-68) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:12,853 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:13,527 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-67) Failed to invoke scheduled method PerformLoadBalancing: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.scheduling.SchedulingManager.PerformLoadBalancing(SchedulingManager.java:562) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:14,158 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:15,856 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:17,160 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:17,305 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-75) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:17,321 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-71) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:18,859 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:20,162 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) [d1eec30] Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:21,861 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:22,309 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-88) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:22,324 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-78) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:23,164 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:23,289 ERROR [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (DefaultQuartzScheduler_Worker-85) Error updating DWH Heart Beat: : org.springframework.dao.DataAccessResourceFailureException: CallableStatementCallback; SQL [{call updatedwhhistorytimekeeping(?, ?, ?)}]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] > at org.ovirt.engine.core.dao.dwh.DwhHistoryTimekeepingDaoDbFacadeImpl.save(DwhHistoryTimekeepingDaoDbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.bll.dwh.DwhHeartBeat.engineIsRunningNotification(DwhHeartBeat.java:42) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:379) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:297) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404) > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1066) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1014) [spring-jdbc.jar:3.1.1.RELEASE] > ... 14 more > >2013-11-14 17:20:24,864 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:26,167 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:27,312 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-89) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:27,326 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-83) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:27,867 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:29,168 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:30,870 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) [1e34a30f] Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:32,171 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:32,318 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-94) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:32,331 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-6) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:33,872 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:35,174 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:36,875 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:37,321 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-100) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:37,334 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-5) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:38,176 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) [581ba5cf] Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:39,877 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:41,178 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:42,323 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-10) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:42,336 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-21) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:42,878 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:44,181 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:45,881 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:47,183 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:47,325 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-17) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:47,337 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-23) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:48,884 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:50,186 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:51,886 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:52,109 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-27) Failed to invoke scheduled method gluster_async_task_poll_event: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:52) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:52,328 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-32) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:52,339 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-33) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:53,188 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:53,290 ERROR [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (DefaultQuartzScheduler_Worker-28) Error updating DWH Heart Beat: : org.springframework.dao.DataAccessResourceFailureException: CallableStatementCallback; SQL [{call updatedwhhistorytimekeeping(?, ?, ?)}]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] > at org.ovirt.engine.core.dao.dwh.DwhHistoryTimekeepingDaoDbFacadeImpl.save(DwhHistoryTimekeepingDaoDbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.bll.dwh.DwhHeartBeat.engineIsRunningNotification(DwhHeartBeat.java:42) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:379) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:297) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404) > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1066) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1014) [spring-jdbc.jar:3.1.1.RELEASE] > ... 14 more > >2013-11-14 17:20:54,888 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:56,189 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:57,330 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-35) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:57,341 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-42) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:20:57,891 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:20:59,192 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:00,893 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:02,194 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:02,332 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-39) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:21:02,343 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-52) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:21:03,894 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:05,196 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:06,897 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:07,334 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-47) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:21:07,345 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-54) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:21:08,199 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:09,900 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:11,202 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:12,337 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-58) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:21:12,348 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-61) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:21:12,902 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:13,530 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-65) Failed to invoke scheduled method PerformLoadBalancing: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.scheduling.SchedulingManager.PerformLoadBalancing(SchedulingManager.java:562) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:21:14,203 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:15,905 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:17,205 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:17,340 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-69) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:21:17,349 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-72) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getallfromvdsgroups(?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more > >2013-11-14 17:21:18,907 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) [3f92c884] Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:20,207 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_40] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.postgresql.util.PSQLException: This connection has been closed. > at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:712) > at org.postgresql.jdbc2.AbstractJdbc2Connection.getAutoCommit(AbstractJdbc2Connection.java:678) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:505) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462) > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:644) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more > >2013-11-14 17:21:21,909 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Timer update runtimeinfo failed. Exception:: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select * from getvdsbyvdsid(?, ?, ?)]; This connection has been closed.; nested exception is org.postgresql.util.PSQLException: This connection has been closed. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImp2013-11-18 17:00:32,174 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2013-11-18 17:00:32,203 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Loaded file "/etc/ovirt-engine/engine.conf". >2013-11-18 17:00:32,203 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2013-11-18 17:00:32,203 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2013-11-18 17:00:32,204 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-java.conf". >2013-11-18 17:00:32,204 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2013-11-18 17:00:32,204 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2013-11-18 17:00:32,205 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2013-11-18 17:00:32,205 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2013-11-18 17:00:32,205 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_AJP_ENABLED" is "true". >2013-11-18 17:00:32,205 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_AJP_PORT" is "8702". >2013-11-18 17:00:32,206 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_APPS" is "engine.ear redhat-support-plugin-rhev.ear". >2013-11-18 17:00:32,206 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2013-11-18 17:00:32,206 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2013-11-18 17:00:32,206 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2013-11-18 17:00:32,206 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_DATABASE" is "engine". >2013-11-18 17:00:32,206 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2013-11-18 17:00:32,206 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_HOST" is "localhost". >2013-11-18 17:00:32,206 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2013-11-18 17:00:32,207 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "75". >2013-11-18 17:00:32,207 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_PASSWORD" is "***". >2013-11-18 17:00:32,207 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_PORT" is "5432". >2013-11-18 17:00:32,207 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_SECURED" is "False". >2013-11-18 17:00:32,207 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2013-11-18 17:00:32,207 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_SSL" is "false". >2013-11-18 17:00:32,207 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2013-11-18 17:00:32,207 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DB_USER" is "engine". >2013-11-18 17:00:32,208 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2013-11-18 17:00:32,208 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2013-11-18 17:00:32,208 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2013-11-18 17:00:32,208 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_FQDN" is "slitmano-rhevm.scl.lab.tlv.redhat.com". >2013-11-18 17:00:32,208 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_GROUP" is "ovirt". >2013-11-18 17:00:32,208 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_HEAP_MAX" is "1g". >2013-11-18 17:00:32,208 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_HEAP_MIN" is "1g". >2013-11-18 17:00:32,208 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2013-11-18 17:00:32,208 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_HTTPS_PORT" is "None". >2013-11-18 17:00:32,209 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2013-11-18 17:00:32,209 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_HTTP_ENABLED" is "false". >2013-11-18 17:00:32,209 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_HTTP_PORT" is "None". >2013-11-18 17:00:32,209 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2013-11-18 17:00:32,209 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2013-11-18 17:00:32,209 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2013-11-18 17:00:32,209 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2013-11-18 17:00:32,209 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PERM_MAX" is "256m". >2013-11-18 17:00:32,210 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PERM_MIN" is "256m". >2013-11-18 17:00:32,210 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2013-11-18 17:00:32,210 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2013-11-18 17:00:32,210 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2013-11-18 17:00:32,210 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2013-11-18 17:00:32,210 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2013-11-18 17:00:32,210 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2013-11-18 17:00:32,210 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2013-11-18 17:00:32,211 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2013-11-18 17:00:32,211 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false". >2013-11-18 17:00:32,211 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PROXY_ENABLED" is "true". >2013-11-18 17:00:32,211 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2013-11-18 17:00:32,211 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2013-11-18 17:00:32,211 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_STOP_INTERVAL" is "1". >2013-11-18 17:00:32,211 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_STOP_TIME" is "10". >2013-11-18 17:00:32,211 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2013-11-18 17:00:32,211 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2013-11-18 17:00:32,212 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_USER" is "ovirt". >2013-11-18 17:00:32,212 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2013-11-18 17:00:32,212 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2013-11-18 17:00:32,212 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "ENGINE_VERBOSE_GC" is "false". >2013-11-18 17:00:32,212 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "JAVA_HOME" is "/usr/lib/jvm/jre-1.7.0-openjdk.x86_64". >2013-11-18 17:00:32,212 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2013-11-18 17:00:32,212 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 37) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2013-11-18 17:00:35,511 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 46) Start initializing Backend >2013-11-18 17:00:38,465 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 46) Running ovirt-engine 3.3.0-0.33.beta1.el6ev >2013-11-18 17:00:38,467 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 46) Start initializing dictionaries >2013-11-18 17:00:38,473 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 46) Finished initializing dictionaries >2013-11-18 17:00:38,474 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 46) Start initializing AuditLogCleanupManager >2013-11-18 17:00:38,474 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 46) Setting audit cleanup manager to run at: 35 35 3 * * ? >2013-11-18 17:00:38,541 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 46) Finished initializing AuditLogCleanupManager >2013-11-18 17:00:38,553 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 46) Start initializing TagsDirector >2013-11-18 17:00:38,572 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 46) Tag root added to tree >2013-11-18 17:00:38,722 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 46) Finished initializing TagsDirector >2013-11-18 17:00:38,725 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 46) Start initializing IsoDomainListSyncronizer >2013-11-18 17:00:38,747 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 46) Finished initializing IsoDomainListSyncronizer >2013-11-18 17:00:38,795 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 46) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2013-11-18 17:00:39,362 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 46) Completed initializing handlers >2013-11-18 17:00:39,363 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 46) Start initializing ErrorTranslatorImpl >2013-11-18 17:00:39,412 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 46) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2013-11-18 17:00:39,413 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 46) Finished initializing ErrorTranslatorImpl >2013-11-18 17:00:39,413 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 46) Start initializing ErrorTranslatorImpl >2013-11-18 17:00:39,427 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 46) Finished initializing ErrorTranslatorImpl >2013-11-18 17:00:39,428 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 46) Mark incomplete jobs as UNKNOWN >2013-11-18 17:00:39,649 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 46) Start initializing JobRepositoryCleanupManager >2013-11-18 17:00:39,650 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 46) Finished initializing JobRepositoryCleanupManager >2013-11-18 17:00:39,651 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 46) Start initializing AutoRecoveryManager >2013-11-18 17:00:39,653 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 46) Finished initializing AutoRecoveryManager >2013-11-18 17:00:39,655 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 46) Start initializing ExecutionMessageDirector >2013-11-18 17:00:39,677 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 46) Finished initializing ExecutionMessageDirector >2013-11-18 17:00:39,896 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (ServerService Thread Pool -- 46) Start initializing UsersDomainsCacheManagerService >2013-11-18 17:00:39,973 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ServerService Thread Pool -- 40) Initialization of AsyncTaskManager completed successfully. >2013-11-18 17:00:39,976 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) Start initializing ResourceManager >2013-11-18 17:00:40,695 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (ServerService Thread Pool -- 46) Start initializing DbUserCacheManager >2013-11-18 17:00:40,696 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (ServerService Thread Pool -- 46) Finished initializing DbUserCacheManager >2013-11-18 17:00:40,696 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (ServerService Thread Pool -- 46) Finished initializing UsersDomainsCacheManagerService >2013-11-18 17:00:40,825 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) Start refreshing all users data >2013-11-18 17:00:41,140 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [30202e03] Running command: LogoutUserCommand internal: false. >2013-11-18 17:00:41,192 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [2d2895aa] Running command: LogoutUserCommand internal: false. >2013-11-18 17:00:41,441 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: UNASSIGNED not exist in string table >2013-11-18 17:00:41,442 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2013-11-18 17:00:41,442 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2013-11-18 17:00:41,443 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2013-11-18 17:00:41,444 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2013-11-18 17:00:41,444 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: DWH_STOPPED not exist in string table >2013-11-18 17:00:41,444 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: DWH_STARTED not exist in string table >2013-11-18 17:00:41,445 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: DWH_ERROR not exist in string table >2013-11-18 17:00:41,511 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: USER_FAILED_REMOVE_VM not have severity. Assumed Normal >2013-11-18 17:00:41,512 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: USER_ATTACH_DISK_TO_VM not have severity. Assumed Normal >2013-11-18 17:00:41,512 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: USER_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2013-11-18 17:00:41,512 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: USER_FAILED_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2013-11-18 17:00:41,512 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not have severity. Assumed Normal >2013-11-18 17:00:41,513 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: GLUSTER_VOLUME_OPTION_CHANGED_FROM_CLI not have severity. Assumed Normal >2013-11-18 17:00:41,513 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: GLUSTER_SERVICES_LIST_NOT_FETCHED not have severity. Assumed Normal >2013-11-18 17:00:41,513 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: GLUSTER_VOLUME_BRICK_ADDED not have severity. Assumed Normal >2013-11-18 17:00:41,513 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: USER_EXTEND_DISK_SIZE_UPDATE_VM_FAILURE not have severity. Assumed Normal >2013-11-18 17:00:41,513 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: VM_MIGRATION_START_SYSTEM_INITIATED not have severity. Assumed Normal >2013-11-18 17:00:41,514 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not have severity. Assumed Normal >2013-11-18 17:00:41,514 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_UP not have severity. Assumed Normal >2013-11-18 17:00:41,514 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_DOWN not have severity. Assumed Normal >2013-11-18 17:00:41,515 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: USER_UPDATED_MOM_POLICIES not have severity. Assumed Normal >2013-11-18 17:00:41,515 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] AuditLogType: USER_FAILED_TO_UPDATE_MOM_POLICIES not have severity. Assumed Normal >2013-11-18 17:00:41,692 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) Entered VdsManager constructor >2013-11-18 17:00:42,002 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) Initialize vdsBroker (rose07.qa.lab.tlv.redhat.com,54,321) >2013-11-18 17:00:42,217 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2895aa] Correlation ID: 2d2895aa, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-18 17:00:42,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [30202e03] Correlation ID: 30202e03, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-11-18 17:00:42,594 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) VDS d71e8f58-702c-42f3-9464-787ae4db30d6 was added to the Resource Manager >2013-11-18 17:00:42,595 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) Entered VdsManager constructor >2013-11-18 17:00:42,731 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) Initialize vdsBroker (monique-vds05.tlv.redhat.com,54,321) >2013-11-18 17:00:42,742 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) VDS d80ddaf1-0131-4088-94cf-d3ad317c10df was added to the Resource Manager >2013-11-18 17:00:42,803 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) Finished initializing ResourceManager >2013-11-18 17:00:42,824 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 40) Initialization of OvfDataUpdater completed successfully. >2013-11-18 17:00:42,828 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Start scheduling to enable vds load balancer >2013-11-18 17:00:42,828 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Finished scheduling to enable vds load balancer >2013-11-18 17:00:42,893 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 40) Init VM custom properties utilities >2013-11-18 17:00:42,894 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 40) Init device custom properties utilities >2013-11-18 17:00:42,899 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Initializing Scheduling manager >2013-11-18 17:00:42,924 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-5-thread-1) MacPoolManager(271518f4): Start initializing >2013-11-18 17:00:43,086 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) External scheduler disabled, discovery skipped >2013-11-18 17:00:43,098 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Initialized Scheduling manager >2013-11-18 17:00:43,099 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 40) Initializing DWH Heart Beat >2013-11-18 17:00:43,107 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-5-thread-1) MacPoolManager(271518f4): Finished initializing >2013-11-18 17:00:44,144 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-2) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2013-11-18 17:00:44,172 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-2) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2013-11-18 17:00:48,234 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GetHardwareInfoVDSCommand(HostName = rose07.qa.lab.tlv.redhat.com, HostId = d71e8f58-702c-42f3-9464-787ae4db30d6, vds=Host[rose07.qa.lab.tlv.redhat.com]), log id: 1bdba25 >2013-11-18 17:00:48,236 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GetHardwareInfoVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df, vds=Host[monique05]), log id: 64a77092 >2013-11-18 17:00:48,341 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GetHardwareInfoVDSCommand, log id: 1bdba25 >2013-11-18 17:00:48,344 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GetHardwareInfoVDSCommand, log id: 64a77092 >2013-11-18 17:00:48,901 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host monique05. >2013-11-18 17:00:49,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host rose07.qa.lab.tlv.redhat.com. >2013-11-18 17:00:49,361 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Initializing Host: monique05 >2013-11-18 17:00:49,500 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-4) [5bb6aa03] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d80ddaf1-0131-4088-94cf-d3ad317c10df Type: VDS >2013-11-18 17:00:49,598 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Initializing Host: rose07.qa.lab.tlv.redhat.com >2013-11-18 17:00:50,019 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-3) [79f6a8c2] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d71e8f58-702c-42f3-9464-787ae4db30d6 Type: VDS >2013-11-18 17:00:51,928 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginAdminUserCommand internal: false. >2013-11-18 17:00:52,300 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-18 17:00:52,997 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-9) [31129595] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: a44a18ab-6f2e-48d4-ab60-889845f498d3 Type: StoragePool >2013-11-18 17:00:56,080 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-9) [31129595] Correlation ID: 31129595, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center new-nfs. Setting status to Non Responsive. >2013-11-18 17:00:56,481 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) [31129595] hostFromVds::selectedVds - rose07.qa.lab.tlv.redhat.com, spmStatus Free, storage pool new-nfs >2013-11-18 17:00:56,482 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) [31129595] Found spm host d80ddaf1-0131-4088-94cf-d3ad317c10df, host name: monique05, according to spmId: 2. >2013-11-18 17:00:56,523 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) [31129595] SpmStatus on vds d80ddaf1-0131-4088-94cf-d3ad317c10df: SPM >2013-11-18 17:00:56,523 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) [31129595] Using old spm server: monique05, no start needed >2013-11-18 17:00:56,904 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) [31129595] Initialize Irs proxy from vds: monique-vds05.tlv.redhat.com >2013-11-18 17:00:57,354 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-9) [31129595] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host monique05 (Address: monique-vds05.tlv.redhat.com). >2013-11-18 17:00:58,052 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-5-thread-20) [31129595] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = a44a18ab-6f2e-48d4-ab60-889845f498d3, ignoreFailoverLimit = false), log id: 7d6cdd9b >2013-11-18 17:00:58,190 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-11-18 17:00:58,334 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-11-18 17:00:58,541 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-5-thread-20) [31129595] -- executeIrsBrokerCommand: Attempting on storage pool a44a18ab-6f2e-48d4-ab60-889845f498d3 >2013-11-18 17:00:58,582 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-5-thread-20) [31129595] START, HSMGetAllTasksInfoVDSCommand(HostName = monique05, HostId = d80ddaf1-0131-4088-94cf-d3ad317c10df), log id: 474ac3a0 >2013-11-18 17:00:58,745 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-5-thread-20) [31129595] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 474ac3a0 >2013-11-18 17:00:58,745 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-5-thread-20) [31129595] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 7d6cdd9b >2013-11-18 17:00:58,745 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-5-thread-20) [31129595] Discovered no tasks on Storage Pool new-nfs >2013-11-18 17:01:11,408 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (DefaultQuartzScheduler_Worker-1) Failed to query rootDSE for LDAP server LDAP://qa2-tlv.qa.lab.tlv.redhat.com:389 due to connection timeout >2013-11-18 17:01:11,412 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-1) Failed ldap search server LDAP://qa2-tlv.qa.lab.tlv.redhat.com:389 using user vdcadmin@QA.LAB.TLV.REDHAT.COM due to connection timeout. We should try the next server >2013-11-18 17:01:42,819 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-1) Error in finding LDAP servers for domain dc.eng.lab.tlv.redhat.com using user vdcadmin@QA.LAB.TLV.REDHAT.COM >2013-11-18 17:01:52,464 WARN [org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand] (ajp-/127.0.0.1:8702-12) [515c35cc] CanDoAction of action RemoveStoragePool failed. Reasons:VAR__TYPE__STORAGE__POOL,VAR__ACTION__REMOVE,ACTION_TYPE_FAILED_NO_VDS_IN_POOL >2013-11-18 17:01:59,585 INFO [org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand] (ajp-/127.0.0.1:8702-3) [632526a6] Running command: RemoveStoragePoolCommand internal: false. Entities affected : ID: 27564aae-63ee-4435-894d-0a47d391e689 Type: StoragePool >2013-11-18 17:02:01,196 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [632526a6] Correlation ID: 632526a6, Job ID: 8ec2f622-66b5-403b-a943-6ea848c8b0a5, Call Stack: null, Custom Event ID: -1, Message: Data Center DC2 was forcibly removed by admin@internal >2013-11-18 17:02:16,350 INFO [org.ovirt.engine.core.bll.RemoveVdsGroupCommand] (pool-5-thread-50) [56101ce] Running command: RemoveVdsGroupCommand internal: false. Entities affected : ID: feace1c4-bae7-412a-9684-24d20813fedc Type: VdsGroups >2013-11-18 17:02:16,368 ERROR [org.ovirt.engine.core.bll.RemoveVdsGroupCommand] (pool-5-thread-50) [56101ce] Command org.ovirt.engine.core.bll.RemoveVdsGroupCommand throw exception: org.springframework.dao.DataIntegrityViolationException: CallableStatementCallback; SQL [{call deletevdsgroup(?)}]; ERROR: update or delete on table "vds_groups" violates foreign key constraint "fk_vds_groups_vm_pools" on table "vm_pools" > Detail: Key (vds_group_id)=(feace1c4-bae7-412a-9684-24d20813fedc) is still referenced from table "vm_pools". > Where: SQL statement "DELETE FROM vds_groups WHERE vds_group_id = $1 " >PL/pgSQL function "deletevdsgroup" line 7 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: update or delete on table "vds_groups" violates foreign key constraint "fk_vds_groups_vm_pools" on table "vm_pools" > Detail: Key (vds_group_id)=(feace1c4-bae7-412a-9684-24d20813fedc) is still referenced from table "vm_pools". > Where: SQL statement "DELETE FROM vds_groups WHERE vds_group_id = $1 " >PL/pgSQL function "deletevdsgroup" line 7 at SQL statement > at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:245) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.remove(VdsGroupDAODbFacadeImpl.java:118) [dal.jar:] > at org.ovirt.engine.core.bll.RemoveVdsGroupCommand.executeCommand(RemoveVdsGroupCommand.java:21) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1134) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1219) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1895) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:210) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:149) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:118) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1239) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:362) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:175) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:94) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_45] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45] > at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45] >Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "vds_groups" violates foreign key constraint "fk_vds_groups_vm_pools" on table "vm_pools" > Detail: Key (vds_group_id)=(feace1c4-bae7-412a-9684-24d20813fedc) is still referenced from table "vm_pools". > Where: SQL statement "DELETE FROM vds_groups WHERE vds_group_id = $1 " >PL/pgSQL function "deletevdsgroup" line 7 at SQL statement > at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2101) > at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1834) > at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510) > at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:379) > at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:297) > at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404) > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1066) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1014) [spring-jdbc.jar:3.1.1.RELEASE] > ... 25 more > >2013-11-18 17:02:16,419 ERROR [org.ovirt.engine.core.bll.RemoveVdsGroupCommand] (pool-5-thread-50) [56101ce] Transaction rolled-back for command: org.ovirt.engine.core.bll.RemoveVdsGroupCommand. >2013-11-18 17:02:16,431 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-5-thread-50) [56101ce] Correlation ID: 56101ce, Call Stack: null, Custom Event ID: -1, Message: Failed to remove Host cluster (User: admin@internal)
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1031717
: 825729